Loading ToolBench
A chatbot backend that uses RAG (Retrieval-Augmented Generation) with MongoDB Atlas vector search to answer questions about Aayushmaan Hooda based on a JSON profile. Includes a React frontend for chat interface.
aayushBot is a simple chatbot exposé with 4 tools. While tool definitions are visible in the FastAPI code (app.py), there are critical gaps in descriptions, parameter documentation, and output schema specifications. Three tools (health, debug_python, debug_mongo) are minimal utility functions with acceptable descriptions but no input schema documentation. The primary tool (chat) has a bare-minimum description and a single parameter with a basic type but no output schema defined. No tools declare return types, error handling guidance, or recovery paths. The server uses HTTP transport (FastAPI), which is current, but tool definitions lack the rigor expected of production-grade agents. Descriptions range from 56 - 89 characters (within baseline 34 - 392), but lack the WHAT/WHEN/WHY context that guides LLM tool selection. Overall, this is a functional proof-of-concept, not a production-ready agent toolkit.
Send a question to the RAG chain to retrieve context from the knowledge base and generate an answer using GPT-4o-mini
Check MongoDB Atlas connection status and retrieve server version information
Retrieve debug information about Python version, OpenSSL version, and certifi path
Check if the API server is running and healthy
No output schemas documented for any tool. LLMs cannot infer return structure, field names, or types. This forces agents to make assumptions and wastes tokens on error recovery when response format is unexpected.
'chat' tool description is incomplete and underdescriptive (89 chars). Does not state what the tool returns, how it handles missing knowledge, or what the 'answer' field contains (is it plain text, markdown, structured data?). No guidance on error recovery.
The 'question' parameter in 'chat' has no description at all beyond its type. LLMs cannot infer context about format (free-form text?), length limits, expected language, or how ambiguous queries are handled.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 47 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
No error handling guidance in tool definitions. The backend catches exceptions and returns them inline in the response (see rag.py line 'return ChatResponse(answer=f"Error: {e}")'), but the tool schema does not document error cases, recovery steps, or how to distinguish success from failure.
Utility tools (health, debug_python, debug_mongo) lack clarity on use cases. LLMs will not naturally choose these unless prompted. Descriptions should explain WHEN to call them (e.g. 'Call if the user reports connection issues' for debug_mongo).
Tool names do not follow verb_noun convention. 'health' should be 'check_health', 'debug_python' should be 'get_python_debug_info', 'debug_mongo' should be 'check_mongo_connection'. Current names are ambiguous (is 'health' a status object or a check action?).