Loading ToolBench
A FastAPI-based chatbot backend with RAG (Retrieval-Augmented Generation) and agent capabilities for answering questions about MCP using indexed documentation and web search
Two tools with minimal schema documentation. kb_search and tavily both have single-string input parameters with no type constraints, enums, or validation guidance. Descriptions are present but generic (58-72 chars), lacking WHEN to use, prerequisites, or output structure. No documented return schemas. Parameters lack descriptions entirely. No error handling guidance. Tool definitions are inferred from LangChain wrapper calls rather than explicit MCP registration, capping confidence.
Search the indexed MCP docs and return relevant passages.
Tavily Web Search tool for retrieving current and real-time information from the web
Input schemas lack type definitions and parameter descriptions. Both tools accept only 'query' (string) with no validation, format guidance, length limits, or LLM-facing description of what query should contain.
No documented output schemas. LLMs cannot infer what fields to expect from kb_search or tavily results, forcing them to guess at response structure and risking failed downstream tool chains.
Tool descriptions are generic and lack WHEN/WHY guidance. 'Search the indexed MCP docs' does not explain when to prefer kb_search over tavily, what context it covers, or what to do if results are empty.
No error handling or recovery guidance. If a search returns no results or times out, the LLM has no instruction on what to try next (e.g., 'Try tavily for external sources' or 'Rephrase the query').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
Tool definitions are inferred from LangChain create_retriever_tool() and TavilySearchResults wrappers, not explicitly registered in MCP schema. Actual tool registration code is not visible in the provided source.