Loading ToolBench
MCP server for searching documentation across multiple libraries (LangChain, LLamaIndex, OpenAI) and performing basic arithmetic operations
Server has 3 tools but suffers from critical definition gaps. get_docs is registered twice (duplicate), creating confusion. Descriptions exist but are generic and lack LLM-optimization guidance. Input schemas are present with basic types and descriptions, but lack enums for the 'library' parameter (should constrain to langchain|openai|llama-index). Output schemas are completely undocumented, LLMs cannot infer what get_docs returns or how to chain results. The add tool has a trivial description (5 words) and no output schema. No error handling guidance, no pagination support for potentially large doc results, and no indication of timeout behavior despite external API calls. Parameter descriptions lack format/constraint details (e.g., query length limits, library enum values).
Add two numbers
Search the latest docs for a given query and library. Supports langchain, openai, and llama-index. Args: query: The query to search for (e.g. "Chroma DB") library: The library to search in (e.g. "langchain") Returns: Text from the docs
Search the latest docs for a given query and library. Supports langchain, openai, and llama-index. Args: query: The query to search for (e.g. "Chroma DB") library: The library to search in (e.g. "langchain") Returns: Text from the docs
Duplicate tool registration: get_docs appears in both main.py and docs.py with identical signatures. This creates ambiguity for clients and wastes definition space. Consolidate into a single canonical implementation.
Missing output schema documentation. get_docs returns text (string) but LLMs cannot infer structure, length, or format. Document: 'Returns a string containing concatenated documentation text from up to 2 sources. May be empty if no results found.'
Parameter 'library' lacks enum constraint. Currently accepts free-form string; should declare enum: ["langchain", "openai", "llama-index"]. This prevents LLM hallucination of invalid library names.
add tool description is 5 words ('Add two numbers'). Rubric requires 10 - 1024 chars and LLM-optimized context. Expand to: 'Add two integers and return their sum. Useful for arithmetic calculations in multi-step workflows.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
No error handling guidance. get_docs calls external APIs (Serper, web fetch) with 30s timeouts but returns bare 'Timeout error' string. Should return structured error: '{"error": "timeout", "message": "Documentation fetch exceeded 30s. Try a more specific query.", "retryable": true}'