Loading ToolBench
MCP server for searching and fetching documentation from multiple libraries (LangChain, LlamaIndex, OpenAI)
Single tool 'get_docs' has a reasonable description (194 chars) and clear verb-noun naming. However, the input schema lacks critical constraints: 'library' parameter has no enum despite supporting only 3 known values (langchain, llama-index, openai), inviting hallucinated inputs. No output schema is documented, the tool returns a string but LLMs cannot plan downstream operations without knowing structure. Parameter descriptions are present but lack format/constraint details. No error handling guidance for unsupported libraries, timeouts, or empty results. The tool performs external API calls (Serper, web fetch) with 30-second timeouts but provides no recovery guidance to the LLM.
Search the docs for a given query and library. Supports langchain, llama-index, and openai. Args: query (str): The query to search for. (e.g. "Chroma DB") library (str): The library to search in. (e.g. "langchain") Returns: Text from the docs.
Missing enum constraint on 'library' parameter. Currently accepts any string; should declare enum: ['langchain', 'llama-index', 'openai'] to prevent hallucinated library names.
No output schema documented. Tool returns a string but LLMs cannot infer structure, length limits, or whether results are paginated. Document: returns plain text (max ~50KB), or structured list of {url, excerpt} objects.
Error handling lacks recovery guidance. ValueError for unsupported library, timeout errors, and 'No results found' responses do not tell the LLM what to do next (retry, ask user, try different query).
Parameter descriptions lack format/constraint details. 'query' and 'library' descriptions do not specify: query length limits, special character handling, or that library must be one of 3 specific values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
No result limits or pagination. Tool fetches up to 3 URLs and concatenates all text without limit, risking context window exhaustion. Should cap total output and offer pagination.