Loading ToolBench
MCP server for querying project documentation using RAG (Retrieval-Augmented Generation)
Single tool 'doc_query' has a clear verb-noun name and basic schema, but suffers from critical gaps: description is only 27 chars (well below 50-200 char baseline for LLM optimization), parameter descriptions are present but generic, no output schema is documented, and error handling provides no recovery guidance. The tool accepts api_key as a parameter (security anti-pattern), lacks enum constraints, and returns a Pydantic model that is serialized to dict without clear field documentation for the LLM. No pagination, no result limits, no idempotency hints.
Query project documentation.
Tool description is only 27 characters ('Query project documentation.'), far below the 50-200 char baseline for LLM-optimized descriptions. LLMs cannot determine when/why to select this tool or what it returns.
api_key exposed as a tool parameter. Credentials must never appear in parameters, they are logged in traces and prompt history. Use server-side secret injection via environment variables.
Output schema not documented. The tool returns a ContextResponse object serialized to dict, but the LLM has no visibility into the fields (text, error, context_id). Without documented output schema, LLMs cannot plan downstream calls or extract data reliably.
No error recovery guidance. Error responses return bare error strings ('Authentication failed', 'processing_failed') without actionable next steps. LLM cannot determine if it should retry, ask the user, or abandon the task.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 37 | <=2025-11-25 | v2 |
No pagination or result limits. RAG queries can return unbounded results (code_snippets, doc_links arrays). Large responses blow context windows and degrade LLM reasoning. Tool description does not state result limits.