Loading ToolBench
Dynamic RAG MCP server for AI agents - reduces hallucinations and repetitive errors
The Agent RAG MCP server exhibits significant definition quality gaps across all four tools. While tool names follow verb-first conventions (get_, ask_), descriptions are present but superficial (avg 80 chars vs 194 baseline), parameter descriptions are almost entirely missing, and schemas lack depth. Most critically, the schema visibility in the provided code is limited to test files (tests/verify_all_tools.py), not source definitions, which raises uncertainty about whether schemas are properly registered in the fastmcp server code itself. The server shows promise for RAG-specific use cases but falls well short of production quality for agentic tool patterns.
Ask for code patterns and implementations using dynamic RAG with TOON format support
Ask a question about the project documentation using RAG
Get the schema template for requests in TOON format
Retrieve information about the current document store configuration
Tool schemas not visible in source code; only inferred from test file. Cannot verify proper JSON Schema registration in fastmcp server definition.
Parameter descriptions almost entirely missing. 'question' param in ask_project_document has a description, but 'request_data' in ask_code_pattern lacks format guidance (TOON vs JSON ambiguity not documented in schema).
Tool descriptions are too brief (avg 80 chars vs 194 baseline). Missing WHEN to use, WHEN NOT to use, and prerequisite information. E.g., 'Get information about the configured document store' does not explain what 'configured' means or what fields are returned.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 6 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 38 | - | v1 |
Output schemas not documented. No description of what get_store_info returns, what fields ask_project_document yields, or structure of ask_code_pattern response. This forces LLMs to guess at downstream tool chaining.
ask_code_pattern accepts 'request_data' as a free-form string that can be TOON or JSON. No enum, no format constraint, no validation guidance. Test code shows defensive double-decoding, suggesting the tool handles malformed input silently rather than rejecting it clearly.
No error handling documentation. What does ask_project_document return if the vector store is empty? What if ask_code_pattern receives invalid TOON? No recovery guidance for LLMs.
get_request_schema_template returns a 'TOON format' schema but does not document the structure or valid keys. LLMs cannot know how to construct a proper request without examples or field descriptions.
No pagination, limits, or result capping documented for ask_project_document or ask_code_pattern. If the RAG returns thousands of documents, the response could exhaust context windows.