Loading ToolBench
MCP server for RAG (Retrieval-Augmented Generation) with classical and OpenAPI-based document indexing and querying capabilities
The server has 4 tools with explicit definitions visible in source code. All tools have descriptions and input schemas with typed parameters. However, multiple issues reduce quality: (1) Output schema for classical_query is only partially documented (McpClassicalRagResponse structure is not fully visible in source); (2) Error handling is minimal, only classical_query and read_file show try/catch with ToolError conversion, but error messages lack recovery guidance; (3) Parameter validation exists (_validate_prefix) but is inconsistent across tools; (4) Tool naming follows verb_noun convention well (list_*, read_*), but descriptions lack depth on when to use each tool vs alternatives. Average tool score: 62/100.
Query the classical RAG knowledge base.
List files in MinIO storage under a given prefix.
List folders in MinIO storage under a given prefix.
Read and extract text content from a file stored in MinIO. Supports 91 file formats including PDF, Office documents, images, HTML, etc. Uses Kreuzberg for document intelligence extraction.
Output schemas are not fully documented. classical_query returns McpClassicalRagResponse, list_files returns list[FileInfoResponse], read_file returns FileContentResponse, but LLMs cannot see these definitions in the tool registration code. Response shape is opaque.
Error messages lack recovery guidance. read_file catches exceptions and returns generic 'Failed to read file: {file_path}'. LLM does not know whether to retry, check the path, or try a different tool. Compare to pattern: 'File not found. Verify path with list_files() or check prefix.'
Tool descriptions are too brief and lack disambiguation. classical_query, list_files, and list_folders form a related set, but descriptions do not explain when to call each or in what order. No guidance on 'call list_files first to explore structure, then read_file to extract content'.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | B | 73 | <=2025-11-25 | v2 |
Parameter validation is inconsistent. list_files and list_folders validate prefix via _validate_prefix, but classical_query accepts working_dir with no visible validation. LLM could pass arbitrary paths, leading to silent failures or security issues.
No pagination support visible. list_files and list_folders may return large result sets, but no limit parameter or cursor support is documented. LLM cannot cap results or iterate safely.
No tool annotations (readOnlyHint, idempotentHint, destructiveHint) present in the code. All four tools are read-only, which should be signaled via toolAnnotations for agent safety and optimization.