Loading ToolBench
MCP server for enabling RAG (Retrieval-Augmented Generation) with document indexing, embedding, and vector search capabilities
The server defines 5 tools with Zod schemas and descriptions, but exhibits significant gaps in parameter documentation, output schema clarity, and error handling guidance. Tool naming follows verb_noun convention ('embedding_documents', 'query_documents', 'remove_document', 'remove_all_documents', 'list_documents'), which is a positive signal. However, parameter descriptions are sparse, output schemas are not formally documented, and error responses lack recovery guidance. The async fire-and-forget pattern in embedding_documents (returning immediately without awaiting completion) creates a poor user experience. Tools operate on a well-scoped domain (RAG operations), but composition and error categorization are weak.
Add documents from directory path or file path for RAG embedding and store to DB. Supported file types: .json, .jsonl, .txt, .md, .csv
List all document paths in the index
Query indexed documents using RAG
Remove all documents from the index
Remove a specific document from the index by file path
Output schemas not formally documented. Tool responses return {content: [{type: 'text', text: string}]} but this structure is not declared in tool definitions or in separate documentation. LLMs cannot plan downstream operations without knowing what fields to expect. Baseline for A+ tools: 100% have documented return types.
Parameter descriptions are minimal or missing. 'path' parameter in embedding_documents is described as 'Path containing .json, .jsonl, .txt, .md, .csv files to index' but lacks guidance on whether it accepts directories, files, or both; whether paths are relative or absolute; or what happens if the path doesn't exist. 'k' in query_documents has a description but no bounds (is it 1 - 1000? 1 - 100?). Baseline: 100% of A+ tool params have descriptions; 94% of descriptions provide format/range/dependency info.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 37 | - | v1 |
Error responses lack recovery guidance and error categorization. When embedding_documents fails, it returns 'Error indexing documents: [error message]' with no indication of whether the error is retryable, user-fixable, or fatal. No suggestion to check VECTOR_STORE_PATH, verify file permissions, or retry. Pattern baseline: error responses must tell the LLM what to do next.
Destructive operations (remove_document, remove_all_documents) lack confirmation or dry-run support. remove_all_documents includes a 'confirm' boolean parameter, but the tool itself does not validate that confirm === true before executing; it checks existence but could silently accept confirm=false and delete anyway if the implementation is weak. No recovery path is offered (e.g., no undo_removal or list_deleted_documents). Pattern baseline: irreversible operations should support dry-run or explicit confirmation.
embedding_documents returns immediately without awaiting indexing completion. It fires .then().catch() in a Promise that is not awaited, then returns 'Running indexed documents... use resource rag://embedding/status to see if it is completed'. This forces the agent to poll embedding-status resource repeatedly and provides no progress feedback. Baseline: tools should return only when the operation is complete, or support explicit progress tracking (e.g., via progress events or a job ID to poll).
list_documents has an empty input schema ({}). While this is valid for read-only discovery tools, the tool description does not explain what it returns (list of file paths? metadata? counts?). No output schema is documented. An LLM cannot determine whether to expect [{path, size, chunks}] or just [path].
query_documents accepts a 'k' parameter (number of chunks) but does not document the retrieval strategy, ranking, or format of returned chunks. Are results ranked by relevance score? Do they include source document paths? Timestamps? The description says 'Query indexed documents using RAG' but does not specify whether this is semantic search, keyword search, or hybrid. LLMs cannot reason about result quality without knowing the retrieval method.
No input validation or constraint documentation for numeric parameters. 'k' in query_documents is a number with no explicit bounds. An LLM could pass k=999999, causing a timeout or memory exhaustion. Baseline: numeric parameters must declare min/max (e.g., k: 1 - 100).