Loading ToolBench
MCP server for RAG (Retrieval-Augmented Generation) over markdown documentation using vector search
The server defines 3 tools with reasonable verb-based naming (search_knowledge, find_similar_content, update_knowledge) and all have descriptions. Schemas are present with Zod validation and typed parameters. However, descriptions are terse (lacking detail about when to use each tool vs alternatives), parameter descriptions are minimal, output schemas are not formally documented, and error handling is basic. The server lacks guidance on multi-step workflows (e.g., instructing users to call find_similar_content before update_knowledge is mentioned in the tool description but not enforced). Tool composition is reasonable but could better guide agents through dependency relationships.
Check for existing similar content before adding/updating knowledge
Search the markdown knowledge base for relevant information
Add new content or replace existing content in the knowledge base. ALWAYS use find_similar_content FIRST to check for duplicates before calling this.
Output schemas are not documented. Tools return JSON text wrapped in { type: 'text', text: JSON.stringify(...) }, but the expected fields (rank, score, repository, source, content, etc.) are not formally declared for the LLM. Without documented output schemas, agents cannot reliably parse or chain results.
Parameter descriptions are generic and lack actionable constraints. 'The question or search query' does not explain format, length, or expected content. 'Number of results (default: 5)' does not state the allowed range (1 - 100?). LLMs cannot infer what values are valid.
Tool descriptions lack context about when to select each tool and how they relate. 'Search the markdown knowledge base for relevant information' does not explain: Should the agent call find_similar_content first? What distinguishes search_knowledge from find_similar_content? Are they for discovery vs. deduplication?
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 58 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 50 | - | v1 |
Error handling is minimal. Tools catch errors and return { isError: true, content: [{ type: 'text', text: `Error: ${error}` }] }, but do not classify errors (retryable vs. fatal), provide recovery steps, or suggest alternatives. An LLM receiving 'Error: connection refused' has no guidance on what to try next.
The update_knowledge tool is destructive (modifies state) but has no confirmation/dry-run step. The description warns 'ALWAYS use find_similar_content FIRST', but this is a guideline, not enforced. An agent could mistakenly call update_knowledge with replaceFile=true and delete an important file.
Response fields are not consistently named or clearly documented. search_knowledge returns { rank, score, repository, source, content }, while find_similar_content returns { score, file, repo, heading, contentPreview }. The field names differ ('repository' vs 'repo', 'source' vs 'file'), making it harder for agents to chain results.
No explicit permission/scope declarations. Tools read/write to Qdrant and embed via Ollama, but there is no documentation of what permissions an agent needs (e.g., read:knowledge_base, write:knowledge_base). Audit trails are minimal.