Loading ToolBench
MCP server for semantic search using local Qdrant and Ollama (default) with support for OpenAI, Cohere, and Voyage AI
Tool definitions are present with schemas and descriptions, but quality is inconsistent. Most tools have adequate naming (verb_noun pattern) and basic descriptions (avg ~120 chars), but parameter descriptions are sparse and output schemas are undocumented. No error handling guidance, no tool annotations (readOnlyHint/destructiveHint), and critical security concerns around path traversal and input validation. Schemas are well-formed JSON Schema with types, but lack constraints (enums, ranges, patterns). The codebase shows defensive programming (validatePath, secret detection) but tools don't expose these safeguards in their contracts.
Add documents to a collection. Documents will be automatically embedded using the configured embedding provider.
Create a new vector collection in Qdrant. The collection will be configured with the embedding provider's dimensions automatically. Set enableHybrid to true to enable hybrid search combining semantic and keyword search.
Delete a vector collection from Qdrant
Get detailed information about a collection including point count and vector size
Index a codebase for semantic search. Scans files, chunks code using AST parsing, generates embeddings, and stores in Qdrant collection
List all vector collections in Qdrant
No output schemas documented. Tools return data but LLMs cannot predict field names, types, or structure. Breaks downstream tool chaining and forces agents to guess at response format.
Missing tool annotations (readOnlyHint, destructiveHint, idempotentHint). delete_collection and index_codebase (forceReindex=true) are destructive but unmarked. LLMs cannot distinguish safe reads from irreversible writes.
Parameter descriptions are minimal or missing context. 'distance' enum values (Cosine, Euclid, Dot) lack explanation of when to use each. 'scoreThreshold' (0-1) lacks guidance on typical ranges. Descriptions average ~40 chars; rubric baseline is 72 chars.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Search indexed codebase for semantically similar code snippets
Search for semantically similar documents in a collection using vector embeddings
No error handling guidance. Tools can fail (invalid collection name, embedding service down, path traversal attempt) but descriptions do not explain recovery steps or error categories. Agents cannot self-correct.
Path traversal validation exists in code (validatePath) but is not exposed in tool contract. index_codebase accepts arbitrary 'path' parameter with no documented constraints. LLMs cannot know what paths are safe.