Loading ToolBench
MCP server for memory management using Qdrant vector database
Memory management server with 9 tools covering CRUD, semantic search, graph operations, and administration. Tool definitions have reasonable structure with schemas and descriptions present, but several critical gaps reduce quality: (1) Output schemas are not documented, LLMs cannot plan what fields to expect in responses; (2) error handling lacks recovery guidance; (3) some descriptions are generic or missing actionable context; (4) parameter relationships underdocumented (e.g., metadata_filter format, history_for/history_limit dependency); (5) no explicit idempotency guarantees or confirmation patterns for destructive operations. The server demonstrates solid engineering (caching, embedding, vector DB integration) but tool definitions prioritize implementation over LLM usability. Naming is clear (verb_noun pattern) and schemas are mostly complete with proper types, but lack the explicit field documentation and error guidance expected of A-grade tools.
Administrative operations: op=list returns collection names, op=delete deletes an entire collection, op=export exports to markdown, op=import imports from markdown, op=stats returns cache and collection statistics.
Load or update the project's working context: product context, active context and system patterns. Call at session start with no update fields; pass product_context or active_context to patch them.
Store one or more memory entries in a single call. Use memory_type to classify each: decisionLog for a choice plus rationale, progress for completed or blocked work, systemPatterns for a reusable rule, productContext/activeContext for project state, customData for verbatim documents. Use metadata for filterable fields such as status, priority or dataType.
Permanently delete memory entries by id.
Knowledge graph over memory entries. op=link creates typed edges between entries, op=neighbors walks the graph outwards from an entry up to the given depth, op=unlink deletes edges by their own ids.
No output schemas documented for any tool. LLMs cannot plan downstream tool calls or extract required fields (e.g., after memory_create, what ID is returned? Does it match the input 'id' or a generated one?). This forces agents to infer structure from response content, increasing hallucination risk.
Destructive operations (memory_delete, memory_admin with op=delete) lack confirmation pattern or dry-run support. Agents can delete all memories without explicit user approval, risking data loss.
Error responses provide no recovery guidance. Tool descriptions do not explain what to do if a call fails (e.g., 'ID not found, available IDs are X, Y, Z' or 'retry with a different memory_type'). Agents cannot self-correct.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 58 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 47 | - | v1 |
Retrieve memory entries. Each query with query_text runs semantic search; without it, lists the most recent entries of the given type. metadata_filter narrows on fields stored via memory_create, with an array value matching any of its elements.
Semantic search across all memory types. Search results are ranked by relevance; use memory_type to restrict to one type or none for cross-type search.
Generate a summary of the provided text using an LLM. Useful for condensing long passages before storing as memory.
Update existing memory entries by id. Content is re-embedded when supplied; metadata is shallow-merged into what is already stored.
Vague tool names and overloaded operations. 'memory_admin' combines list, delete, export, import, and stats into one tool, agents must understand op enum to invoke correctly. 'memory_context' name does not indicate load vs. update behavior. Should split into canonical single-purpose tools (list_collections, delete_collection, export_memory, import_memory, get_stats, load_context, update_context).
Parameter relationships underdocumented. (1) memory_read: metadata_filter format not shown, is {status: 'pending'} or {status: ['pending']}? Schema shows array values but description does not. (2) memory_context: history_for and history_limit are coupled but described independently. (3) memory_graph: direction parameter default 'both' may be expensive on large graphs; no guidance on when to restrict to 'outgoing' or 'incoming'. (4) memory_admin: markdown parameter format not specified for import.
Batch operations lack per-item error reporting. If memory_create receives 10 items and one fails (duplicate ID, invalid memory_type), does the tool return partial success or full failure? Current schema/description does not clarify. Pattern requires per-item success/failure for batch ops.
No idempotency guarantees documented. memory_create with explicit 'id' appears to overwrite, but is this behavior guaranteed? memory_update with only metadata changes: does it re-embed? memory_summarize: can it be called twice safely on same input? Agents need explicit idempotency statements to safely retry.
Result limits not enforced or documented. memory_read defaults limit=5, memory_search defaults limit=10, but memory_graph neighbors with depth>1 on a large graph could return thousands of entries and blow context window. No guidance on pagination or max result caps.
No clear tool composition chain. Tools are mostly independent, no guidance on typical workflows (e.g., 'First call memory_context to load project state, then memory_search to find related decisions, then memory_update to refine'). Users must infer sequencing.
memory_summarize integrates external LLM (OpenAI, Gemini) but implementation details not exposed. Which provider is active? How are costs tracked? What if LLM API fails? Description does not address.