Loading ToolBench
MCP Memory Server using LanceDB for vector-based semantic memory storage and retrieval
Memory Server has explicitly visible tool definitions with schemas and descriptions in index.js, but the quality is significantly below production baselines. Tool names lack clear action verbs (memory_add, memory_query use weak patterns), descriptions are minimal (10-40 chars, well below the 194 char baseline), parameter descriptions exist but are sparse, and there is NO documented output schema. The server uses a pseudo-embedding approach (deterministic hash-based vectors instead of real semantic embeddings) which undermines the semantic search promise. Error handling is minimal (raw e.message returns). All 4 tools are directly visible in the ListToolsRequestSchema handler, so no inference penalty applies, but quality gaps are substantial.
Add a memory to the vector store
Clear all memories
Count total memories
Query memories by semantic similarity
All tool descriptions are 18 - 42 characters, far below the 194-char production baseline. Descriptions lack WHEN to use, prerequisites, and consequences. LLMs cannot reliably select tools based on minimal docstrings.
memory_clear (destructive operation) has no confirmation step, dry-run mode, or reversibility guidance. Description does not warn that this is irreversible. Agents will accidentally delete all memories without safeguards.
No output schemas documented for any tool. Agents cannot predict response structure, field types, or ordering. LLMs must guess whether a result is an object, array, or string.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 40 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 35 | - | v1 |
Pseudo-semantic embedding via deterministic hash function (getEmbedding) does not perform true semantic search. Vector distance is meaningless; search results are deterministic only by hash collision, not semantic similarity. Undermines the core value proposition of 'memory query'.
Tool names do not all start with clear action verbs. 'memory_add', 'memory_query', 'memory_count', 'memory_clear' are weak, should be 'create_memory', 'search_memories', 'count_memories', 'delete_all_memories' to match verb_noun convention and improve LLM parsing.
Error handling returns raw e.message in JSON. No recovery guidance, no categorization (retryable vs fatal), no suggestions for next steps. A query failure returns {error: '...'} with no context for LLM to self-correct or retry intelligently.
Parameter memory_type in memory_add defaults to 'context' but no enum constraint or explanation of valid types. memory_query accepts memory_type filter but does not document what filtering logic is applied (exact match? prefix? case-insensitive?)
memory_query limit parameter has no bounds specified (e.g., 1 - 1000). Agent could request 1,000,000 results, exhausting memory and context window. Rubric requires 'Specify minimum and maximum for numeric parameters'.
memory_count implementation fetches all memories with an empty query (search ''), then filters system record. This is inefficient for large stores. No pagination; does not scale.
No idempotency guarantee. Calling memory_add twice with the same content creates two separate records (different UUIDs). For retry safety, agent cannot safely retry memory_add without risking duplicates.