Loading ToolBench
A powerful, pluggable vector memory and MCP server for semantic search.
The server registers 8 tools with explicit Zod schemas and descriptions. All tools have non-empty descriptions (range 60-180 chars, within production baseline of 194 chars avg). All parameters are typed via Zod and have descriptions. However, several critical gaps reduce quality: (1) tool names lack action verbs and are inconsistently formatted (add_to_scoped_memory vs search_scoped_memory inconsistency); (2) output schemas are not documented, only text responses are visible, no structured field definitions returned to LLMs; (3) error handling is minimal, no actionable recovery guidance (e.g., 'No matching memories found' does not suggest alternatives); (4) no pagination/limit enforcement in universal memory tools; (5) destructive operations (clear_*, forget_*) lack confirmation or dry-run patterns; (6) parameter descriptions lack format constraints (e.g., sessionId format, what constitutes valid metadata keys).
Store information in session-scoped vector memory
Store information in universal (session-independent) vector memory
Clear all stored memories for a session
Clear all stored memories in universal (session-independent) memory
Remove a specific session-scoped memory item by its ID
Remove a specific memory item by its ID from universal (session-independent) memory
Retrieve information from session-scoped vector memory using similarity or DTS search
Tool names lack action verbs and are overly verbose. All 8 tools use 'add_to_*/search_*/clear_*/forget_*' pattern. Standard patterns would be 'store_memory', 'retrieve_memory', 'delete_memory', 'remove_memory' or simpler 'save', 'find', 'clear', 'forget'. The current naming is inconsistent with Arcade baseline where 90% of A+ tools start with clear action verbs like get, list, create, search, update. LLMs must infer intent from lengthy compound names rather than recognizing immediate patterns.
Output schemas are not documented. All 8 tools return text responses with hardcoded formats (e.g., 'Successfully added to memory (ID: ${item.id})...'). There is no structured return schema definition visible to LLMs. Per pattern:tool and baseline (100% of A+ tools have documented return types), each tool should declare: 'Returns {content: [{type: "text", text: string}]}' or richer structures like '{id: string, content: string, score: number, metadata: object}' for search results. This forces LLMs to parse free-text responses instead of consuming typed fields.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 60 | 2026-07-28+ | v2 |
| 2026-04-07 | F | 26 | - | v1 |
Retrieve information from universal (session-independent) vector memory using similarity or DTS search
Destructive operations lack confirmation or dry-run patterns. clear_scoped_memory and clear_universal_memory permanently delete all memories for a session/globally with no confirmation step. Per pattern:confirmation-request, irreversible operations should support a dry-run or explicit confirmation. An agent could accidentally clear an entire session's memory with no recovery path. This violates the pattern and risks data loss.
Error handling lacks actionable recovery guidance. When search_*_memory returns 'No matching memories found', there is no suggestion (e.g., 'Try a broader query', 'Search universal memory', 'List all memories'). Per pattern:recovery-guide, errors must tell the LLM what to do next. Current error responses are terminal, they give the agent no path forward.
Parameter descriptions lack format/constraint documentation. 'sessionId' is described as 'The session ID to scope this memory to' but does not specify format (e.g., must be UUID, alphanumeric, max length). 'metadata' is 'Optional metadata for the memory item' but does not define valid key names, value types beyond 'any', or max size. Per review:param-validation-rules, format, range, and allowed values must be explicit. This invites LLM hallucination of invalid inputs.
No pagination or result limit enforcement for universal memory tools. search_universal_memory defaults to limit=5 (good), but clear_universal_memory and forget_universal_memory operate globally on potentially millions of items with no batch operations or safety guards. Per pattern:paginated-result and mxe:enforce-result-limits, even destructive operations should have confirmation or pagination thresholds.
Tool descriptions are generic and lack 'when to use' context. E.g., search_scoped_memory says 'Retrieve information from session-scoped vector memory using similarity or DTS search' but does not explain when to use cosine vs euclidean vs dts, or when to choose scoped vs universal search. Per pattern:tool-description baseline (194 chars avg, p90=392), descriptions should guide LLM selection by clarifying use cases and prerequisites.