Loading ToolBench
Versioned memory for AI agents — structured keypaths, conflict detection, and token-efficient memory management. Acts as an adaptive MCP proxy that dynamically proxies all tools, resources, and prompts from the Memstate hosted MCP server.
Memstate MCP has 6 tools with explicit schemas and basic descriptions. Naming follows verb_noun convention well (memstate_set, memstate_get, memstate_delete). However, parameter descriptions are minimal (1-2 phrases), and output schemas are not documented in the provided code. Tools lack error handling guidance, confirmation patterns for destructive operations, and details about return structures. The benchmark code shows integration patterns but not the actual tool response format or error categorization. Descriptions fall short of the 50-200 char LLM-optimized range (most are 40-90 chars). Parameters have types but descriptions are terse. No evidence of rate limiting, input validation guidance, or recovery hints in error cases.
Delete a specific memory entry by keypath
Delete all memory for a project
Retrieve a fact from memory by keypath or browse memory subtree
View the version history of a specific memory keypath
Search memory by natural language query
Store a fact in memory with a structured keypath
Destructive operations (memstate_delete, memstate_delete_project) lack confirmation/dry-run patterns. Descriptions do not warn of irreversible consequences or recommend confirmation steps.
Output schemas are not documented in provided code. Benchmark adapter shows tool calls but no response structure definitions. LLM cannot plan downstream tool calls or extract return data without knowing what fields to expect.
Parameter descriptions are terse (1-2 phrases). Do not explain expected format, constraints, or when to use each parameter. E.g., 'keypath' is described as 'Dot-separated keypath' but lacks guidance on valid characters, max length, or nesting depth.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Tool descriptions do not explain when to use each tool vs. alternatives or any prerequisites. E.g., memstate_search vs memstate_get, when should an LLM choose search over direct retrieval? Ambiguous tool selection increases hallucination.
No error categorization or recovery guidance. If memstate_get returns 'keypath not found', the LLM does not know whether to retry, search instead, or fail gracefully. Errors should guide agent next steps.
memstate_history has no documented return structure. Does it return an array of versions, timestamps, or diffs? Without output schema, the LLM cannot reason about what data it will receive or how to use it.
No evidence of pagination support in memstate_search or memstate_history. If these return unbounded results, large result sets can exhaust context windows. Should accept limit/offset/cursor parameters and return total counts.
Tool descriptions for destructive operations do not explicitly state they are irreversible or suggest confirmation patterns. memstate_delete_project description is only 30 chars and does not convey severity.