Loading ToolBench
Durable memory control plane. MCP server over Postgres and pgvector.
Strong foundation with 12 well-named tools using clear verb_noun patterns (memory_search, memory_write, registry_get, etc.). All tools have descriptions (avg ~180 chars, within 10-1024 baseline). Input schemas are present and typed for all tools. However, output schemas are not documented in the provided source, critical for LLM planning. Parameter descriptions are detailed and include constraints (e.g., RFC 3339 formats, enum values for 'kind'), but some lack explicit type suffixes (e.g., 'id' vs 'fact_id'). Error handling guidance is minimal, tools describe what they do but not how to recover from failures. Security is well-considered (no secrets in params, credential-ref pattern). Composition is strong: tools chain naturally (search → write → history). Missing: output schema documentation, error recovery guidance, and confirmation patterns for destructive ops.
List all aliases in a namespace or across all namespaces the user may read.
Record that two names refer to the same subject. Use it when the user states that two names are the same thing, or when something read the pair out of a fact.
Load context about the current project and user from the memory store. Returns facts tagged as context that are relevant to the current session.
Delete a fact from the memory store by its id. The fact is marked as deleted but the deletion is recorded in history.
Every version of one fact, oldest first, the versions a later correction retired included. Call it when the user asks what was believed before, or when a fact looks wrong and you need to see what it replaced. It takes a memory id from memory_search or memory_write and never a phrase. Versions your credential may not read are counted in withheld rather than shown, so a chain with a withheld count is a partial answer: say so rather than reading it as the whole story. One indexed walk, bounded by a depth cap it reports as depth_capped.
Output schemas not documented. LLMs cannot plan downstream tool calls or extract required fields (e.g., what does memory_search return? Does it include fact_id for memory_history chaining?). Critical for multi-step agent reasoning.
Error handling lacks recovery guidance. Tools describe what they do but not what to do if they fail. E.g., memory_search with no results should suggest 'Try a broader query' or 'Check namespace spelling'. Agents need actionable next steps.
Destructive operations (memory_forget, review_decide with 'delete' verdict) lack confirmation or dry-run patterns. Agents can permanently delete facts without a safety gate. Should support a 'confirm_before_execute' pattern or require explicit confirmation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 69 | 2026-07-28+ | v2 |
Search the memory store for facts matching a query. Returns matching facts with their ids, content, tags, and metadata. Supports filtering by namespace, date range, and similarity threshold.
Record a new fact or update an existing one in the memory store. Takes content, optional tags, and when the fact became true. Returns the id of the written fact.
Retrieve an operational value from the registry by key. Keys are canonical and dotted like 'services.lumberroom.port'. Supports namespace filtering and project-specific lookups.
What a registry key used to hold, newest first. The value it holds now is not in the answer: registry_get is one call away for that, and this is what the key stopped holding. Call it when an operational value changed and the old one still matters, as in "where did the backups live before we moved them". A key reached through a redirect answers here too, and resolved_from names the key the versions came from. Bounded to 20 versions unless you ask for more, 200 at most.
Record an exact operational value under a canonical key: a host, a service endpoint, where a credential lives, a model route, a dataset. Use it when the user states something another tool will act on and a wrong guess would break; use memory_write for anything a person would say in a sentence. Keys are canonical and dotted, 'services.lumberroom.port' rather than 'lumberroom port', and a key that gets rejected is remembered as a redirect so the next caller reaching for the same wrong name lands on the right row instead of inventing a third. Never put a secret in the value. Record a credential-ref naming where the secret lives, and leave the secret where it is.
Make a decision on a review queue item: supersede, merge, keep_both, delete, confirm, apply, or dismiss. The decision is recorded and the item is removed from the queue.
Read the review queue: facts that conflict, are stale, or are proposals waiting for a decision. Filters by source, limit, offset, age, and similarity threshold.
Parameter naming inconsistency: 'id' appears in memory_forget and memory_history without type suffix. Should be 'fact_id' to match memory_search output and prevent LLM confusion. Baseline: 100% of A+ tools use suffixed identifiers.
context_bootstrap has empty input schema ({}). Description is vague ('Load context about the current project'). Should clarify: what context? How is it selected? What fields are returned? Currently scores 65 due to missing schema detail.