Loading ToolBench
Claude's persistent memory context system - an MCP server for storing and retrieving memories across sessions with semantic search, vault integration, and federated search capabilities
Claude Innit provides 14 tools with comprehensive descriptions and structured schemas. Tool naming follows verb_noun conventions consistently (get_context, search, remember, forget, save_session, list_memories, vault_index, vault_search, vault_related, vault_stats, vault_tag, federated_search, admin_sync, admin_check_integrity). All tool descriptions exceed the 10-char minimum and most range 150-250 chars, matching production baselines. Input schemas are present for all tools with proper type definitions and required field declarations. However, several critical gaps reduce the score: (1) Output schemas are completely undocumented, no tool describes what it returns, forcing LLMs to reason about response structure blindly; (2) Parameter descriptions lack constraint detail (format, ranges, allowed values); (3) Error handling is absent, tools show no recovery guidance or error classification; (4) Some design issues like 'remember' requiring a conditional 'project' parameter without explicit mutual-exclusivity documentation.
Operator tool: Check database health and repair issues. Not needed in normal sessions — only call when experiencing search or sync failures. Checks FTS index sync, orphaned embeddings, and SQLite integrity.
Operator tool: Re-sync markdown files to database. Not needed in normal sessions — only call if memories are out of sync after manual file edits.
Search across vault, book-library, and session memory with unified results ranked by Reciprocal Rank Fusion.
Permanently delete a memory. Requires the memory_id — use list_memories first to discover IDs. Use when an improvement has been implemented, a fact is no longer true, or a memory is outdated.
Load all persistent memory for this session. Call this once at the start of every session before doing any other work. Pass the project name to filter to relevant project and session memories.
List stored memories with their IDs, previews, and categories. Call this to discover memory IDs before using forget(), or to audit what is stored. Filter by category ('personal', 'project', 'session') or project name.
NO OUTPUT SCHEMAS DOCUMENTED. All 14 tools lack documented return types. LLMs cannot determine what fields to expect, requiring them to infer response structure from context or trial-and-error. This violates pattern:tool and mxe:strip-api-responses.
PAGINATION NOT DECLARED FOR LIST OPERATIONS. list_memories and vault_search return lists but lack limit defaults, max limits, or next_cursor/offset mechanisms. Rubric baseline: 'Tools returning lists should accept page/offset and limit parameters and return a total count or next_cursor.' vault_search defaults to 10, federated_search to 30, but list_memories has no stated limit.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 58 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 57 | - | v1 |
Store new information persistently across sessions. Use for facts, decisions, preferences, or project state that should be recalled in future sessions. Choose category: 'personal' for user preferences/identity, 'project' for per-project state, 'session' for session summaries.
Save a summary of this session for future recall. Call once at the end of a working session — not after each sub-task. Include what was completed, what to do next, and any key decisions made.
Find stored memories by keyword or concept. Call this when you need information from past sessions that is not in the current get_context result. Short keywords (1-3 words) use exact text match. Longer phrases use semantic/concept search.
Index vault markdown files into the search database. Call this to update the vault search index after file changes. Skips unchanged files by default (hash-based). Use force=true to reindex everything.
Find vault files semantically related to a given file by heading/chunk. Returns related files ranked by similarity.
Search vault files by text or semantic similarity. Use text search for keywords, semantic for concept-based queries.
Get statistics about the vault index (file count, total content size, last updated).
Tag or untag vault files with custom labels for organization and filtering.
NO ERROR HANDLING OR RECOVERY GUIDANCE. Tools lack error classification (retryable vs fatal) and recovery hints. Rubric requires: 'Error responses must tell the LLM what to do next.' E.g., forget() should document 'memory_id not found, call list_memories() to discover valid IDs' or 'permission denied, you cannot delete memories created by another user.'
PARAMETER CONSTRAINTS UNDERSPECIFIED. Parameters lack format, range, and valid-value documentation. E.g., 'vault_root' in vault_index has no constraint (must be absolute path? relative? must exist?). 'query' in search and vault_search should specify max length. 'limit' parameters should state min/max (e.g., 1 - 100). Rubric: 'Describe the expected format, range, and allowed values directly in the parameter description.'
CONDITIONAL PARAMETER DEPENDENCIES NOT ENFORCED IN SCHEMA. remember() requires 'project' only when category='project', but the schema does not express this via oneOf/if-then. Current schema allows {"content": "X", "category": "personal", "project": "Y"} which is ambiguous. Save_session() allows project to be omitted but its utility depends on it. Rubric: 'When parameters are mutually exclusive or dependent, state this in descriptions. LLMs will pass both unless told not to, causing ambiguous or failing calls.'
VAGUE VERB NAMES REDUCE CLARITY. admin_sync (sync from what to what?), admin_check_integrity (check and do what?), vault_tag (add or remove tags?). While descriptions clarify intent, names alone should convey action. Rubric baseline: '90% of A+ tools start with an action verb' and 'The tool name alone should convey what happens.' Consider renaming: admin_sync → admin_resync_database, admin_check_integrity → admin_repair_database, vault_tag → vault_update_tags.
MISSING RESPONSE CHAINING FIELDS. If federated_search returns matching vault files, does it include file_path so the LLM can call vault_related()? If search() returns a memory_id, can it be passed directly to forget()? Rubric: 'If the next likely action requires a team_id, channel_id, and message_id, the current response must return all three.' Tools should return IDs needed for downstream operations.
ADMIN TOOLS LACK PERMISSION GATES DOCUMENTATION. admin_sync and admin_check_integrity are operator-only but descriptions do not state required permissions or gate mechanisms. Rubric: 'Each tool should declare what permissions it requires (e.g. 'read:email', 'write:calendar'). This enables least-privilege agent configurations and clear audit trails.'
DESTRUCTIVE OPERATIONS LACK CONFIRMATION PATTERN. forget() deletes permanently but has no dry-run or confirmation step. Rubric: 'Irreversible operations (delete, send, publish) should support a dry-run or confirmation step. Agents make mistakes, a confirm_before_execute pattern prevents catastrophic errors.' Consider adding forget(memory_id, confirm=false) where confirm=false returns 'Deletion preview' and requires a second call with confirm=true.