Loading ToolBench
Persistent memory system for AI agents with reproducible benchmarks (98.94-99.15% R@5 on LongMemEval, CPU-only, MCP-native)
Mnemos demonstrates solid definition quality with complete input schemas, detailed parameter descriptions, and clear tool naming. All 6 tools follow verb_noun conventions (memory_store, memory_search, etc.). Schemas are comprehensive with proper type definitions, enums, and constraints. However, tool descriptions vary in depth, some are excellent (memory_search at 280+ chars with mode options), while others are adequate but could be more prescriptive about when to use them. Parameter descriptions are consistently present and detailed. Output schemas are not explicitly documented in the source, which is a notable gap for LLM planning. Error handling guidance is minimal, tools lack recovery hints or categorization of error types.
Find-and-replace across memories in one call. Default dry_run=true returns a preview (matched count, affected count, per-memory before/after snippets) without touching the DB. Set dry_run=false to commit. max_affected caps the operation: if more memories would change than the cap, the call errors out without writing anything (prevents runaway rewrites). Re-embeds every modified memory. Namespace-scoped, active-only. Use_regex=true switches from plain substring to Python regex syntax.
Retrieve a single memory by ID. Returns full record including metadata, links, and audit trail.
List all tags in the namespace, optionally filtered by project. Returns tag names and their frequency counts.
Hybrid search: FTS5 + vector + RRF + optional rerank. Auto-widens on thin results. Supports snippet extraction and linked-memory expansion for bandwidth-aware callers.
Store a new memory. Auto-detects duplicates and contradictions. FORMAT GUIDANCE, when to use CML vs prose: Use CML (Compressed Memory Language) for: facts, decisions, contacts, configs, preferences, warnings. CML prefixes: D:(decision) C:(contact) F:(fact) L:(learning) P:(preference) W:(warning) R:(restriction). Symbols: → ↔ ← ∵ ∴ △ ⚠ @ ✓ ✗ ~ ∅ … ; > #N. Dense, one-line-per-fact chains with ; Use plain prose for: runbooks with ordered steps, long-form reference documents, code blocks, creative writing, multi-paragraph narrative. These suffer from CML compression. For prose-format memories, set consolidation_lock=true (via memory_update after store) to prevent the Nyx cycle from cemelifying them later. When in doubt: if the content is primarily a set of atomic facts → CML. If it has essential ordering, structure, or code → prose.
Output schemas not documented. LLMs cannot plan downstream calls or extract required fields (e.g., memory_id from memory_store, linked memory structure from memory_search). This forces agents to guess at response structure.
memory_get and memory_list_tags have minimal descriptions (under 100 chars). They lack context about when to use them vs. memory_search, or what 'full record including metadata, links, and audit trail' actually contains.
No error handling guidance. Tools lack recovery hints (e.g., 'If duplicate detected, call memory_search to review existing memory'). Error responses will not tell LLMs what to do next.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 65 | <=2025-11-25 | v2 |
Update an existing memory. Partial updates allowed: omitted fields are unchanged. Re-embeds if content changes. Supports consolidation_lock toggle.
memory_bulk_rewrite lacks confirmation/dry-run guidance in description. While dry_run=true is the default, the description should explicitly state this is a safe preview pattern and emphasize max_affected safety cap.
memory_search's linked_depth parameter (1-3) lacks guidance on token cost vs. recall tradeoff. LLMs cannot reason about whether depth=2 or depth=3 is appropriate without context.