Loading ToolBench
A dual-path memory system for proactive agents: facts and procedures in a local graph, exposed as a CLI, MCP tools and skills, no API key.
Memoose has 23 well-intentioned tools with mostly complete schemas and descriptions. Naming is verb-forward (describe_, add_, remember, recall, etc.), following pattern:tool conventions. Descriptions are substantive (100 - 300 chars typical), explaining WHAT and WHEN. However, several critical gaps reduce confidence: (1) Parameter descriptions are present but often generic ('Optional dataset name' repeated 23 times); (2) No output schemas documented, LLMs cannot predict response structure for chaining; (3) Error handling is minimal, guard() catches OntologyError and ValueError but returns bare {'error': type, 'message'} without recovery guidance; (4) No enums for constrained inputs (e.g., recall mode accepts 'hybrid|facts|neighbourhood|lexical|summaries|temporal|rules|session' but is free-form string); (5) Destructive tools (forget, mark_contradiction) lack confirmation/dry-run patterns. Strengths: tool names are clear and action-oriented; descriptions explain domain concepts (Procedures, Transitions, hotspots); parameter types are declared in schema. Baseline: avg tool description ~180 chars (within 10 - 1024 range), avg params ~5 per tool (within p10 - p90). Most tools score 55 - 70 individually.
Add an entity type when nothing in describe_ontology fits. PascalCase name; optional parent (collapses onto a basic type) and aliases.
Facts around given entities or relations, grouped by subject, with hotspots where one subject holds several values for one relation. A Procedure's branches are not hotspots. Judge them with the memoose-upkeep skill.
Add relations between existing entities proposed by memify_candidates(cross_connect).
Declare relation names that hold a single current value per subject (e.g. current_owner, deployed_in). New assertions then supersede older ones automatically; nothing is deleted.
Entity types, functional relations, imported ontologies, and store stats for a dataset. Call once per session before remember.
Decline a maintenance candidate by its key (from maintain or contradiction_candidates) with the reason, so it is not proposed again and the reason is shown next time.
No output schemas documented. LLMs cannot predict response structure (fields, types, nesting) for downstream tool chaining. E.g., recall() returns 'ranked raw entities, facts, chunks' but no schema is visible, agents cannot extract entity IDs for follow-up calls.
Destructive/irreversible tools (forget, mark_contradiction, supersede) lack confirmation or dry-run patterns. Agents can permanently delete entities or mark contradictions without user approval, risking data loss.
Error handling is minimal. guard() catches exceptions but returns bare {'error': type, 'message'} without recovery guidance. LLMs cannot determine if errors are retryable, user-fixable, or fatal. E.g., OntologyError should suggest 'Call describe_ontology() first' or 'Add the missing entity type with add_entity_type()'.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | 2026-07-28+ | v2 |
Forget an entity (and its relations) by exact name, a relation by id, a session, or a whole dataset. Prefer supersede for facts that were true once. Confirm with the user first.
What memory says comes next from a Procedure: its outgoing Transitions two hops out, grouped by hop, each with condition, advice, pitfall and how past sessions that took it ended. Raw and local; you decide. Superseded transitions are excluded; declined changes to these transitions are listed under dismissed.
Audit trail: who did what to an entity or relation, when, and why. Useful for understanding how memory evolved and who made changes.
Import an OWL/RDF/Turtle ontology (text). Classes become entity types with parents collapsing onto basic types; labels become aliases.
Maintenance candidates: hotspots (contradictions), open contradictions, consolidations (merge entities), cross-connects, stale summaries, and undistilled sessions. Propose actions; you judge with dismiss, merge_entities, cross_connect, set_bucket_summary, publish_lessons.
Record that two stored facts cannot both be true. Adds a contradicts edge between their subjects. Use supersede instead when the newer fact simply replaces the older in time.
Merge two entities that denote the same thing. Relations and chunks move to `keep`; `drop` is removed and recorded as an alias.
Distill a session into lessons: facts that hold across turns, and rules (Transitions) that worked or failed. Stores them as Lesson entities and updates Transition outcomes.
Recall memory. Routes the query to a mode unless given: hybrid (default), facts, neighbourhood, lexical (quoted phrase), summaries, temporal, rules, session. Searches the project dataset then the user dataset. Returns ranked raw entities, facts (with evidence, validity, superseded/contested flags), chunks; you synthesise.
Store memory as a typed graph: entities (name, type, description) and relations (source --name--> target, one-sentence description, evidence, optional valid_from/valid_to). A relation between two Procedures is a Transition and may carry condition, advice and pitfall. Pass source_text and a summary so recall can find it lexically. Validates against the ontology, merges entities by name, supersedes functional relations, records provenance, and warns about hotspots that may be contradictions.
Record a turn in the session: a Procedure (where you are) and optional outcome (what happened). Returns guidance for that Procedure.
End a session with an outcome (succeeded, failed, abandoned). Transitions taken learn from it.
Begin a session: a named context for work, with a timeline of turns and lessons learned. Returns session_id for remember() and session_add_turn().
The turns taken in a session, with their outcomes and the Transitions that were active.
Write the summary for a global-context bucket returned by memify_candidates(stale_summaries).
Mark an older fact as superseded by a newer one. The old fact stays in history and leaves default recall.
Render the graph as an interactive HTML visualization (entities, relations, types, hotspots). Opens in a browser.
Constrained inputs lack enums. recall(mode=...) accepts 8 modes but is a free-form string, LLMs may hallucinate invalid modes. Should declare mode as enum: ['hybrid', 'facts', 'neighbourhood', 'lexical', 'summaries', 'temporal', 'rules', 'session'].
Parameter descriptions are generic and repetitive. 'Optional dataset name' appears 23 times without explaining when/why to override the default. Descriptions should clarify: 'If omitted, uses the current project dataset. Pass a different name to query a shared or archived dataset.'