Loading ToolBench
Event-sourced knowledge graph memory for AI coding agents (MCP server)
Mnemograph has well-structured tool definitions with mostly complete schemas and descriptions. However, there are critical gaps: the add_observations tool has an incomplete input schema (missing the 'observation' field within items), several parameters lack descriptions, and error handling lacks recovery guidance. The server provides canonical naming with action verbs, but parameter documentation is inconsistent. Descriptions are generally good (80-150 chars) but some lack actionable context about when to use each tool vs. alternatives.
Add atomic facts to existing entities. One fact per observation — don't dump paragraphs. Use prefixes: 'Gotcha: ...', 'Warning: ...', 'Status: ...', 'Source: ...'. For relations, use create_relations instead of 'X is related to Y' observations.
Create new entities in the knowledge graph. AUTO-BLOCKS if similar entity exists (>80% match) — returns warning with suggestion. Use create_entities_force to override. Types: concept, decision, project, pattern, question, learning. Naming: use canonical names ('Python' not 'python language'), prefix decisions ('Decision: Use Redis').
Create entities BYPASSING duplicate check. Use when you're certain the entity is distinct despite similar names (e.g., 'React' library vs 'React' conference).
Create relations (edges) between entities. Every entity should have at least one relation. Use specific types: uses, implements, part_of, depends_on, alternative_to, decided_by, affects. Avoid generic 'related_to' when a specific type fits.
Store knowledge atomically — entity + observations + relations in ONE call. PRIMARY TOOL for storing new knowledge. Prevents orphan entities. AUTO-BLOCKS if similar entity exists (>80% match). Use force=True to override.
add_observations input schema is malformed: the 'observations' array items object is missing the 'observation' property definition. The schema shows only 'entityName' required but no 'observation' or 'text' field to store the actual fact being added. This makes the tool non-functional as defined.
Parameter 'relationType' in create_relations has a description but no enum constraint. The description lists valid types (uses, implements, part_of, etc.) but these should be enforced as an enum to prevent LLM hallucination of invalid relation types.
Tool descriptions do not explain when to use create_entities vs remember vs create_entities_force. The rubric requires tool descriptions to state 'WHEN to use it instead of a similar tool'. Users/agents cannot distinguish: should they call remember (atomic) or create_entities separately?
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | D | 59 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 25 | - | v1 |
create_relations description lacks guidance on error recovery. If both entities don't exist, what happens? The tool description should say 'Both from and to entities must exist, or this will fail. Use create_entities first if needed.'
No output schemas documented for any tool. The rubric requires 'Document the output schema. LLMs need to know what fields to expect.' Tools should specify: what does create_entities return on success? On duplicate detection? On error?
Error messages lack actionable recovery guidance. The create_entities description mentions 'AUTO-BLOCKS if similar entity exists (>80% match), returns warning with suggestion', but no tool definition shows the actual error response structure or how the LLM should act on the suggestion.
The 'observations' parameter in add_observations has no 'minItems' or 'maxItems' constraint. Array parameters should be bounded to prevent the LLM from passing extremely large batches that could overwhelm the service.