Loading ToolBench
Local-first project memory system for AI coding agents. Records decisions, discoveries, warnings, mutations, and outcomes. Exposes tools for posting events, querying, generating briefings, and managing event lifecycle.
Engram presents a mixed quality profile. Strengths: all 10 tools have descriptions (average ~150 chars, within baseline), input schemas are comprehensive with typed parameters, and the naming convention is clear (verb-first: post_, resolve_, query, briefing, status, etc.). Weaknesses: descriptions lack WHEN/WHY context that LLMs need for tool selection; parameter descriptions are generic ('optional', 'optional') without format hints; no output schema documentation visible; error handling is minimal (no recovery guidance, no categorization); and the server uses STDIO transport which is not remotely accessible. The consultation tools (start_consultation, continue_consultation) expose external API provider selection as a parameter, which is a design concern but not a credential leak. Query and briefing tools accept loose time formats ('7d', 'yesterday', ISO8601) but don't document parsing rules. Overall, the tools are functionally defined but fall short of production-grade LLM-optimized tooling.
Generate a project briefing — executive summary of active state, warnings, and recent changes.
Continue an ongoing consultation with follow-up questions or clarifications.
List all previous consultations.
Post an event to the Engram project memory. Use this to record: - discovery: something important found about the codebase - decision: a design choice and its rationale - warning: something that should NOT be done, and why - mutation: a file change and what/why it was changed - outcome: whether a previous action worked or failed
Search Engram events using full-text search and/or structured filters.
Reopen a resolved event, setting it back to active. Use when a resolved issue resurfaces. Superseded events cannot be reopened — post a fresh event instead.
Output schemas not documented. No visible specification of what post_event, query, briefing, or status return. LLMs cannot plan downstream tool calls or extract required fields without knowing response structure.
Parameter descriptions lack actionable constraints. 'Filter events since a time (ISO8601, relative like '7d', or 'yesterday')' mixes example values with spec, should be: 'ISO8601 string, relative (e.g. 7d, 30d), or natural (yesterday, today, last_week). Relative formats: Nd for days, Nw for weeks, Nm for months.' Examples buried in description encourage LLM copy-paste.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 66 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 38 | - | v1 |
Mark an active event as resolved so it stops surfacing as live. Use when a warning has been addressed or an open item is done. Only active events can be resolved. Prefer this over posting a second "fixed" event and hoping the reader correlates them.
Start an LLM consultation on a specific topic using external AI providers (OpenAI, Gemini, Anthropic).
Get Engram status: project name, event counts, and server configuration.
Mark an event as replaced by a newer one so briefings show only the live state. Use when a decision reverses or a fact changes: post the new event first, then supersede the old one with the new id. Only active events can be superseded; superseded events cannot be reopened.
status tool has vague description ('Get Engram status: project name, event counts, and server configuration.'). Missing context: WHEN to call it. Should be 'Call this first to verify server is initialized and connected to the project database. Returns project name, total event count by type, and active session info.'
Error handling not visible in source. No guidance for LLM on what to do if post_event fails (content too long? database locked? area inference fails?). Agents need recovery instructions: 'Content exceeds 2000 chars. Summarize or split into linked events (post detail first, then reference via related_ids).' This guidance IS in the code but not exposed in the tool description.
Consultation tools (start_consultation, continue_consultation) allow agents to select external LLM providers as a parameter ('openai', 'gemini', 'anthropic'). This requires agents to know which providers are available and requires API keys to be configured server-side. Design is correct (secrets not exposed), but parameter description lacks: 'Available providers depend on server configuration. If a provider is not available, the tool will return an error.' This prevents LLM confusion when calls fail.
query tool accepts 'format' parameter (compact|json) but briefing and status do not explicitly expose format option in parameter set shown. Inconsistent interface, either all read tools should support format selection or none should. If supported, must be documented.