Loading ToolBench
Event log ingestion and semantic search system with MCP server interface for agent memory management
Server exposes 6 tools with partial schema coverage. Tool names follow verb_noun convention (record_event, query_events, semantic_search, report_usage, propose_edge, health_check), which is good. However, descriptions vary widely in quality and actionability. Input schemas are present but lack formal JSON Schema type declarations in the code, parameters are documented in docstrings only. Output schemas are not documented. Error handling is minimal; most tools return generic error dicts without recovery guidance. The semantic_search tool has a complex return structure (trace_id + results) that is documented in the docstring but not in a formal schema. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are present. The payload parameter in record_event is a free-form dict with only a 'summary' requirement stated in text, no schema validation visible.
Probe infrastructure (Ollama, Qdrant, events.db) and return health status.
Propose a typed reasoning edge between two events (supports, contradicts, evolved_into, supersedes, depends_on, related_to, informs, triggered_by). Edges are stored in event_edges table if the schema is applied.
Filtered SQL query over events. Returns list of events.
Append one event to the event log. The canonical write path.
Close the recall-trace feedback loop. Call after answering a question that was informed by semantic_search. Tells the brain which retrieved events actually informed the answer.
Natural-language search over events via vector similarity. Returns {"trace_id": int|None, "results": [{...hit}, ...]}. If schemas/recall_traces.sql is applied, a trace row is written and trace_id is returned — call report_usage() afterwards to close the loop on which results you actually used. If the schema isn't there, trace_id is null and behavior matches the pre-trace version.
No formal JSON Schema definitions in code. Parameters documented only in docstrings; LLMs cannot parse constraints from text alone. Enums (e.g., relation types in propose_edge: 'supports', 'contradicts', etc.) are listed as strings in descriptions, not as schema enums.
Output schemas not documented. semantic_search returns {trace_id, results} but no schema for result objects. query_events returns list[dict] with no field documentation. Agents cannot plan downstream calls without knowing response structure.
Error handling is generic. record_event and semantic_search return {status: 'error', error: stderr} with no recovery guidance. LLMs cannot determine if error is retryable, user-fixable, or fatal.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
No tool annotations. Tools like record_event and propose_edge (WRITE operations) and query_events (READ-ONLY) lack readOnlyHint/destructiveHint annotations. Agents cannot distinguish safe from risky operations without reading descriptions.
Payload parameter in record_event is a free-form dict. Only constraint is 'MUST include a summary field' stated in text. No schema validation, no enum for type field, no format constraints on ts (ISO 8601). Invites invalid input.