Loading ToolBench
Server has 12 tools with explicit schemas and descriptions. Naming follows verb_noun pattern (remember, recall, update, forget, etc.). Descriptions are present but brief (10-80 chars), below the 50-200 char LLM-optimized baseline. Parameter schemas are well-typed with Pydantic Field constraints (ge, le, validation_alias), but descriptions for enum-like types (MemoryKind, EntityKind, MemoryRecurrence, MemoryAttribute, CollectorProfile, SituationState, MuteScope) are not visible in the source, only type names appear. Output schemas are JSON responses but not formally documented in tool definitions. Error handling is absent from visible code. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk classifications (WRITE, READ_ONLY, REVERSIBLE).
Record that the user handled one delivered situation.
Confirm that the MCP host received a proactive-check result.
Soft-archive a memory item by id.
Return one situation with its isolated evidence.
Return active entities as a JSON object.
List stored situations, optionally filtered to one state.
Mute one delivered situation, or its whole type in one transaction.
Enum type descriptions missing. MemoryKind, EntityKind, MemoryRecurrence, MemoryAttribute, CollectorProfile, SituationState, MuteScope are referenced but their valid values and meanings are not documented in tool descriptions. LLMs cannot infer valid enum options without explicit documentation.
Tool descriptions are too brief (10-80 chars vs. 50-200 char baseline). Descriptions lack context on WHEN to use each tool, dependencies, and what the output contains. E.g., 'Store a memory item from an agent conversation' does not explain memory kinds, recurrence patterns, or lead_days semantics.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
Evaluate, lease what attention policy allows, report what is held.
Return matching active memory items as a JSON object.
Store a memory item from an agent conversation.
Hold one delivered situation until an aware future instant.
Replace a memory item's mutable values while retaining its id.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite explicit risk classifications in metadata (WRITE, READ_ONLY, REVERSIBLE). Tools like 'forget' (REVERSIBLE) and 'remember' (WRITE) should declare their side effects to the MCP client.
Output schemas not formally documented. Tools return JSON strings (model_dump_json()) but the structure of MemoryItemResponse, RecallResponse, EntityResponse, etc. is not visible in tool definitions. LLMs cannot plan downstream calls without knowing response fields.
No error handling guidance visible. Tools do not document what errors can occur, how to recover, or what the LLM should do next. E.g., 'memory_id not found', should the agent retry, search for the ID, or fail?