Loading ToolBench
Two tools with strong descriptions and clear purposes. hippocampus_query has excellent documentation (250+ chars) explaining when to use it and what it returns. hippocampus_log is exceptionally detailed (500+ chars) with explicit two-phase flow, token validation, and relationship semantics. Both tools have complete input schemas with types and descriptions. However, output schemas are not formally documented, responses are free-text strings rather than structured objects. Parameter descriptions are thorough but could benefit from explicit enum constraints for category/weight/relationship-type values. No error handling guidance for common failure modes (e.g., invalid token, malformed JSON in relationships param).
Record an architectural decision. Two-phase flow: Phase 1 — call without confirmed=True. Returns classification and related decisions as candidates for relationship linking, plus a token. Read the candidates carefully: if any constrained your choice, include them in Phase 2. Phase 2 — call again with confirmed=True, the token from Phase 1, and the relationships list (JSON string: '[{"type":"depends-on","target":"DR-0001"}]', or '[]' for none). Writes the record and updates the index. The token proves Phase 1 actually ran for this exact description before anything gets written. confirmed=True with no token, or with a token issued for a different (or since-edited) description, does not write — it returns Phase 1's output again, with a fresh token, instead. This is deliberate: an agent that logs a decision with no Why produces a record worth nothing, and Phase 1's related-decisions list is the mechanism for catching that before it's written, not an optional courtesy. Relationship types: depends-on | supersedes | conflicts-with weight options: heavy | standard | deferred (omit to auto-classify) category options: architectural | domain | data | security | api | performance | dependency | testing | error-handling | state | naming | operational | compliance | cost | team | ux-product alternatives — what was considered and rejected, as a JSON array of strings: '["RabbitMQ — extra ops burden", "Redis streams — no durability guarantee we need"]'. Each entry should name the option AND the reason it was rejected ("option — reason"), not just the option name; a bare option name is not useful to a future reader. Optional for standard records, but encouraged — it is the whole point of
Semantic search over past architectural decisions. Call this before any non-trivial decision: choosing a library, designing a schema, picking an interface, or any fork with real alternatives. Returns ranked results with inline Why, Rejected alternatives, and Depends-on so you do not need to open record files.
Output schemas not documented. Both tools return free-text strings; LLMs cannot parse structured results or chain downstream calls. hippocampus_query returns ranked results with inline metadata, but the response format (header structure, field ordering, badge syntax) is implicit in the code, not declared in the tool schema.
Parameter enums not declared. 'category' accepts 14 values (architectural, domain, data, security, api, performance, dependency, testing, error-handling, state, naming, operational, compliance, cost, team, ux-product) and 'weight' accepts 3 (heavy, standard, deferred), but these are documented only in the description text, not as JSON Schema enums. LLMs cannot validate against undeclared enums and may hallucinate invalid values.
No error handling guidance. If an LLM passes an invalid token, malformed JSON in 'relationships', or a non-existent target DR-ID, the tool will fail. No recovery hints are documented (e.g., 'If token is invalid, call hippocampus_log again with confirmed=false to get a fresh token').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
Relationship type values not enumerated in schema. 'relationships' parameter accepts JSON with 'type' field that must be one of 'depends-on', 'supersedes', 'conflicts-with', but this constraint is only in the description, not in the schema. LLMs may pass invalid relationship types.
No idempotency or confirmation pattern for destructive operations. hippocampus_log with confirmed=true writes records to disk. The two-phase flow (Phase 1 token validation) is a safeguard, but no explicit dry-run or confirmation-request pattern is documented. An LLM could accidentally log a decision with incomplete metadata.