Loading ToolBench
Metacognitive AI agent oversight: adaptive CPI interrupts for alignment, reflection and safety
Server has 5 well-defined tools with explicit schemas and descriptions. Tool names follow verb_noun convention (vibe_check, vibe_learn, update_constitution, reset_constitution, check_constitution). Most parameters have type definitions and descriptions. However, several parameters lack detailed descriptions explaining their purpose and constraints. Output schemas are not documented for any tool. Error handling is minimal, no recovery guidance provided. The metacognitive/constitution pattern is novel but under-documented for LLM reasoning.
Return the current constitution rules for this session
Overwrite all constitutional rules for this session
Append a constitutional rule for this session (in-memory)
Metacognitive questioning tool that identifies assumptions and breaks tunnel vision to prevent cascading errors
Pattern recognition system that tracks common errors and solutions to prevent recurring issues
Output schemas not documented. Tools define inputs well but provide no guidance on what fields the response contains. LLMs cannot plan downstream operations or extract data without knowing response structure.
Constitution tools (update_constitution, reset_constitution, check_constitution) have minimal descriptions. 'Append a constitutional rule' and 'Overwrite all constitutional rules' lack context about what a 'constitutional rule' is, when to use each tool, and what the LLM should expect in responses.
No error handling or recovery guidance. Tools do not document what happens on failure (e.g., invalid sessionId, missing rule, API failure calling LLM provider). LLMs have no actionable next steps when errors occur.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | B | 71 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 34 | - | v1 |
sessionId parameter in vibe_check and constitution tools is marked 'optional' but its role is unclear. Description says 'Optional session ID for state management' but does not explain: what happens if omitted? Is state per-request or per-session? Can LLMs query or reset session state? This ambiguity will cause misuse.
modelOverride parameter in vibe_check has no constraints on the 'model' string field. LLMs can hallucinate invalid model IDs (e.g., 'gpt-5-super'). Should validate against a known list or at minimum document the format more strictly.
uncertainties parameter in vibe_check is an array but the description does not explain format: are these natural-language strings? Structured objects? Single-word keywords? Should LLMs pass ['uncertain about X'] or ['deployment planning']? Ambiguity invites format errors.