Loading ToolBench
A CEO decision brain: typed-fact memory, deterministic retrieval, cited answers, human-in-the-loop decisions, MCP interface.
Server has 7 tools with clear naming (verb_noun pattern) and reasonable descriptions (avg ~150 chars). Input schemas are present and mostly typed. However, several critical gaps: (1) output schemas are undocumented, LLMs cannot predict response structure; (2) error handling is absent, no recovery guidance or categorization; (3) parameter descriptions lack constraints (enums, ranges, formats); (4) no pagination guidance despite query tools returning lists; (5) ask() and log_decision() lack idempotency/confirmation patterns for irreversible operations. Composition is sound (single responsibility per tool), but schema completeness and error patterns fall short of production baseline.
Ask the decision brain a CEO-level question. Returns a cited brief plus a pending decision id — the brief is a recommendation only; nothing is recorded as decided until you call log_decision.
List the conflicting claims the brain has detected in memory, optionally filtered to just the resolved or unresolved ones.
Feed new raw items (calls, emails, notes, tweets, docs) into the brain — this is the only way to write to memory, and each item runs through the full extraction pipeline.
Record your approval or rejection of a pending decision. A given decision_id can be finalized exactly once; replays return an error.
Look up the canonical people, companies, and investors the brain knows about (read-only), optionally filtered by type or name.
Search the brain's memory for typed, evidence-tiered facts (read-only), optionally filtered by predicate or the subject they're about.
Output schemas undocumented. ask() returns {decision_id, brief, pending} but LLM cannot predict structure. query_* tools return lists but no pagination metadata (total_count, next_cursor) documented. LLMs cannot plan downstream calls or extract chaining IDs.
No error handling or recovery guidance. Tools lack error categorization (retryable vs user-fixable vs fatal). ask() and log_decision() have no guidance for failure modes (e.g., 'decision_id not found' or 'already finalized'). Agents cannot self-correct.
query_* tools accept optional filters but lack constraint documentation. 'limit' parameter has no min/max stated (code shows 1-200 default 50, but description omits this). 'status' enum in query_signals not documented in description. LLMs may pass invalid values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
Browse aggregated signals and where each sits on the promotion ladder (candidate → emerging → validated → decision_grade), optionally filtered by type or status.
log_decision() lacks confirmation/dry-run pattern for irreversible operation. Once a decision_id is finalized, it cannot be changed. No guidance on how to handle accidental approvals or how to verify before committing.
ingest_items() accepts array but no per-item success/failure reporting. If 1 of 50 items fails validation, response only shows aggregate counts. Agent cannot retry individual items or understand which failed.