Loading ToolBench
Read-only MCP operations console over a sanitized fleet snapshot. Exposes operational health telemetry: fleet status, heartbeat coverage, reconciliation runs, alert history, audit volumes, and monitoring topology.
All 6 tools have descriptions (10-60 chars, below the 50-200 char LLM-optimized baseline) and empty input schemas ({}). Tool names follow verb_noun convention (get_*) and are clear. However, every tool lacks parameter documentation, output schema documentation, and error handling guidance. The descriptions are minimal and do not explain WHEN to use each tool or what the LLM should expect in the response. No tool has documented return types or field structures. This is a read-only ops console with no parameters, but the absence of output schema documentation violates pattern:tool and pattern:response-shaper. Descriptions average ~35 chars vs. the 194-char baseline for A+ tools.
Alert events in the window: class, severity, paged, recovery timing.
Audit event volumes by generic class over the window.
Per-component run status and engine counts across the fleet.
Heartbeat coverage per component over the monitoring window.
Component counts and watchdog arming state.
Account reconciliation run counts and consecutive verified streak.
No output schema documentation. Tools return complex dicts (e.g., get_fleet_status returns components, engines_running_total, all_running, generated_at) but LLMs have no formal schema to understand field types, nesting, or what to extract for downstream use.
Descriptions are minimal (10-60 chars, well below 50-200 char LLM-optimized baseline). They state WHAT but not WHEN to use or what the response structure is. E.g., 'Heartbeat coverage per component over the monitoring window' does not explain whether to call this first, what fields are returned, or how to interpret coverage_pct.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
No error handling guidance. Tools call _screen() which raises on forbidden tokens, but LLMs have no recovery path documented. If a tool fails, the agent cannot determine whether to retry, ask the user, or move on.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All 6 tools are read-only and idempotent, but this is not declared in the tool definition. LLMs cannot infer safety properties and may treat these as potentially destructive.
STDIO transport only. Server is not remotely accessible and cannot be used by hosted MCP clients. This is a hard architectural limitation.