Loading ToolBench
Fan-out/fan-in incident response orchestrator built with ADK, A2A, and MCP. Coordinates multiple specialist agents (telemetry, runbook, memory, planner) to triage incidents, correlate signals, and synthesize remediation plans.
Server has 11 tools with basic schemas and descriptions, but significant gaps prevent production use. All tools have descriptions (10-80 chars, mostly short), and input schemas are present with type definitions. However, parameter descriptions are almost entirely missing, LLMs cannot infer what 'service', 'symptom', 'window_minutes', or 'summary' mean without explicit guidance. Output schemas are undocumented; callers must reverse-engineer response structure from code. Error handling is minimal (generic 'not found' messages with no recovery guidance). No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels. Naming is verb-forward but some tools are vague: 'emit_audit_stamp' and 'save_machine_summary' lack clarity on side effects. No parameter constraints (enums, ranges, patterns) despite tools accepting free-form strings like 'risk_level' and 'change_type'. Composition is reasonable, tools are single-purpose, but output chaining is weak (e.g., find_similar_incidents returns incident_id but no downstream tool accepts it as input).
Emit a traceable audit record for the incident artifact.
Return likely correlated signals for a service and observation window.
Find similar incidents and extract what worked and what failed.
Fetch the current metric snapshot for a service.
Fetch a single postmortem by incident id.
Fetch a runbook by id.
List the active incident rows, optionally filtered by service.
Parameter descriptions missing across all tools. 'service', 'symptom', 'window_minutes', 'risk_level', 'change_type', 'operator', 'summary' have no guidance. LLMs cannot infer meaning from names alone.
Output schemas undocumented. Callers must reverse-engineer response structure from code. E.g., find_similar_incidents returns {matches: [{incident_id, service, symptom, score, summary, what_worked, what_failed}]} but this is invisible to LLMs.
No tool annotations despite clear risk levels. WRITE tools (request_change_approval, emit_audit_stamp, publish_incident_brief, save_machine_summary) lack destructiveHint. READ tools lack readOnlyHint. Agents cannot distinguish safe from risky calls.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
Persist the final incident brief to a local artifact file.
Create a human approval packet for a risky mitigation.
Persist a JSON summary alongside the markdown brief.
Search incident runbooks by service and symptom.
Error handling is generic. get_postmortem returns {error: 'incident X not found'} with no recovery guidance. Should suggest: 'Try search_runbooks() or find_similar_incidents() to locate related incidents.'
No parameter constraints. 'risk_level' accepts free-form strings; code checks for 'high' or 'critical' but LLMs have no enum to guide input. 'window_minutes' is unbounded (could be 0 or 999999). Missing min/max and enum declarations.