Model-agnostic AI agent for first-line production-incident triage; read-only evidence tools exposed as MCP servers for logs, commits, and metrics.
Three read-only tools with basic schemas and descriptions. All tools have input schemas with typed parameters and descriptions, meeting minimum requirements. However, descriptions are terse (50-80 chars), lacking context on WHEN to use each tool or HOW results chain together. No output schemas documented. Parameters lack constraints (enums, ranges, formats). Error handling and recovery guidance absent. Tools follow verb_noun naming but lack composition hints (e.g., which tool to call first, how results feed downstream). No tool annotations (readOnlyHint present in metadata but not in schema). Baseline: avg tool description ~194 chars; these average ~65 chars.
List recent deploys newest-first; optional since/limit; commits carry sha/ts/msg/files.
Query structured incident logs; optional since/level/route; rows carry a stable int id.
Query metric time-series (memory/connections/queue depth) for resource incidents; optional name/since; each series carries a `metric` name (cite it), `unit`, and `points`.
Descriptions lack context on WHEN to use each tool and HOW to chain results. 'Query structured incident logs' does not explain when to call query_logs vs query_metrics, or what downstream tools consume its output.
No output schemas documented. LLMs cannot plan downstream calls or extract required fields. E.g., query_logs returns 'rows carry a stable int id' but the schema structure is not visible in the tool definition.
Parameters lack constraints. 'since' accepts any string; no format (ISO 8601?), range, or validation hint. 'level' has no enum (INFO, ERROR, WARN, DEBUG?). 'limit' has no min/max bounds.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | <=2025-11-25 | v2 |
No error handling or recovery guidance. If query_logs returns empty, should the agent retry with different filters, or call a different tool? No guidance provided.
Tool annotations present in metadata (Risk: READ_ONLY) but not in JSON Schema. readOnlyHint should be in schema for LLM visibility.