Server provides 10 tools with explicit schemas and descriptions. Naming follows verb_noun convention (list_, describe_, call_, get_, create_, update_, who_is_on_call). Descriptions are present but vary in quality (10-200 chars). All tools have typed input schemas with parameter descriptions. However, output schemas are not documented, error handling lacks recovery guidance, and several tools expose low-level API details that could be abstracted. Generic tools (call_read, call_write, call_delete) bypass the curated incident workflow, increasing cognitive load on LLMs. No tool annotations visible in code despite ToolAnnotations objects being defined.
Call a DELETE operation on the All Quiet API.
Call a GET operation on the All Quiet API.
Call a POST, PUT, or PATCH operation on the All Quiet API.
Create a new incident.
Describe an All Quiet API operation: parameters, request body, response, required API key scope.
Get details of a specific incident.
Output schemas not documented. Tools return data but LLMs cannot predict field names, types, or structure. Breaks downstream tool chaining and forces agents to guess at response format.
Generic call_* tools expose low-level API details (method, path templates, query params). LLMs must understand HTTP semantics and OpenAPI structure. Prefer high-level domain tools (list_incidents, create_incident) over generic API wrappers.
Tool annotations (READ, WRITE, DELETE) defined in code but not registered with tools. ToolAnnotations objects exist but are never passed to tool registration. LLMs cannot see read_only_hint or destructive_hint.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 69 | 2026-07-28+ | v2 |
List incidents, optionally filtered by status, team, or organization.
List All Quiet API operations as `METHOD path — summary [required key scope]`.
Update an existing incident.
Get the current on-call person for a team or organization.
Error handling returns ToolError with minimal context. No recovery guidance (e.g., 'Use list_operations to find valid paths'). Errors like 'path_params is missing X' lack actionable next steps for LLM.
list_incidents and similar tools lack pagination documentation. No mention of limit/offset behavior, max results, or whether results are truncated. Agents cannot plan for large datasets.