A scheduling and on-call rotation management system with team coverage analysis, conflict detection, and notification routing. Includes intentional bugs for educational lab exercises.
Three tools with clear names and basic descriptions, but significant gaps in schema completeness and parameter documentation. All tools are READ_ONLY and well-scoped, but descriptions lack actionable context for LLM selection. Input schemas present but parameters lack detailed type info and constraints. Output schemas not documented. Error handling and recovery guidance absent.
Retrieve a scenario payload containing team coverage analysis, notifications, conflicts, and detected issues for a given scenario ID
List all available scheduling scenarios with their metadata (id, title, description)
Generate an agent triage packet for a scenario, including detected issues, recommended ticket, and next commands
Output schemas not documented. LLMs cannot plan downstream operations or extract required fields from ScenarioPayload and AgentTriagePacket responses.
Parameter descriptions lack actionable constraints. 'scenarioId' accepts only 4 known values (baseline, coverage-inflation, escalation-hole, timezone-ghost-conflict) but this is not documented as an enum or constraint in the schema.
Tool descriptions lack WHEN-to-use context. 'get_scenario' and 'triage_scenario' both retrieve scenario data but the distinction is unclear, when should an LLM call one vs the other?
No error handling or recovery guidance. If an invalid scenarioId is passed, the server likely returns a generic error with no hint for the LLM to call list_scenarios() or retry with a valid ID.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
list_scenarios has empty input schema (no parameters) but description does not explain what it returns or when to call it as a discovery tool.