MCP server for support ticket agent with confidence-gated tool routing, RAG pipeline with hyperparameter tuning, and evaluation metrics
Server has 5 tools with basic naming and descriptions, but critical gaps in schema documentation, parameter descriptions, and error handling. Tool names follow verb_noun convention (get_ticket, add_comment, escalate_ticket, close_ticket, delete_ticket), which is good. However, descriptions are minimal (10-50 chars), parameter descriptions are sparse or missing, and output schemas are not documented. No error recovery guidance. Destructive tools (close_ticket, delete_ticket) lack confirmation/dry-run patterns. Risk levels are declared in code but not exposed to the LLM via tool annotations.
Add a comment to an existing ticket
Close a ticket (destructive operation requiring confirmation)
Delete a ticket permanently (destructive operation requiring confirmation)
Escalate a ticket to a higher priority status
Retrieve ticket information by ticket ID
Destructive tools (close_ticket, delete_ticket) lack confirmation/dry-run pattern. No mechanism to prevent accidental irreversible operations.
Output schemas not documented. LLM cannot infer what fields to expect from tool responses, breaking downstream tool chaining.
Parameter descriptions missing or trivial. 'ticket_id' described only as 'The ticket identifier', no format, length, or example constraints.
No error recovery guidance. Functions return {'error': 'not found'} with no hint to the LLM on what to do next (e.g., search for ticket, ask user).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | 2026-07-28+ | v2 |
Risk levels (READ_ONLY, WRITE, IRREVERSIBLE, DESTRUCTIVE) declared in code but not exposed via tool annotations. LLM cannot see which tools are safe to retry.