Loading ToolBench
MCP Server for customer success agent that processes inbound customer messages, detects escalation triggers, searches knowledge base, and returns channel-appropriate formatted responses
Stage 1 prototype with basic tool definitions but significant quality gaps. All 5 tools have descriptions and input schemas with types, but descriptions are generic (avg 80 chars, well below 194-char baseline). Parameters lack depth: no enums for constrained values (channel, priority, severity, status), no format guidance, no dependency documentation. Output schemas are completely undocumented, LLMs cannot plan downstream calls. Error handling absent. No security considerations (credentials, permissions, audit). Tool composition is reasonable (single responsibility) but lacks chaining metadata. Naming follows verb_noun convention correctly.
Create a support ticket in the in-memory ticket store.
Flag a ticket for human agent review and generate an escalation holding response.
Retrieve customer interaction history and profile information.
Search the local knowledge base for content relevant to the query.
Format and return a channel-specific response to the customer.
No output schemas documented. LLMs cannot infer return types, field names, or structure for downstream tool chaining. E.g., search_kb returns results but format is unknown; create_ticket returns ticket_id but structure undocumented.
Constrained parameters lack enums. 'channel' accepts email/whatsapp/web_form but is free-form string; 'priority' and 'severity' accept known values but no enum constraint; 'status' in create_ticket is free-form. LLMs will hallucinate invalid values.
Descriptions are generic and lack actionable context. E.g., 'Retrieve customer interaction history and profile information' (56 chars) does not explain when to call it vs search_kb, what fields are returned, or prerequisites. Baseline is 194 chars; these average ~80.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | 2026-07-28+ | v2 |
No error handling guidance. Tools do not document failure modes, recovery steps, or actionable error messages. E.g., if search_kb finds no results, what should the agent do? If create_ticket fails, is it retryable?
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Agents cannot distinguish safe reads from destructive writes. create_ticket and escalate_to_human modify state but lack destructiveHint; search_kb and get_history are read-only but lack readOnlyHint.