Loading ToolBench
Demo MCP server for ticket workflows — local stdio and remote HTTP. Teaches real-world MCP patterns: tool naming, attribution, schema discovery, auth, and PII gating.
Server has 10 tools with complete input schemas and descriptions. Naming follows verb_noun convention (search_tickets, create_ticket, get_ticket, etc.). Descriptions are present and actionable (avg ~80 chars), explaining WHAT and WHEN. However, several descriptions lack depth on error cases and recovery paths. Parameter descriptions are adequate but some lack format constraints (e.g., ticket_id format 'TCK-XXXX' not formally specified). Output schemas are not documented in the visible source, only inferred from descriptions. Error handling guidance is minimal. Security considerations (auth, rate limits) mentioned in describe_server but not enforced per-tool.
Comment on a real ticket_id from search_tickets or get_ticket
Resolve a ticket (destructive, idempotent). Optional resolution note
Open a ticket. ALWAYS pass requester_email or the service account owns it
Call first (and after any denial): identity, auth mode, your scopes, rate limit, and every tool
Fields and filterable keys for one schema. Call after list_schemas
Fetch one known ticket id including comments and attribution
List queryable schemas before run_query. There is no query_tickets tool
Output schemas not documented. Tool descriptions infer return types but no formal schema is visible for any tool. LLMs cannot plan downstream calls or extract fields reliably.
Parameter format constraints missing. ticket_id described as 'e.g., TCK-1001' but no regex or enum enforces format. requester_email lacks email validation hint. LLMs may pass invalid values.
Error handling guidance absent. No tool description explains what happens on failure (ticket not found, invalid email, permission denied) or how to recover. Agents cannot self-correct.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | <=2025-11-25 | v2 |
Customer record by email. Phone is PII and stays redacted without the pii scope
The one query tool. Pass schema from list_schemas — do not invent query_* tools
Find tickets by status / requester / keyword. Default status is open; empty means no match, not a broken server
Pagination not documented. search_tickets and run_query accept 'limit' but no mention of offset/cursor, total count, or next_page behavior. Large result sets may exceed context window.
Idempotency not declared. close_ticket claims 'idempotent' in description but no explicit idempotentHint annotation. create_ticket and add_comment lack idempotency guarantees, risking duplicate records on retry.