Loading ToolBench
A multi-server MCP system for customer support operations, including knowledge base search, order management, and support ticket handling
This server has 12 tools with moderate quality definition. Naming follows verb_noun convention (search_kb, get_order_by_id, create_ticket, etc.), which is positive. However, descriptions vary widely in completeness, most are present but many are generic (e.g., 'Retrieve the full contents of a knowledge base document' lacks context about when/why to use it). Input schemas are present for all tools with proper JSON Schema format, but output schemas are NOT documented anywhere. Most parameters have descriptions, but they are minimal. The server exposes both read-only and write operations without clear idempotency hints or confirmation mechanisms for destructive operations. Error handling is absent from the visible code, no guidance for recovery or error classification. The baseline for this category is ~50-60 due to STDIO-only transport and missing output documentation, though naming and parameter presence push slightly higher.
Create a new support ticket for a customer. Returns the new ticket ID.
Look up a customer's profile (name, phone, join date) by email.
Return all orders placed by a customer identified by their email address, sorted newest first.
Get all tickets raised by a specific customer (by email).
Retrieve the full contents of a knowledge base document.
Fetch full details of a single order by its order ID (e.g. ORD-2001). Returns product, status, tracking number, amount, and timestamps.
Fetch details of a specific ticket by its ticket ID.
Output schemas not documented. No return type specifications visible in tool definitions. LLMs cannot determine what fields to expect, breaking downstream tool chaining and forcing agents to guess.
No error handling guidance in any tool. Agents receive raw errors or exceptions with no recovery hints. Missing 'actionable errors' pattern, errors should tell LLMs what to do next.
Destructive/write tools lack confirmation or dry-run support. update_order_status, create_ticket, and resolve_ticket can modify state irreversibly with no safeguards against agent mistakes.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
List all open (unresolved) support tickets, sorted by priority.
List the N most recent orders across all customers.
Mark a ticket as resolved and record the resolution notes.
Search the knowledge base for information about refunds, shipping, or product FAQs. Returns relevant document sections that match the query.
Update the status of an order. Valid statuses: processing | shipped | delivered | cancelled | refunded.
Tool descriptions are generic and lack context. E.g., 'Retrieve the full contents of a knowledge base document' (get_document) doesn't explain WHEN to use it vs search_kb or when it's preferred. Average description length is ~60 chars; rubric baseline is 194 chars (p10=34, p90=392).
No pagination mechanism visible for list_recent_orders or list_open_tickets. Both accept limit but no offset/cursor or total_count. Large result sets will blow context window.
Tool composition issue: create_ticket requires 'customer_email' but get_customer_orders returns orders without explicit customer_email in output. Chain is broken if output does not include email.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) visible. update_order_status and resolve_ticket are destructive but lack annotations to signal this to agents.
Parameter descriptions are minimal. E.g., create_ticket has 'priority' with enum but description doesn't explain what each level means or when to use each. Baseline is 72 chars per param description.