Loading ToolBench
MCP server that exposes customer data, order management, payment history, and Zendesk ticket search tools for Zendesk Copilot integration with Claude AI
15 tools with complete input schemas and descriptions, but significant quality gaps. All tools are READ_ONLY (good for safety), but descriptions are generic and lack LLM-optimization guidance. Parameter descriptions are minimal (1-10 words). No output schemas documented. Error handling is absent, no recovery guidance, no actionable error messages. Tools like query_databricks expose complex SQL with minimal constraint documentation. Missing idempotency hints, permission scopes, and composition guidance. Naming is verb-noun compliant but descriptions don't explain WHEN to use each tool vs. alternatives (e.g., search_customer vs. list_individuals both find people but criteria differ).
Get full customer details by customer ID.
Get eSIM profile status by ICCID resource ID. Shows provisioning status and QR code availability.
Get individual (subscriber) details by individual ID. Includes MSISDN, IMSI, status.
Check for network outages by ZIP code.
Get details of a specific order by order ID.
Get the execution timeline and status for an order. Shows each step (payment, provisioning, activation).
query_databricks tool exposes raw SQL with minimal input validation. Description lists table names and SQL patterns but lacks constraints on query complexity, timeout limits, or injection prevention. Parameter 'sql' has no format, length, or allowed-operation constraints.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields. E.g., search_customer returns 'list of matching customers' but structure is undefined, does it include customer_id, email, phone? Required for tool chaining.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
Run Root Cause Analysis (RCA) on an order. Shows why an order failed or is stuck.
List all orders for a customer by customer ID.
Get payment history for a customer. Note: amount fields may be NULL in some records.
Get the port-in status for a phone number (MSISDN).
Get SIM products and services for an individual. Returns active SIM, plan, and service details.
Find an individual by MSISDN or IMSI. Use resourceType "msisdn" or "imsi" with the value.
Execute a read-only SQL query on Databricks. Use this for historical/aggregated data like ticket counts, Mochi conversations, onboarding metrics, order history, etc. Key tables: - prod_catalog.customer_support.zendesk_tickets (tickets: id, created_at, status, type, priority, tags) - prod_catalog.customer_support.zendesk_ticket_metrics (ticket_id, solved_at, reply_time_in_minutes, reopens) - prod_catalog.telco.customer (id, status, created_at) - prod_catalog.telco.payment (id, name, payer_id, status, payment_date — amount fields are NULL) - `rds-prod_catalog`.cj_prod.mvno_order (order_id, user_id, status, esim_status, select_number_type, portin_response, created_at) - `rds-prod_catalog`.cj_prod.conversations (conversation_id, user_id, title, message_count, metadata JSON with human_escalated, main_category) - `rds-prod_catalog`.cj_prod.stripe_customers (user_id, customer_email, customer_name, customer_phone) - `rds-prod_catalog`.cj_prod.user_onboarding_flow (user_id, order_id, payment_completed_time, activation_completed_time) - `rds-prod_catalog`.cj_prod.customer_airvet_info (user_id, is_airvet_activated, activated_at) Important: - rds-prod_catalog has a hyphen — ALWAYS wrap in backticks: `rds-prod_catalog` - Dedup Zendesk snapshots: ROW_NUMBER() OVER (PARTITION BY id ORDER BY updated_at DESC) as rn, then WHERE rn = 1 - Exclude deleted tickets: WHERE status != 'deleted' AND tags NOT LIKE '%pagerduty%' - Port-in filter: select_number_type = 'PORTIN' OR portin_response LIKE '%PortInMSISDN%' - Escalation detection: get_json_object(metadata, '$.human_escalated') = 'true' - Only SELECT queries allowed
Search for a customer by name, email, or MSISDN (phone number). Returns a list of matching customers.
Search Zendesk tickets by customer email. Returns the 10 most recent tickets for a customer, sorted by newest first.
Parameter descriptions are minimal (1-10 words). E.g., 'The customer ID' for customer_id, 'The order ID' for order_id. No guidance on format, length, or valid ranges. LLMs cannot validate inputs or self-correct on invalid values.
No error handling or recovery guidance. Tools provide no actionable error messages. E.g., if search_customer finds no matches, what should the LLM do next? Try a different search term? Call list_individuals? No guidance.
Tool descriptions lack WHEN-to-use guidance. search_customer and list_individuals both find people but criteria differ (name/email/phone vs. msisdn/imsi). LLM cannot distinguish when to call each without trial-and-error.