Loading ToolBench
Local MCP server built with FastMCP that exposes six customer support tools to Claude Desktop for handling order lookups, refund eligibility checks, policy searches, ticket creation, issue summarization, and escalation recommendations.
Server has 6 well-named tools with clear descriptions and documented parameters. All tools follow verb_noun naming (lookup_, check_, search_, create_, summarize_, recommend_). Descriptions are substantive (100-250 chars) and explain WHEN to use each tool. Input schemas are present with type declarations and parameter descriptions. However, output schemas are not documented, LLMs cannot predict what fields to expect from responses. Error handling is minimal; tools do not guide recovery or categorize failures. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels. Parameter constraints lack formal enums (e.g., priority, sentiment, issue_type are free-form strings despite documented allowed values). Missing idempotency guidance for write operations.
Determine whether a customer's order is eligible for a refund. Applies deterministic business rules — the decision is NOT inferred by Claude. Rules: order must be within the 30-day refund window and must not have a 'cancelled' status. Call lookup_order_status first to confirm the order exists. Returns an eligibility decision with a plain-English reason.
Create a new support ticket and save it to local storage. Use this after gathering the key facts about a customer's issue. A ticket ID (TKT-XXXX) is generated automatically. Args: customer_name: Full name of the customer. issue_description: Clear description of the reported problem. order_id: Related order ID, if applicable (e.g. ORD-1042). priority: Urgency level — 'low', 'medium', or 'high'. THIS IS A WRITE ACTION. Confirm the details are correct before calling.
Look up a customer order by its order ID and return the full record. Returns order status, item description, order date, estimated delivery date, and total price. Use this whenever a customer references an order number. Order IDs follow the format ORD-XXXX (e.g. ORD-1042).
Recommend whether a support case should be escalated to a human agent. Uses rule-based logic — the recommendation is deterministic, not inferred. Returns a recommendation and reasoning. Does NOT take any action itself. Args: issue_type: Category of the issue. Options: refund | shipping | complaint | legal | fraud | informational refund_eligible: Result from check_refund_eligibility, if already called. sentiment: Observed customer tone. Options: calm | neutral | frustrated | hostile | threatening order_value: Order total in USD (pass the value from lookup_order_status). priority: Current ticket priority: low | medium | high
Output schemas not documented. LLMs cannot predict response structure, forcing them to guess field names and types. This breaks tool chaining and wastes tokens on clarification.
Parameter constraints (priority, sentiment, issue_type) documented in descriptions but not formalized as enums. LLMs may hallucinate invalid values like 'urgent' instead of 'high', or 'angry' instead of 'hostile'.
No tool annotations (readOnlyHint/destructiveHint/idempotentHint). create_support_ticket is marked WRITE but lacks destructiveHint annotation. Agents cannot distinguish safe retry behavior from irreversible operations.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Search the company's support policy document for a relevant section. Use this to answer questions about refunds, shipping delays, cancellations, returns, or damaged goods. Provide a keyword or short phrase (e.g. 'refund', 'shipping delay', 'damaged'). Returns the most relevant policy section title and body text.
Generate a concise 2-3 sentence summary of a customer's reported issue. Useful for creating clean ticket descriptions or handoff notes. This tool calls the Anthropic Claude API internally and requires ANTHROPIC_API_KEY to be set in the project's .env file. Args: issue_text: The raw customer message or a description of the issue.
Error handling absent. Tools return dict responses but do not document error cases, recovery paths, or actionable error messages. A failed lookup returns nothing; agent has no guidance on next steps.
summarize_customer_issue calls external Anthropic API but does not document timeout behavior, rate limits, or fallback strategy. If API fails, agent receives no guidance.