Loading ToolBench
MCP server providing tools to retrieve and analyze automotive sales data from Databricks, including orders, customers, and products
This server presents 4 READ_ONLY tools for automotive sales data querying via Databricks. Tool definitions are present with descriptions and basic schemas, but several quality issues prevent a higher score. Naming is clear and verb-based (get_*). Descriptions range from adequate to good (100-250 chars), explaining when to use each tool. However, schema completeness is uneven: get_orders and get_customers have full input schemas with descriptions, but get_product_category has minimal schema detail, and NO tool documents its OUTPUT schema structure. Error handling is generic ('Error retrieving customers' with no recovery guidance). Parameters lack enums where appropriate (e.g., region should be enum-constrained to 'NA', 'EU', etc.). The server returns raw list-of-dicts, risking unstructured output variance. No tool annotations (readOnlyHint, idempotentHint) present despite being a 100% READ_ONLY service. Overall: functional but under-polished for production LLM agent consumption.
Retrieve customer information. Use this tool when the question references customers by ID, name, industry, or region. Supports filters: - customer_id: return a specific customer - industry: filter customers by industry (e.g., 'Automotive', 'Aerospace') - region: filter customers by sales region (e.g., 'NA', 'EU') - limit: maximum number of customers to return (default: 100) Always return customer_id and customer_name. Example use cases: - "Who are our top aerospace industry customers?" - "Get customer 42's details."
Retrieve sales orders, including nested order lines and product details. Use this tool when the question is about sales, revenue, discounts, customer purchasing behavior, or product sales. Supports filters: - customer_id: only return orders from a specific customer - product_id: only return orders containing a specific product - start_date / end_date: restrict to an order date range (YYYY-MM-DD) - region: filter by sales region (e.g., 'NA', 'EU') - limit: maximum number of rows to return (default: 100) Always includes order_id, customer_id, customer_name, order_date, region, product_id, quantity, unit_price, and line_unit_price.
Resolve a user-provided product category string into the canonical category name stored in the database. Use this when the category in a user question may be misspelled, pluralized, or formatted differently (e.g., 'Brake Pads' vs 'Brake Pad'). Returns: { "input": "Brake Pads", "resolved_category": "Brake Pad", "confidence": 0.92 }
No output schema documented for any tool. LLMs cannot reliably parse or chain results without knowing the structure of returned fields (order_id, customer_name, etc.). Tools return List[dict] but actual field set is implicit.
Parameters lack enum constraints where values are known. 'region' accepts free-form strings but should be enum: ['NA', 'EU', ...]. 'category' for get_products similarly should enumerate valid categories. Free-form strings invite LLM hallucination of invalid values.
Error handling is non-recoverable. get_customers returns generic [{"error": "Error retrieving customers"}] on exception. Provides no guidance: is this retryable? Should the user check permissions? Are there logs? LLM cannot self-correct or take next action.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | B | 72 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 11 | - | v1 |
Retrieve product catalog data. Use this tool when the question involves product details, pricing, or categories. Supports filters: - product_id: return a specific product - category: filter by category (e.g., 'Brakes', 'Tires') - limit: maximum number of products to return (default: 100) Always return product_id, product_name, product_category, unit_cost, and unit_price. Example use cases: - "What is the unit price of product 7?" - "List all products in the 'Brakes' category."
get_product_category has minimal parameter description: name is just 'product category string to resolve'. No guidance on format, length, or examples. Description is vague (60 chars total).
No tool annotations present. All 4 tools are READ_ONLY and idempotent, but lack readOnlyHint/idempotentHint in schema. LLMs cannot reason about side-effect safety or retry logic without explicit hints.
Pagination guidance missing. get_orders and get_customers accept 'limit' but no offset/cursor. If a user asks 'list all customers', the agent can retrieve only first 100, with no clear way to fetch the rest. No 'total_count' or 'has_more' returned.
Incomplete docstring in get_product_category. Code shows a Levenshtein distance algorithm but docstring return example does not match actual implementation output. Says it returns {"input", "resolved_category", "confidence"} but code is incomplete (source cut off).