Loading ToolBench
FastMCP server providing tools for a multi-tenant café ordering platform, enabling AI agents to search products, manage checkouts, fetch orders, and access business analytics.
The server has 15 tools with basic definition structure, but significant quality gaps prevent higher scoring. Tool descriptions are present but inconsistent in depth and actionability. Input schemas exist for most tools but lack comprehensive type information and validation constraints. Error handling guidance is absent. No tool annotations (readOnlyHint/destructiveHint) are present despite clear WRITE vs READ-ONLY distinction in the tool metadata. Output schemas are not documented. The average tool description is ~150 chars, within the 10-1024 baseline, but many descriptions lack prerequisite guidance or error recovery hints. Multi-step workflows (e.g., add_item_to_checkout requires prior search_product_by_name) are documented informally in descriptions rather than through structured composition patterns. Security concerns exist around token handling (set_auth_token tool accepts raw JWT strings without sanitization guidance). The server is HTTP-based (fastmcp), which is positive for protocol readiness, but lacks idempotency hints and request-level _meta configuration.
Adds an item to the user's checkout basket. Args: product_id: The MongoDB ObjectId of the product (must be 24 hex characters) product_name: The name of the product (for reference) quantity: How many items to add (default: 1) user_email: The email of the user making the purchase Important: Always search for the product first using search_product_by_name() to get the correct product_id before calling this function.
Adds an item to a customer order.
Completes the user's order, removing all items from the checkout basket to create an order. Use this to complete a user's order.
Fetches the current user's checkout basket.
Fetches all orders made by a specific user, including order history and total order count. Use this to inform the user of what they have purchased and how many times.
CRITICAL: set_auth_token accepts JWT tokens as tool parameters, violating secret-injection pattern. Credentials in tool parameters leak into agent traces and logs.
Missing output schemas for ALL tools (15/15). LLM cannot plan multi-step workflows or extract chaining IDs (e.g., order_id after completing an order). This prevents composition.
No pagination support documented on search_all_products and list-like tools (get_business_products). Without limit/offset/cursor, large datasets blow context windows.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 56 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 35 | - | v1 |
Fetch comprehensive analytics including revenue, orders, trends, and top products. Use this as the primary data source for performance questions.
List all products in the business's catalog with details. Use this to understand current offerings and identify product gaps.
Get anonymized, aggregated competitor benchmarks by category. Use this to compare performance and identify market opportunities. IMPORTANT: All data is aggregated across multiple businesses - no individual business names or identifiable data.
Retrieves the current status of an order.
Analyze trends and provide natural language insights with recommendations. Use this to summarize performance and suggest improvements.
Suggest products to add based on competitor offerings and platform trends. Use this when owners ask what products they should add or how to expand.
Removes an item from the user's checkout basket. Args: product_id: The MongoDB ObjectId of the product (must be 24 hex characters) product_name: The name of the product (for reference) user_email: The email of the user making the purchase Important: Always search for the product first using search_product_by_name() to get the correct product_id before calling this function.
Searches the mongoDB via the express backend REST API for all products. This tool will provide the agent with a list of all products, allowing them to understand what is available
Searches for a product by name and returns its MongoDB ObjectId and details. Use this to find the product_id before adding to checkout.
Sets the authentication token to be used when calling protected backend endpoints. Pass either the raw JWT or the full "Bearer <token>" string.
No error handling guidance. Tools do not document retryability, user-fixable errors, or recovery paths. E.g., if product search returns empty, does the tool suggest alternatives? If checkout is empty, can complete_user_order fail gracefully?
Tool annotations missing. Despite clear WRITE vs READ-ONLY distinctions in metadata, tools lack destructiveHint and readOnlyHint annotations. This prevents agents from reasoning about safety and side effects.
Inconsistent parameter validation constraints. Some tools describe constraints in description text only (e.g., product_id '24 hex characters') rather than in JSON Schema format. LLMs cannot reliably parse prose constraints.
Numeric parameters lack min/max bounds. 'quantity' parameter on add_item_to_checkout and add_item_to_order has no range constraint, LLM could pass 0 or 1000000.
Unclear composition: add_item_to_order and add_item_to_checkout are similar tools with no clear distinction. When should an agent use one vs the other? Duplicate tool names signal multiple responsibilities.
Missing chaining IDs. After complete_user_order, the agent needs an order_id to call get_order_status or add_item_to_order. No output schema documents whether order_id is returned.
Natural language outputs are unstructured. get_performance_insights returns 'natural language insights with recommendations' as free-text. LLM cannot extract, reason about, or validate individual recommendations.