Loading ToolBench
A mock merchant-catalog MCP server exposing product search, availability, and ordering tools over the Model Context Protocol (stdio transport).
Three tools with complete input/output schemas and descriptions. Naming follows verb_noun pattern (search_products, check_availability, place_order). Descriptions are clear and action-oriented (avg ~120 chars). All parameters have types and descriptions. Output schemas are documented with structured content. Error handling is present but minimal, check_availability returns isError flag for missing products, but place_order lacks validation error guidance. No tool annotations (readOnlyHint/destructiveHint) despite place_order being destructive. Composition is sound: tools chain naturally (search → check → order). Missing: per-parameter constraints (e.g., quantity bounds), recovery guidance in error messages, and explicit idempotency statements.
Check the stock status and on-hand quantity for a single product, given its product id.
Place an order for a given quantity of a product. Validates the product exists and that enough stock is available, decrements stock, and returns a mock order confirmation.
Search the merchant catalog by keyword, with an optional category filter. Returns matching products with id, name, price, and stock.
place_order lacks destructive operation safeguards. No confirmation step, dry-run mode, or explicit warning in description that this modifies state. Description says 'decrements stock' but does not warn agents this is irreversible.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) in tool registration. place_order should be marked destructive; search_products and check_availability should be marked readOnly. These hints guide agent planning and safety.
place_order error handling incomplete. If stock is insufficient, the tool returns isError but does not include the constraint violated (e.g., 'Requested 10 units but only 5 in stock'). LLM cannot self-correct without this detail.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2025-06-18+ | v2 |
quantity parameter in place_order lacks explicit bounds in description. Schema enforces .positive() but description does not state min/max. Rubric baseline: numeric params should specify range (e.g., '1 - 1000 units').
search_products returns full product objects including description field. For catalog search, returning only id, name, price, stock (as promised in tool description) would reduce token waste. Full descriptions should be fetched via a separate get_product_details tool if needed.