Loading ToolBench
A Python library for managing connections to multiple Model Context Protocol (MCP) servers
This inventory management server demonstrates moderate definition quality with reasonable parameter documentation but significant gaps in error handling and output schema documentation. All 5 tools have input schemas with typed parameters and multi-line descriptions explaining functionality. However, descriptions lack the specificity recommended by the rubric (10-1024 chars, LLM-optimized 50-200 chars target). Tool naming follows verb_noun convention consistently (add_category, add_supplier, add_product, add_supplier_product, add_inventory_item), but all tools cluster around a single verb ('add_'), reducing semantic differentiation. No output schemas are documented, LLMs cannot predict response structure. Parameter descriptions are present but often generic (e.g., 'Unique supplier identifier (required, max 50 chars)' lacks context on format or use cases). No enumerated constraints are visible for status fields, forcing LLMs to guess at valid values. Error handling is entirely absent from the tool definitions, no guidance on what happens if a category already exists, a product references a non-existent category, or validation fails.
Add a new product category to the inventory database. Categories are used to organize products into logical groups (e.g., 'electronics', 'clothing'). Category names are case-insensitive and stored in lowercase.
Add a new inventory item for stock tracking at a specific location. Inventory items track the actual stock quantities and pricing for products at specific locations. Multiple inventory items can exist for the same product (e.g., different warehouse locations).
Add a new product to the inventory database. Products are items that can be stocked in inventory. Each product must belong to a valid category and have a unique name (case-insensitive) and SKU if provided.
Add a new supplier to the inventory database. Suppliers provide products to the inventory. Each supplier must have a unique ID.
Add a supplier-product relationship to link a supplier with a product. This creates the many-to-many relationship between suppliers and products, allowing multiple suppliers to provide the same product with different pricing and terms.
No output schemas documented. LLMs cannot predict what fields are returned (e.g., does add_category return {id, name, description} or just {success: true}?). This blocks downstream tool chaining and forces agents to guess at response structure.
Status parameter in add_inventory_item lists valid values in description ('active', 'inactive', 'out_of_stock', 'discontinued') but does not declare as JSON Schema enum. LLMs may hallucinate invalid statuses like 'pending' or 'active_stock'.
No error handling guidance. If add_category is called with a category that already exists (case-insensitive), what happens? Does it fail silently, return a conflict error, or upsert? No recovery hints provided to LLM.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 32 | - | v1 |
All tools cluster around verb 'add_'. No READ tools provided (e.g., get_category, list_suppliers, search_products). This limits agent ability to verify prerequisites before write operations (e.g., does the category exist before adding a product?) and forces agents to operate blindly.
Parameter descriptions lack actionable constraints. E.g., 'Product name (required, unique, 1-100 chars)' does not clarify: is uniqueness case-sensitive? What characters are allowed? Does it reject leading/trailing whitespace? LLMs infer poorly from vague specs.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All tools are WRITE operations but lack the destructiveHint flag. LLMs cannot distinguish which tools are safe to retry vs. risky without explicit metadata.
add_supplier_product and add_inventory_item accept product_id as UUID but do not explain how to obtain it. Agents cannot know whether to call a get_product_by_name tool (which doesn't exist) or manually craft a UUID.