Loading ToolBench
A commerce discovery protocol server that provides product search and inventory synchronization capabilities for merchants
Single tool with a compound name ('sync_and_search') that violates single-responsibility principle. Tool parameters have types and descriptions, but descriptions are too brief (averaging ~45 chars, below the 50-200 char LLM-optimized baseline). No output schema documented, the mock return value is visible in code but not formally declared. No error handling guidance, no validation of merchant_id format, and no idempotency guarantees. The tool is essentially a mock implementation with no real backend, shipping with mock data is a red flag for production use. Tool description lacks WHEN to use it and any prerequisites.
Synchronizes merchant inventory and searches for products matching a query with optional minimum stock filtering
Tool name combines two actions ('sync' and 'search'), violates single-responsibility principle. Should be split into two tools: 'sync_inventory' and 'search_products'.
Tool description is only 109 characters and lacks actionable detail. Missing: WHEN to use (vs other search approaches), what 'merchant_id' scope means, whether sync is automatic or manual, what happens if sync fails. LLM-optimized descriptions should be 50-200 chars with clear purpose and context.
Parameter 'merchant_id' lacks format/constraint documentation. Is it an opaque UUID, a domain name, a numeric ID? Description just says 'unique identifier', ambiguous. Should specify: format (e.g. 'merch_[0-9]{4}'), example, and whether human-readable names (e.g. merchant display name) are accepted.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 36 | 2026-07-28+ | v2 |
Output schema not formally documented. Code reveals return structure (structured_catalog array with ucp_sku, title, price_usd, stock_count, availability; protocol_version string), but no JSON Schema declaration or type contract visible in the tool registration. LLMs cannot plan downstream calls without knowing output shape.
No result limiting or pagination declared. If mock_inventory grows to 10,000 items, all matching results are returned without limit. Real inventory search tools must cap results (e.g. 50) and offer cursor/offset pagination to prevent context window exhaustion.
No error handling or recovery guidance. Code returns mock data for all inputs, no validation that merchant_id exists, that query is non-empty, or that min_stock >= 0. If sync_and_search fails (e.g. invalid merchant_id), the LLM receives nothing to act on. Should return structured errors with recovery hints.
Tool is implemented as mock/stubbed code returning fixed test data. No real integration with a merchant database, inventory system, or search backend. Shipping mock tools as a 'skill' is unsuitable for production, agents cannot perform real commerce operations.