Loading ToolBench
An AI customer support assistant that combines RAG (Retrieval-Augmented Generation) with product search capabilities using MCP tools. Routes user queries to specialized agents for billing, orders, technical support, policy, and product search.
Single tool 'search_products' has a clear verb-noun name and comprehensive parameter set with type annotations. However, the tool description (89 chars) falls below the 194-char baseline and lacks WHEN/WHY guidance. Parameter descriptions are present but generic (e.g., 'Product model name to search for' lacks format/constraint details). No output schema is documented, the code returns raw product objects from PRODUCTS list, but the LLM has no visibility into the structure. No error handling guidance (what if products.json is missing? what if no results match?). The tool is read-only and safe, but lacks production-grade polish for agent reasoning.
Search products by various filters including model name, brand, wireless/wired, price, color, bluetooth version, water resistance, noise control, compatible devices, form factor, and features.
Output schema not documented. Code returns raw product objects from PRODUCTS list, but LLM cannot see the structure of returned fields (model_name, price_inr, brand_name, etc.). This forces the LLM to guess what data is available and risks hallucinated field access.
Tool description is 89 characters, below the 194-char baseline. It states WHAT the tool does but omits WHEN to use it and any prerequisites. Lacks guidance on result limits, pagination, or filtering strategy.
Parameter descriptions are generic and lack format/constraint details. E.g., 'Product model name to search for' does not explain case sensitivity, substring vs exact match, or valid length. 'Maximum price in INR' lacks min/max bounds. No enum constraints for categorical params like 'noise_control', 'water_resistance', 'form_factor'.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
No error handling or recovery guidance. Code does not validate inputs, handle missing products.json, or guide the LLM on what to do if no results match. A 500 error or empty result set leaves the agent without actionable next steps.
No result limits or pagination. The tool returns all matching products without cap. If 1000 products match, the response bloats the context window. Baseline pattern requires limit/offset and total count.