Loading ToolBench
MCP server that converts any OpenAPI spec into AI-agent-callable tools
Server provides 5 tools with basic schemas and descriptions. However, critical gaps undermine quality: (1) descriptions lack actionable context for LLM selection, they state WHAT but not WHEN or WHY to use each tool; (2) parameter descriptions are minimal (e.g., 'Search term' for query, 'Filter by order status' for status) and lack constraints/formats; (3) no output schemas documented, LLMs cannot predict response structure for downstream chaining; (4) execute tool accepts free-form 'method' and 'endpoint' strings with minimal validation guidance; (5) error handling returns raw API errors without recovery guidance. search() and execute() are generic OpenAPI wrappers (inferred tool definitions, not explicit MCP registration visible), capping their scores at 50. Domain-specific tools (list_orders, store_dashboard, sales_report) are better-scoped but still lack output documentation.
Execute an API endpoint call with authentication. Use `search` first to discover available endpoints and their parameters.
List orders from the store, optionally filtered by status. Returns order id, reference_id, total, status, and date for each order, plus pagination info showing total count.
Generate a sales summary report for a date range. Fetches orders within the specified period and calculates totals, average order value, and status breakdown.
Search available API endpoints in the OpenAPI specification. Returns matching endpoints with path, method, summary, parameters, and request body details so you know how to call them via `execute`.
Get a quick dashboard overview of the store. Returns product count, order count with recent orders, and basic store info — all in a single call. Useful for getting a snapshot of the store.
No output schemas documented. LLMs cannot predict response structure, forcing them to guess field names for downstream tool calls. E.g., list_orders returns 'orders' array but structure of each order object is undocumented.
Parameter descriptions lack constraints and formats. 'date_from' and 'date_to' in sales_report state format is 'YYYY-MM-DD' but no validation guidance if LLM passes wrong format. 'method' in execute accepts free-form string with no enum constraint, LLM could pass invalid HTTP verbs.
Tool descriptions lack WHEN/WHY context. 'Execute an API endpoint call' does not explain when to use execute vs search, or when to call search first. LLMs cannot disambiguate tool selection without explicit guidance.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | <=2025-11-25 | v2 |
Error handling returns raw API errors without recovery guidance. E.g., 'Connection failed: {e}' or 'Request timed out' gives LLM no actionable next step. Should include 'Retry in 5 seconds' or 'Check API token configuration'.
execute tool accepts 'endpoint' as free-form string with minimal validation. No guidance on path parameter syntax (e.g., /orders/{id}). LLM could pass malformed paths. Should validate against OpenAPI spec or provide clear format rules.