Loading ToolBench
A standalone MCP (Model Context Protocol) server that exposes tools, resources, and prompts. Reuses tool implementations from the main FastAPI application to provide web search, calculator, and other capabilities to external MCP-compatible clients.
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. web_search lacks descriptions for 'max_results' parameter; calculator lacks any parameter descriptions. Neither tool documents output structure or error recovery guidance. Naming is acceptable (verb_noun pattern), but descriptions are minimal (10-34 chars, below the 50-200 char LLM-optimized baseline). No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Error handling returns generic dicts without actionable recovery guidance.
Evaluate a basic arithmetic expression, e.g. '2 * (3 + 4)'.
Search the internet for current information and return a list of results.
Parameter descriptions missing or incomplete. 'max_results' in web_search has no description in TOOL_SPECS (only in function signature). 'expression' in calculator has no description in TOOL_SPECS.
Tool descriptions are too short (10-34 chars vs. 50-200 char baseline). 'Search the internet for current information and return a list of results.' (67 chars) lacks WHEN to use it, prerequisites, or output structure. 'Evaluate a basic arithmetic expression, e.g. '2 * (3 + 4)'.' (60 chars) includes example values (anti-pattern) instead of constraints.
Output schemas not documented. LLMs cannot plan downstream calls or extract fields without knowing response structure. web_search returns {results: [{title, url, snippet}], note?}; calculator returns {result: float} or {error: string}. Neither is declared in tool metadata.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | 2026-07-28+ | v2 |
Error handling lacks recovery guidance. calculator returns {error: 'Could not evaluate expression: ...'} with no hint on how to fix it. web_search returns {note: '...'} for missing API key but no actionable next step.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Both tools are read-only and idempotent, but this is not declared in the Tool schema. LLMs cannot infer safety properties without explicit hints.