Loading ToolBench
Single tool 'calculate' has proper schema registration and basic error handling for division by zero, but suffers from critical gaps in documentation and design patterns. The tool name lacks a clear action verb (should be 'perform_calculation' or 'calculate_math'), parameter descriptions are completely absent, and the output schema is not documented. The description is generic (49 chars, below the 50-200 char production baseline). No recovery guidance in errors, no input validation messaging, and the tool combines multiple operations (add/subtract/multiply/divide) without clear dispatch logic exposed to the LLM. Error handling for division by zero is present but other invalid inputs (non-numeric a/b, invalid operation enum) would fail ungracefully. The composition violates the single-responsibility principle, this should either be split into separate tools or the enum-based dispatch should be more transparent.
Perform basic mathematical calculations
Tool name lacks clear action verb. 'calculate' is generic; 'perform_calculation', 'execute_math_operation', or 'compute' would better signal intent to LLMs. Current name provides minimal semantic clarity for tool selection.
Zero parameter descriptions. The 'operation', 'a', and 'b' parameters have no descriptions explaining what they control, valid ranges, or format constraints. LLMs cannot reliably infer that 'operation' must be one of the enum values without explicit guidance.
Missing output schema documentation. Callers do not know what structure the response will have (e.g., is the result a number, a string, or a complex object?). Documentation must define: response type, field names, and field types.
Tool description is too short and generic (49 characters). 'Perform basic mathematical calculations' lacks context on WHEN to use this tool, what makes it different from alternatives, or its constraints. Production baseline is 50-200 chars with specific guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 43 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 34 | - | v1 |
Single tool combines multiple operations without clear separation. The enum-based dispatch (add/subtract/multiply/divide) couples unrelated concerns. Consider: (1) split into separate tools (add_numbers, subtract_numbers, etc.) or (2) if keeping as one tool, document the enum explicitly in parameter descriptions and add operation-specific error guidance.
Division by zero error lacks recovery guidance. Error message 'Error: Division by zero is not allowed' tells the LLM what went wrong but not what to do next. Should include: 'Cannot divide by zero. If you need to verify b ≠ 0 before calling, check your input data.'
No input validation for non-numeric parameters. If an LLM passes a string for 'a' or 'b', or an invalid operation, the error will be ungraceful. Add explicit validation that returns: 'Invalid input: a and b must be numbers. Got a=[type], b=[type].'