Loading ToolBench
MCP server that provides tools to echo text and calculate magic matrices via an external MATLAB service
Two tools with minimal descriptions and incomplete parameter documentation. 'echo' is trivial (10 chars description). 'calculate_magic_matrix' has a 24-char description and lacks parameter constraints (no min/max for numeric input). Neither tool documents output schema structure. No error handling guidance for LLMs. Parameter descriptions exist but are sparse. No tool annotations (readOnlyHint, etc.). Naming is acceptable but descriptions fall well below the 50-200 char baseline for A+ tools.
Calculate Magic Matrix
Echo the input text
Tool descriptions are critically short (10-24 chars). Baseline for A+ tools is 50-200 chars. LLMs cannot determine when/why to select these tools.
Output schemas are not documented. 'calculate_magic_matrix' returns a dict with fields (magic_square, rows, cols, raw, error) but the tool definition does not declare this structure. LLMs cannot plan downstream calls or extract fields reliably.
Numeric parameter 'in_param' lacks constraints. No min/max bounds documented. LLMs may pass absurd values (negative, zero, very large) that break the MATLAB service or cause timeouts.
Error handling does not guide LLM recovery. 'calculate_magic_matrix' returns generic error dicts without actionable next steps. E.g., 'Invalid parameter: X. Must be a number.' does not tell the LLM whether to retry, ask the user, or call a different tool.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | 2026-07-28+ | v2 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Both tools are marked READ_ONLY in metadata but this is not reflected in the tool definition itself, limiting agent reasoning about safety and retry behavior.