Loading ToolBench
MCP server: parse and roll dice-notation expressions (1d20+5, 4d6, 2d10*3).
Two simple, read-only tools with clear names and basic schemas. Both tools have descriptions and input schemas with type definitions. However, parameter descriptions are minimal (under 20 chars each), and output schemas are not documented. No error recovery guidance. Tool names follow verb_noun pattern (roll, parse) but descriptions lack context on when to use each tool or what distinguishes them. Schemas present but sparse, no constraints, ranges, or format guidance for the 'expression' parameter.
Parse a dice expression without rolling. Returns the components.
Roll a dice expression (1d20, 4d6+2, 3d8*2). Returns each die plus total.
Parameter descriptions missing or trivial. The 'expression' parameter in both tools has a 60-char description but lacks format constraints, valid examples, or error cases (e.g., 'Dice notation: NdM±modifier*multiplier, e.g. 1d20, 4d6+2, 3d8*2. Sides must be 2 - 10000, count 1 - 1000.').
Output schemas not documented. LLMs cannot plan downstream operations or extract fields without knowing the structure of RollResult or ParsedExpr. Responses are JSON but the schema is invisible to the agent.
No input validation or error recovery guidance. Errors like 'dice count must be in [1, 1000]' are thrown but not surfaced in tool descriptions. LLMs cannot anticipate constraints and will retry blindly without understanding what went wrong.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
Tool descriptions do not explain when to use 'roll' vs 'parse'. Both accept the same parameter; the distinction (one randomizes, one does not) is buried in the description text. LLMs may pick the wrong tool.
No input constraints in schema. The 'expression' parameter is a bare string with no pattern, minLength, maxLength, or enum. LLMs can pass arbitrary strings; validation happens at runtime and returns opaque errors.