Loading ToolBench
Offline, source-backed and interactive laboratory for learning, simulating and designing AI systems. Includes micro reasoning lab with tool execution, model architectures, test-time compute, MoE, RAG, and tools.
Two tools with minimal descriptions (both ~100 chars, below baseline 194 chars). LOOKUP and CALC have basic parameter schemas but lack depth. No error handling guidance, no output schema documentation, no tool annotations. Parameter descriptions are terse. No evidence of idempotency hints, permission gates, or recovery guidance. Naming is acceptable (verb-noun style) but descriptions do not explain WHEN to use each tool or what downstream tools expect.
Perform a modular arithmetic calculation. Supports ADD, SUB, or MUL operations on two operands in range 0-9, returning result modulo 10.
Look up a variable value from the facts dictionary. Variable must be one of A-H and the stored value must be in range 0-9.
Tool descriptions are too brief (100 chars vs baseline 194 chars). LOOKUP and CALC lack context on WHEN to use them, what they return, and how they chain with other tools.
No output schema documented. LLMs cannot infer what LOOKUP and CALC return (type, structure, range). Agents cannot plan downstream calls without knowing return types.
Parameter descriptions are minimal. 'Variable name (A-H)' and 'Operation: ADD, SUB, or MUL' lack format guidance, constraints, and error cases. No mention of what happens if invalid input is passed.
No error handling guidance. If LOOKUP references undefined variable or CALC receives out-of-range operands, no recovery path is documented. LLMs cannot self-correct.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
No tool annotations (readOnlyHint, idempotentHint). Both tools are marked READ_ONLY in metadata but this is not exposed in schema or description for LLM visibility.