Loading ToolBench
Deterministic math tools for small language models via MCP
ReasonForge provides 5 tools across two servers (Math: 4 tools, Code: 1 tool). All tools have descriptions and input schemas with parameter types. However, several description lengths fall below the 50-200 character LLM-optimized baseline, parameter descriptions are sparse or missing context, and output schemas are not documented. Error handling does not include recovery guidance. Tool names follow verb_noun convention but lack specificity in some cases (e.g., 'math_tool', 'code_tool' are generic). The math tools are well-structured with detailed operations documented, but descriptions are dense and parameters lack per-parameter context. The code_tool accepts arbitrary code execution with sandboxing but lacks detailed error recovery guidance.
All-in-one calculus tool. Use for calculus. Operations: differentiate, integrate, limit, series, summation, partial_fraction, trigsimp, ode_solve, laplace. Expression is the FULL equation in SymPy syntax. For ode_solve, use 'f' as the function name (e.g. f'' - f = exp(x)).
Sandboxed Python code execution tool. Use for running Python code, syntax checking, and code structure inspection. Operations: run (execute code), check (syntax-only), ast_inspect (structure analysis). Code is executed in an isolated subprocess with timeout and blocked dangerous imports.
All-in-one algebra + number theory tool. Use for arithmetic, algebra, number theory. Operations: compute, solve, simplify, factor, expand, gcd, lcm, prime_factors, divisors, mod_inverse, nsolve, crt. 'compute' supports sympy functions: totient, fibonacci, catalan, bell, factorial, binomial, lucas, mobius, isprime, nextprime.
All-in-one matrix tool. Use for matrix math. Operations: determinant, inverse, eigenvalues, eigenvectors, rank, rref, transpose, multiply, add, trace, nullspace, columnspace, charpoly, norm, adjugate, solve (Ax=b). Pass matrix as [[row1],[row2]].
All-in-one statistics tool. Use for statistics. Operations: describe, mean, median, mode, std, variance, correlation, regression, percentile, zscore, skewness, kurtosis, geometric_mean, harmonic_mean. Pass data as array of numbers.
Tool names lack verb-specificity. 'math_tool', 'calculus_tool', 'matrix_tool', 'statistics_tool' are overly generic and do not convey action. LLMs cannot infer the single primary operation from these names alone. Should be 'compute_math', 'differentiate_equation', 'solve_matrix', 'compute_statistics' to clarify primary intent.
Parameter descriptions are missing or trivial for most parameters. For example, 'domain' in math_tool has no description (only an enum list). 'variable' defaults to 'x' but the description does not explain when to override it. 'operation' enum values are listed but not described (e.g., what does 'mod_inverse' do? When use 'crt'?). LLMs cannot disambiguate without per-parameter context.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 47 | - | v1 |
Output schemas are not documented. Tools return results but there is no specification of what fields are returned, what type each field is, or what the agent should expect. LLMs cannot plan downstream tool calls or extract needed data without knowing the response structure. E.g., does math_tool return {result: string} or {result: number, steps: string}?
Error handling lacks recovery guidance. No indication of what the LLM should do if a tool call fails. No distinction between retryable errors (e.g. timeout) and user-fixable errors (e.g. invalid syntax). No hints to guide the agent toward correction.
code_tool description is vague ('Sandboxed Python code execution tool') and does not explain which operations are safe vs dangerous. 'Blocked dangerous imports' is mentioned but not listed. The description should clarify: What is blocked? What timeout defaults apply? What happens if code hangs?
matrix_tool parameter 'matrix_b' description says 'Second matrix for binary operations' but does not specify which operations accept this parameter. 'solve' operation requires matrix_b but this dependency is not documented. Undocumented parameter relationships cause silent misuse.
statistics_tool has 'data_y' and 'percentile_value' parameters but no indication of which operations use which parameters. 'correlation' and 'regression' need 'data_y' but 'describe' does not. LLMs will pass extraneous parameters if dependencies are not explicit.
Tool descriptions do not include use-case guidance. When should an LLM choose 'calculus_tool' over 'math_tool'? The descriptions list operations but do not explain the distinction or help the agent select the right tool. This forces the LLM to guess.