Loading ToolBench
Symbolic mathematics computation server providing exact algebraic and calculus operations through the Model Context Protocol
Arithma MCP Server provides 3 well-named mathematical verification tools with structured schemas and clear parameter descriptions. Tool names are action-oriented (verify_chain, verify, equivalent) and follow the verb_noun pattern. Input schemas are properly defined with JSON Schema types and descriptions. However, there are significant gaps: (1) descriptions are sparse and lack context about WHEN to use each tool and WHAT differentiates them; (2) no output schemas are documented, callers must infer structure from test code; (3) error handling is minimal, no guidance on recovery or parameter validation; (4) parameter relationships (e.g., variable field required only when relation='derivative_of') are undocumented; (5) no pagination, batching, or composition helpers despite this being a mathematical verification service that agents might call in loops. The test code reveals the actual output structure (result_status, steps array with mechanism/verdict fields), but this is not formally documented as part of the tool contract. The descriptions provided are too brief (20-40 chars) to guide LLM selection between verify_chain, verify, and equivalent.
Check if two expressions are mathematically equivalent, handling domain restrictions
Verify that two expressions are mathematically equivalent
Verify a chain of mathematical transformations step-by-step with support for multiple relation types (equals, derivative_of, etc.) and assumptions
No tool output schemas documented. Tests reveal result_status with nested verdict, mechanism, steps array, and counterexample fields, but this structure is not contractually documented. LLMs cannot plan downstream operations without knowing what fields to expect.
Tool descriptions are too brief (25-130 chars) and lack differentiation. LLM cannot understand when to select verify_chain vs verify vs equivalent. All three operate on expressions, but their exact distinctions (chain vs pair, multi-step vs single, domain handling) are not explained.
Parameter dependency undocumented: verify_chain's 'variable' field is only relevant when relation='derivative_of', but this constraint is not stated in parameter descriptions. LLMs may pass variable for non-derivative relations or omit it when required.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
No error handling guidance. Tests show counterexample objects on failure, but no documentation of what errors can occur (unparseable LaTeX, undefined variables, type mismatches), whether they're retryable, or what the LLM should do. Tests confirm failures are handled, but users cannot read test code.
No batch variants. Agents verifying multiple expression chains or equivalences must call the tool sequentially, wasting latency and token overhead. A batch_verify_chains tool would be more efficient.
Assumptions parameter (verify_chain) is poorly described. 'Domain assumptions mapping variables to constraint lists (e.g. {"x": ["positive"]})' tells LLM what structure to pass, but not WHY or WHEN to use it, what constraint values are valid, or how assumptions affect verification (e.g., does x > 0 enable algebraic simplifications?)