Loading ToolBench
A Rust SDK for building Model Context Protocol servers and clients with support for tools, resources, and prompts
This is a Rust SDK framework, not a concrete MCP server. The evaluation focuses on the example servers (calculator and counter) provided. Tool definitions are present with schemas and descriptions, but lack sophistication expected of production tools. Descriptions are minimal (10-40 chars), parameter descriptions are absent or incomplete, error handling is not visible, and output schemas are not documented. The framework itself shows good architectural intent (async-trait, schemars integration, macro-based tool definition) but the examples don't demonstrate best practices. Most tools fall in the 35-50 range.
Perform basic arithmetic operations
Decrement the counter by a specified quantity
Get current value of counter
Increment the counter by a specified quantity
Parameter descriptions missing. Input schema shows type information (x: i32, y: i32, operation: string) but no parameter descriptions explaining the meaning, constraints, or expected format of 'operation' values. The 'operation' parameter accepts free-form strings instead of an enum.
Tool descriptions too short and vague. 'Perform basic arithmetic operations' (36 chars) and 'Increment the counter by a specified quantity' (46 chars) lack context on when to use them or what they return. Baseline: 194 chars average for A+ tools; p10=34, p90=392. These fall at p10.
No output/return schema documentation. Tools define input schemas via JSON Schema, but no description of what fields are returned or what the output structure looks like. LLMs cannot plan downstream tool calls without knowing the response shape.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 43 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 40 | 2024-11-05+ | v1 |
operation parameter accepts free-form string instead of enum. The 'calculator' tool's operation parameter should declare valid values (add, subtract, multiply, divide) as an enum constraint, not as part of the description. Free-form strings invite hallucinated values from LLMs.
No error handling guidance. Responses for invalid inputs (e.g., divide by zero, invalid operation) are not documented. LLMs need to know what happens on error and how to recover.
quantity parameter in increment/decrement lacks description. The u32 type is declared but no description explains the expected range, what 'quantity' means semantically, or whether there are bounds (e.g., max 1000).