Loading ToolBench
A basic MCP server implementation in Java demonstrating tool registration and server capabilities
Single tool 'calculator' has a valid JSON Schema with typed parameters (operation, a, b as strings/numbers), but critical gaps prevent higher scoring. Tool description is only 16 characters ('Basic calculator'), below the 20-char minimum for any credit. No parameter descriptions exist, LLMs cannot infer what 'operation' accepts or what 'a' and 'b' represent. Schema lacks enum constraints for 'operation' (what operations are valid?), ranges for numeric inputs, or output documentation. Tool implementation returns hardcoded 'Hello world' regardless of input, indicating incomplete implementation. No error handling, validation, or recovery guidance.
Basic calculator
Tool description is 16 characters ('Basic calculator'), below 20-char minimum. Provides no context for LLM selection or usage guidance.
No parameter descriptions. 'operation', 'a', 'b' lack any documentation of expected values, ranges, or constraints. LLMs cannot infer valid inputs.
Schema lacks enum constraint for 'operation' parameter. Free-form string invites hallucinated operations (e.g., 'divide_by_zero'). Should declare valid operations explicitly.
No output schema documented. Tool returns hardcoded 'Hello world' text regardless of input, implementation does not match description. LLMs cannot plan downstream actions.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 36 | <=2025-11-25 | v2 |
No error handling or validation. Invalid inputs (e.g., operation='xyz', a='not a number') will silently return 'Hello world'. No recovery guidance for LLM.