Loading ToolBench
A native Zig implementation of the Model Context Protocol (MCP), an open standard by Anthropic for connecting AI applications to external systems
The server has 8 tools with basic definitions, but quality is inconsistent. All tools have short, minimal descriptions (10-30 chars) that fall below the 34-char baseline for production quality. Input schemas are present and properly typed, but lack depth. Naming is mostly verb_noun compliant but generic (add, subtract, multiply, divide, greet, echo are overly simple). Parameter descriptions exist but are terse. No output schemas documented. No error handling guidance visible. The framework provides ToolBuilder and helper utilities (getString, getFloat, etc.) showing good architecture, but example tools lack sophisticated error classification and recovery guidance expected at 70+.
Add two numbers
Divide two numbers
Echo back the input message
Get weather alerts for a US state
Get weather forecast for a location
Greet a user by name
Multiply two numbers
Subtract two numbers
All tool descriptions are under 34 characters (baseline p10 is 34 chars). Descriptions lack context for LLM selection, they do not explain WHEN to use the tool, WHAT the output is, or WHY it exists. For example, 'Subtract two numbers' does not convey whether this is for math, financial calculations, or metric deltas.
No output schemas documented. The ToolResult struct supports structured content (text, image), but examples do not define what fields or data structure each tool returns. LLMs cannot plan downstream calls or extract referenced IDs without knowing the response schema.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
No error handling guidance visible in tool definitions. The ToolError enum defines InvalidArguments, ExecutionFailed, PermissionDenied, etc., but examples do not show how recovery messages are constructed. An LLM receiving an error gets no guidance on whether to retry, ask the user, or abandon the goal.
Tool names are generic and lack context. 'add' and 'subtract' are ambiguous, do they apply to floats, integers, currency, durations? 'greet' and 'echo' provide no indication of use case. 'get_alerts' and 'get_forecast' are slightly better (verb_noun), but 'get_alerts' does not explain why it is weather-specific.
Parameter descriptions are terse (typically 2-5 words). The rubric baseline expects 72 chars average per parameter annotation. For example, 'First number to add' is clearer than 'First number', but does not specify data type constraints, acceptable range, or format.
divide tool has no explicit handling for division by zero, NaN, or Infinity. The error handling section of ToolError exists but no examples show how to return actionable errors like 'Cannot divide by zero' vs a generic ExecutionFailed. Agents need to know the constraint violated.
get_forecast and get_alerts accept latitude/longitude and state_code respectively, but do not document acceptable ranges. For latitude, the valid range is -90 to +90; for longitude, -180 to +180. Without these constraints, LLMs may pass invalid coordinates. State code is undescribed, is it ISO 3166-2, FIPS code, or abbreviation?