Loading ToolBench
This is a test fixture server with 10 well-defined tools. All tools have clear descriptions (34 - 72 chars, within baseline), proper JSON Schema input definitions with typed parameters, and appropriate risk annotations. Naming follows verb_noun convention (echo, calculator, kv_put, kv_get, slow, fail, large_result, duplicate, risky_delete, resource_update). However, parameter descriptions are entirely absent, the schema shows types and constraints but no human-readable guidance for LLMs. Output schemas are not documented. Error handling is minimal (fail tool returns a message but no recovery guidance). The server is well-structured for testing but lacks production-grade parameter documentation and output schema clarity.
Performs deterministic two-number arithmetic.
Returns this server instance ID to test cross-server name collisions.
Returns the supplied text unchanged.
Returns a tool-level error without failing the JSON-RPC exchange.
Reads a value from the fixture's in-memory key/value map.
Stores a value in the fixture's in-memory key/value map.
Returns a deterministic large text result.
Parameter descriptions entirely absent. Schema defines types and constraints (minLength, enum, minimum, maximum) but LLMs cannot read JSON Schema, they rely on description text. E.g., 'delayMs' in slow tool has no description explaining its purpose or valid range.
Output schemas not documented. Tools return results but LLMs have no schema to understand response structure. E.g., calculator returns a number, but is it wrapped in an object? What fields does kv_get return?
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2025-06-18+ | v2 |
Advances the fixture resource revision and notifies subscribers.
A non-destructive fixture that advertises destructive risk annotations.
Waits while emitting progress and logging notifications.
Error handling lacks recovery guidance. 'fail' tool returns a message but does not tell the LLM whether to retry, ask the user, or abandon the operation. No error classification (retryable vs fatal).
Tool 'duplicate' has vague description ('Returns this server instance ID to test cross-server name collisions') and unclear purpose. LLMs cannot determine when to call it or what to do with the result.
Tool 'risky_delete' advertises DESTRUCTIVE risk but is non-destructive (fixture). This mismatch trains LLMs to ignore risk annotations. Either make it truly destructive or remove the annotation.