Loading ToolBench
MCP server harness and sandbox for testing and optimizing MCP server implementations
Whetstone-MCP is a test harness with 15 tools across multiple fixture servers. Most tools have basic descriptions (10-200 chars) and simple schemas, but critical gaps undermine production readiness. Tool naming is inconsistent (three 'lookup' tools, two 'fetch', two 'ratio', two 'limit', ambiguous for LLM selection). Descriptions lack context on when to use each tool vs. similar ones. Parameters are typed but many lack descriptions. Error handling is minimal (bare 'error' strings). No tool annotations (readOnlyHint/destructiveHint). The 'wipe' tool is labeled DESTRUCTIVE but has no confirmation pattern. Output schemas are undocumented. This is a test fixture, not a production server, which explains the gaps, but scored as if it were a real MCP server.
Add two numbers and return the sum.
Return the TOY_TOKEN this server process was started with.
Evaluate one record against the limit for one period, end to end. Use this for any brief that asks whether a record is within its ceiling, or how far from it. It reads the record, converts to watts, reads the period's limit and states the verdict, so no arithmetic is left to the caller. Returns JSON: `figure_w`, `limit_w`, `ratio`, `compliant`, `verdict`. Example: evaluate_record(record="alpha", period="evening")
Get a value for a record.
Approximate reading of a record's figure, rounded to the nearest 100 milliwatts. Legacy. Prefer `lookup` for anything that is compared against a limit.
Return the limit that applies in one period, in watts.
Return the limit.
Duplicate tool names across servers (3× 'lookup', 2× 'fetch', 2× 'ratio', 2× 'limit'). LLMs cannot disambiguate which variant to call. Each server should namespace tools or the harness should expose only one canonical version.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). The 'wipe' tool is labeled DESTRUCTIVE but has no schema-level hint or confirmation pattern. Agents cannot distinguish safe reads from destructive writes.
Error handling is minimal. Tools return bare 'error' strings (e.g., defective_server.py lookup/fetch/limit) with no guidance on what went wrong or what to do next. No recovery hints, no invalid-value echoes, no next-step suggestions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
Look up a stored value by key.
Look up a stored value by key.
Return the exact, unrounded stored figure for one record, in milliwatts. Use this whenever a brief needs a record's figure. Do NOT use `fetch`, which is approximate. Example: lookup(key="alpha") -> "1512.5"
Divide a by b.
Compare a figure against a limit, both in watts. Returns the bare quotient on the first line, then a JSON object stating the verdict: `ratio`, `compliant` (the figure is within the limit), and a one-sentence `verdict`. Relay the verdict; do not re-derive it.
Text carrying a fresh timestamp.
Structured content only.
Delete all stored values.
Output schemas are undocumented. Tools return strings or JSON but the response structure is not declared in tool definitions. LLMs cannot plan downstream calls or extract structured data reliably.
Parameter descriptions are sparse or missing context. 'key' and 'period' parameters lack guidance on valid values, format, or constraints. The 'period' enum in fixed_server is good, but defective_server omits it entirely.