Protocol-aware load tester and benchmarker for MCP servers. A concurrent JSON-RPC 2.0 load tester that fires requests against stdio or HTTP/SSE MCP servers, measures per-tool latency distributions, throughput, and error rate.
mcpbench is a test harness, not a production MCP server. All 8 tools have minimal descriptions (10-50 chars, well below the 194-char baseline). No input schemas are visible in the code, tools parse JSON manually without formal schema registration. Tool names lack action verbs (ping, echo, slow, crash are generic). No parameter descriptions exist. No output schemas documented. Error handling is absent (bad_json writes malformed output; crash exits abruptly). This is intentional: the server is designed to stress-test MCP clients, not to model production tool quality.
Writes a malformed JSON line (no id)
Returns a large string of params.size repeats of 'a'
Exits with code 7
Returns params unchanged
Returns a JSON-RPC error
Prints a line to stderr then replies
Returns {"ok":true}
Waits params.ms before responding
No formal input schemas. All 8 tools parse JSON manually without JSON Schema registration. Tools like 'slow' and 'big' accept parameters but have no schema definitions visible.
Descriptions are trivial (10 - 50 chars, far below 194-char baseline). Examples: 'Returns {"ok":true}' (18 chars), 'Exits with code 7' (17 chars). No guidance on when/why to call each tool.
Tool names lack action verbs. 'ping', 'echo', 'crash', 'slow' are generic and do not signal intent. Production tools should use verb_noun (e.g., 'test_latency', 'simulate_crash').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 20 | 2026-07-28+ | v2 |
No parameter descriptions. 'slow' accepts 'ms' and 'big' accepts 'size', but neither parameter has a description explaining units, range, or constraints.
Error handling is hostile to agents. 'bad_json' writes malformed output; 'crash' exits with code 7; 'errorMethod' returns a generic JSON-RPC error. No recovery guidance or actionable error messages.