Declarative, language-agnostic test runner for MCP servers. Test any Model Context Protocol server with plain YAML — no SDK, no boilerplate.
mcpt-runner is a test runner tool, not a production MCP server. The demo-server (examples/demo-server/server.js) exposes 3 simple tools with basic schemas and descriptions. All tools have input schemas with typed parameters and descriptions present. However, descriptions are minimal (10-50 chars), parameter descriptions lack detail on constraints/formats, and no output schemas are documented. Error handling is present but generic. The tools are READ_ONLY and trivial (add, echo, weather), so composition and chaining concerns are moot. This is a reference implementation, not production-grade.
Add two numbers and return the sum.
Echo a message back, uppercased on request.
Return canned weather data for a city, as JSON.
Descriptions are too brief (10-50 chars). Rubric baseline is 194 chars avg; these lack context on when/why to use each tool and what they return.
Parameter descriptions are missing or minimal. 'city' in get_weather has no description; 'uppercase' in echo lacks guidance on default behavior.
No output schemas documented. LLMs cannot infer what fields to expect from responses (e.g., get_weather returns JSON with city/temperature/unit/condition, but this is not declared).
Error messages are generic ('a and b must be numbers', 'unknown tool'). No recovery guidance or actionable next steps for LLMs.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2025-06-18+ | v2 |
Tool 'add' naming is weak. Verb-noun pattern is missing; 'add' alone is ambiguous (add to what? return sum?). Better: 'sum_numbers' or 'calculate_sum'.