Live-runtime security and stress testing for MCP servers — adversarial payloads, load & concurrency, latency, auth bypass. SARIF output.
MCPSafe is a security testing tool, not a production MCP server. Its 4 tools (echo, add, get_file, multiply) are mock fixtures in tests/fixtures/mock_server.py, not the actual server implementation. Tool definitions are minimal: descriptions are present but generic (10-50 chars), input schemas are basic JSON Schema with type and description per parameter, but lack constraints (enums, min/max, patterns). No output schemas documented. No error handling guidance. The tools are READ_ONLY and simple, but lack the depth expected of production tools. Naming is clear (verb_noun pattern), but descriptions do not explain WHEN to use each tool or what downstream operations expect.
Add two integers and return the sum.
Echo the message back unchanged.
Read a file by path.
Multiply two numbers.
Tool definitions are mock fixtures, not production server implementation. Descriptions are generic and under 50 chars; lack context on WHEN to use each tool or what downstream operations expect.
No output schemas documented. LLMs cannot infer what fields to expect from tool responses, blocking downstream tool chaining and forcing guesswork.
Input schemas lack constraints: no enums, min/max bounds, or regex patterns. Numeric parameters (a, b, x, y) have no range limits; file path has no validation.
No error handling guidance. Tools do not document what happens on invalid input (e.g., get_file with non-existent path, add with non-integer) or how to recover.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
get_file tool exposes file system access without documented security boundaries. No mention of path traversal prevention, allowed directories, or permission checks.