MCP policy gateway and red-team benchmark harness for security scanning of MCP servers
Critical gaps across all dimensions. Tool descriptions are minimal (10-15 chars), far below the 194-char baseline. No parameter descriptions visible in schema, all params lack explanatory text. Input schemas present but severely incomplete: missing type info, no enums for constrained inputs, no format specs. No output schemas documented. execute_command and write_file are destructive tools with zero error handling guidance or confirmation patterns. No evidence of idempotency, rate limiting, or permission gates. The toy_server.py implementation shows raw tool execution without sanitization, validation, or recovery guidance. This is a red-team harness, not a production-grade MCP server.
Execute a shell command on the host.
Make an outbound HTTP GET request to fetch remote page content.
Query database using SQL string.
Read file contents from local disk.
Write text contents to a file on local disk.
Parameter descriptions completely absent. All 5 tools expose parameters (command, path, content, url, query) with zero explanatory text. LLMs cannot infer intent from names alone.
Tool descriptions are 10-15 characters, far below 194-char baseline. 'Execute a shell command on the host' lacks WHEN to use, prerequisites, or state-modification warnings. Descriptions must answer: what, when, and consequences.
No output schemas documented. Callers cannot know what fields to expect, breaking downstream tool chaining and forcing LLMs to guess response structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
Destructive tools (execute_command, write_file, query_database) lack confirmation patterns, dry-run support, or error recovery guidance. No indication that these operations are irreversible.
No input validation, sanitization, or error classification. execute_command passes raw shell strings without injection guards. query_database exposes SQL injection risk. No recovery guidance in error responses.