Static source inference · medium confidence · detected: Logging
Deprecated protocol patterns detected
Summary
MCP Kali Server has serious definition quality gaps across all dimensions. Tool names lack action verbs and are inconsistent (nmap_scan vs metasploit_run vs hydra_attack). Descriptions are present but minimal (10-40 chars), providing insufficient context for LLM selection. Parameter schemas are partially defined but lack comprehensive type information and constraints. Most critically, the implementation uses Flask with HTTP POST endpoints rather than proper MCP protocol tool registration, tools are inferred from API endpoints rather than explicitly registered with the MCP server. This is a fundamental architectural issue that caps the entire evaluation.
Tools (7)
dirb_scanread onlysource verified45/100
Execute Dirb web content scanner.
gobuster_scanread onlysource verified48/100
Execute Gobuster to find directories, DNS subdomains, or virtual hosts.
Not a valid MCP server implementation. Uses Flask HTTP POST endpoints instead of proper MCP tool registration via fastmcp. Tools are inferred from API routes, not explicitly registered with schemas and descriptions in the MCP protocol.
Tool naming is inconsistent and lacks action verbs. Names like 'nmap_scan', 'gobuster_scan', 'nikto_scan' use _scan suffix but 'metasploit_run' uses _run and 'hydra_attack' uses _attack. Best practice is consistent verb_noun pattern (e.g., scan_nmap, execute_metasploit, crack_password_hydra). LLMs cannot reliably infer intent from inconsistent naming.
Expand tool descriptions to 50-200 characters. Include WHAT (what does the tool do?), WHEN (when should it be called vs similar tools?), and WHAT IT RETURNS (what structured output should the LLM expect?). Example: 'Scan target IP/hostname with Nmap to discover open ports and service versions. Use this first to map the attack surface. Returns list of open ports, services, and versions.'
Add comprehensive parameter descriptions with format specs and constraints. Example for 'target': 'Target IP address (e.g., 192.168.1.1) or FQDN (e.g., example.com). Required for scan to proceed.'
Define enums for categorical parameters. In gobuster_scan, replace free-form 'mode' with enum: ["dir", "dns", "fuzz", "vhost"]. In hydra_attack, define 'service' as enum with common values: ["ssh", "ftp", "http-post-form", "mysql", "postgres", ...].
Document output schemas for all tools. Define return type as object with fields: {stdout: string, stderr: string, return_code: int, success: bool, timed_out: bool, parsed_results: {open_ports: [{port: int, service: string, version: string}]}}. This allows LLM to extract structured data.
Add confirmation/dry-run support for destructive operations. Add optional 'dry_run' parameter to metasploit_run and hydra_attack. Modify hydra_attack to support '--dry-run' as default; require explicit 'confirm_password_cracking: true' to actually attempt credentials.
Spec posture evidence
Inferred effective spec: <=2025-11-25.
Relies on Logging (deprecated) - log to stderr or use OpenTelemetry
Score history
Overall score trend
↑ 9 points across a rubric change (v1 → v2)
41/100
Scored
Grade
Overall
Spec posture
Rubric
2026-09-22
F
41
<=2025-11-25
v2
2026-03-09
F
32
-
v1
Descriptions are extremely brief (10-40 characters) and lack context for LLM selection. E.g., 'Execute Nikto web server scanner.' does not explain when to use it vs other tools, what it returns, or prerequisites. Baseline for A+ tools is 50-200 chars with clear WHAT/WHEN/WHY.
Parameter descriptions are missing or trivial. 'The target URL', 'Service to attack', 'Additional Nikto arguments' lack actionable constraints. Missing: format specs (e.g., 'valid IP or FQDN'), ranges, valid enum values for 'service' param (ssh, ftp, http-post-form vs others). This violates tool-description pattern.
No enums defined for categorical parameters. 'mode' in gobuster_scan lists allowed values in error response (dir, dns, fuzz, vhost) but schema does not enforce them via enum constraint. Similarly, 'service' in hydra_attack accepts free-form string with no validation. Free-form strings invite hallucinated values.
No output schemas documented. Tools return raw subprocess output (stdout, stderr, return_code) without structure. LLM cannot predict what fields are present or their types. Baseline for A+ tools is 100% documented return types.
Destructive and irreversible operations (metasploit_run with risk DESTRUCTIVE, hydra_attack with risk IRREVERSIBLE) lack confirmation or dry-run support. No dry_run parameter, no confirmation_required field. Agents can irreversibly modify systems or crack passwords without safeguards.
Error handling is generic. Server returns JSON with 'error' field but does not classify errors as retryable, user-fixable, or fatal. No guidance on recovery (e.g., 'Try with a different wordlist' or 'Check target is reachable'). Violates recovery-guide pattern.
Security risk: Additional arguments ('additional_args' in all tools) accept arbitrary shell input. While server uses shlex.split() for some commands, 'additional_args' could still enable command injection. No input validation or sanitization against path traversal or malicious payloads.
No rate limiting or timeouts enforced per-tool. While CommandExecutor has COMMAND_TIMEOUT (180s), tools can be invoked repeatedly without rate guards. An agent in a retry loop could launch thousands of nmap scans or Hydra attacks.
metasploit_run accepts 'options' as free-form object (dict) with no schema. LLM cannot know what keys are valid or what types values should have. No description of required vs optional options.
metasploit_run
Implement error classification and recovery guidance. Instead of generic 'error', return: {error_code: 'TARGET_UNREACHABLE', message: 'Target does not respond to ping. Verify target IP/hostname and network connectivity.', recovery: 'Try with -Pn flag (skip ping) or check target is accessible from this network.', retryable: true}.
Validate and sanitize 'additional_args' parameter. Implement whitelist of allowed Nmap/Gobuster/etc. flags; reject unknown flags with clear error. Example: 'Unsupported Nmap flag: -A. Allowed: -sV, -sC, -O, -Pn, -p, -T1-T5. Use only documented flags.'
Implement per-tool rate limiting. Use a token bucket or sliding window to limit nmap_scan to 5 calls/min, hydra_attack to 1 call/min (resource-intensive). Return 429 'Rate limit exceeded' with backoff guidance.
Define metasploit_run 'options' parameter with explicit schema. Replace free-form dict with structured object, e.g., {RHOSTS: {type: string, required: true, description: 'Target IP/CIDR'}, LHOST: {type: string, description: 'Attacker IP'}, LPORT: {type: int, default: 4444}}. Or document a fixed set of common options.
Add state information to responses. When nmap_scan returns open ports, include metadata: {scan_id: 'uuid', target: '192.168.1.1', scan_start_time: '2024-01-15T10:30:00Z', status: 'completed', results: [...]}. This enables followup tools to reference the scan.
Remove 'generic_command' endpoint (/api/command). It accepts arbitrary shell input, major security hole. Forces users down a single flexible-but-dangerous path instead of safe parameterized tools.