Loading ToolBench
Read-only, governed security-operations MCP servers for defensive and offensive security automation. Includes reconnaissance, vulnerability scanning, log analysis, intrusion detection, host auditing, and kill-chain orchestration.
Significant gaps across naming, descriptions, and schemas. Tool names lack consistent verb prefixes (audit_host, scan_intrusions, scan_window are acceptable; recon, vuln_scan, detect, respond lack clarity). Most tools have descriptions (good), but many are vague or lack parameter details. Input schemas present but sparse: most tools have minimal or empty parameter objects. No output schemas documented. Error handling guidance absent. Composition is reasonable (tools are single-purpose), but parameter descriptions are inconsistent. This is a typical community-grade server with foundational issues.
Run read-only security-posture checks on THIS machine (Windows or macOS): firewall, disk encryption, antivirus / real-time protection, UAC, listening ports, SIP. Returns findings tagged pass/warn/fail/unknown with a summary. Query-only; never changes the host.
Scan phantom-mesh's own agents.toml for plaintext API keys, weak cluster_secret, and exposed (0.0.0.0) listeners. Returns findings without echoing secret values.
Blue detection stage. Ingest lab logs, match anomalies, and triage into prioritized alert groups. Independent of the red stages.
Red recon stage. Run nmap (or canned data in mock mode) against an in-lab service and record open ports. FIRST step of the kill-chain.
Closeout stage. Compose the prose-only exploit suggestions (never a runnable PoC), correlate the triaged alerts into actor narratives, and write the pentest + incident reports with the MTTD metric. Requires vuln_scan and detect to have run first.
Host intrusion detection: read recent Windows event-log entries (PowerShell Operational, System) and match Sigma-style rules for attacker behaviour (encoded PowerShell, download-and-execute, AMSI bypass, credential dumping). Returns alerts ordered by level. Read-only.
Tools 'recon', 'vuln_scan', 'detect', 'respond' lack action verbs. Names like 'recon' and 'detect' are vague, LLMs cannot infer whether these read, write, or modify state. Should be 'run_recon', 'run_vuln_scan', 'run_detection', 'generate_response'.
Tools 'detect' and 'respond' have empty input schemas (no properties). LLMs cannot understand what parameters control behavior. If these are stateless, document that explicitly; if they depend on prior tool outputs, add explicit parameters (e.g., 'alert_ids', 'vuln_ids').
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., alert IDs, port lists). Add explicit return type documentation for each tool.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Scan a log file for known attack patterns (sqli/traversal/xss/admin/scanner). URL-decodes each line before matching. Returns alert objects.
Run nmap against an in-lab service (e.g. juice-shop, dvwa). Returns parsed open ports + service versions. Refuses any target that is not a known lab service.
Scan a filesystem path with Trivy for known-vulnerable OS packages and language dependencies. Returns findings ordered by severity (fixable-first) with a summary. Read-only; use it to triage which CVEs actually matter and draft remediation.
Poll the lab access logs (reports/lab-logs/*.log) for known attack patterns (sqli/traversal/xss/admin/scanner) and APPEND matched alerts to reports/lab-logs/alerts.jsonl. Designed to be called repeatedly by the blue-log-anomaly agent. Returns the count of alerts emitted and the journal path.
Red vuln-scan stage. Run nuclei against the HTTP endpoints found by recon. Requires recon to have run first.
Parameter descriptions are inconsistent. 'max_events' and 'max_alerts' in scan_intrusions are well-described; 'severity' in vuln_scan lacks format guidance (comma-separated? enum?). Standardize all parameter descriptions to include format, range, and valid values.
No error handling guidance. Tools like 'scan_target' refuse unknown targets but do not explain recovery (e.g., 'Call list_lab_services() to see available targets'). Add actionable error messages and recovery hints.