MCP server for SentinelPipe LLM red-teaming and security testing pipeline
sentinelpipe-mcp provides 7 tools with complete JSON schemas and descriptions. Naming follows verb_noun convention (redteam_*) and is action-oriented. Descriptions are present (avg ~100 chars) but lack depth on WHEN to use each tool and dependencies between them. Parameters are typed and constrained (enums, minItems/maxItems), but descriptions are sparse, most params have 1-2 word descriptions. Output schemas are not documented in the source. Error handling is minimal; no recovery guidance visible. Security: tools accept file paths (config_path, artifacts_dir) without visible sanitization against path traversal. Composition is good, tools are single-purpose and chainable (e.g., redteam_run → redteam_compare). No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels (READ_ONLY vs WRITE).
Run multiple SentinelPipe configs and report pass/fail deltas across candidates.
Compare 2-5 historical SentinelPipe runs to spot regressions and gate changes.
Check that the configured target endpoint is reachable and the model appears available.
List built-in SentinelPipe attack packs and preset coverage groups.
List recent SentinelPipe runs from the local artifacts directory.
Preview SentinelPipe scenarios from a run config without executing the target.
Run SentinelPipe against a configured target and return deterministic findings.
Output schemas not documented. LLMs cannot plan downstream calls or extract required fields (e.g., what does redteam_run return? Is it a list of findings with risk_score, description, remediation?). Baseline: 100% of A+ tools document return types.
Parameter descriptions are minimal (1-2 words). E.g., 'config_path' → 'Path to the SentinelPipe run configuration file' lacks format guidance, constraints, or examples. Baseline: avg param annotation 72 chars; these are ~50 chars. LLMs cannot infer valid formats.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Risk levels are declared in metadata but not exposed to LLM via schema. redteam_run and redteam_batch are WRITE operations, LLMs should know these have side effects and may not be retryable.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | 2025-06-18+ | v2 |
Path traversal risk: config_path and artifacts_dir parameters accept arbitrary file paths. No visible validation in source. Agents could be tricked into reading/writing outside intended directories. Requires input sanitization.
No error recovery guidance. Tool descriptions do not explain what to do if config_path is invalid, endpoint unreachable, or model unavailable. Baseline: error responses must tell LLM what to do next (e.g., 'Try redteam_doctor first to verify connectivity').