Loading ToolBench
Security auditor for MCP servers on Archestra. Analyzes MCP server tools for security vulnerabilities including prompt injection, excessive permissions, data exfiltration risks, command injection, PII exposure, and missing input validation.
MCP Guardian is a security auditor for MCP servers with 6 tools. Tool naming follows verb_noun conventions (scan_server, test_server, generate_policy, trust_score, monitor, audit_report), which is solid. Descriptions are present and detailed (150-300 chars), exceeding the 10-char minimum. Input schemas are declared via Zod and properly serialized. However, several critical gaps emerge: (1) Output schemas are NOT documented in code, only inferred from TypeScript function returns; (2) Tool annotations (readOnlyHint/destructiveHint/idempotentHint) are absent despite most tools being READ_ONLY and one being WRITE; (3) Error handling lacks recovery guidance, most tools reference Archestra APIs but provide no fallback behavior if servers are unavailable; (4) Parameters lack validation descriptions in most cases (e.g., 'serverName' just says 'Exact catalog name' but does not specify format, length, allowed character ranges). The code reveals solid security intent but incomplete LLM-optimized definition surfaces.
Generate comprehensive audit reports across all MCP servers or a specific server. Includes vulnerability summary, trust scores, policy coverage, and recommendations.
Auto-generate Archestra security policies (tool invocation and trusted data policies) for an MCP server based on vulnerability scan results.
Monitor real-time MCP tool execution activity. Tracks error rates, call volumes, suspicious patterns, and policy violations. Can alert on anomalies.
Analyze an MCP server's tools for security vulnerabilities including prompt injection, excessive permissions, data exfiltration risks, command injection, PII exposure, and missing input validation. Returns a vulnerability report with trust score.
Auto-generate and run security test cases against an MCP server's tools. Tests include valid inputs, edge cases, malformed inputs, injection payloads, and overflow inputs. Uses LLM analysis via Archestra's proxy to evaluate tool safety.
Output schemas are not documented; return types inferred only from TypeScript function signatures
Tool annotations (readOnlyHint, destructiveHint, idempotentHint) are missing despite clear risk classifications (5 READ_ONLY, 1 WRITE)
Parameter descriptions lack validation rules: serverName accepts 'hyphenated names' but no length limits, character set, or format specification
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
Calculate a comprehensive trust score for an MCP server across 6 security dimensions: tool description safety, input validation, permission scope, data handling, tool integrity, and policy compliance. Returns a 0-100 score with grade (A+ to F) and recommendations.
Error handling lacks recovery guidance. Tools interact with Archestra APIs but do not document fallback behavior or self-correction hints when servers unavailable
Alerting thresholds in monitor tool (errorRate, callsPerMinute, suspiciousPatterns) are defined but no action guidance on what LLM should do when alerts fire
generate_policy tool accepts dryRun=true but code clarity on what 'applying' policies does and whether it is reversible is missing
test_server LLM integration (gpt-5-mini model call) references non-existent model; fallback behavior if Archestra LLM proxy fails is undocumented