A security tool for AI agents, CLI tools, and MCP servers that scans for permission risks, evaluates security threats, and provides sandboxed execution environments
AIAgentGuard demonstrates severe definition quality issues across nearly all dimensions. While tool naming follows verb-noun convention (scan, fix, run, init, report), the majority of tools lack essential metadata. Of 21 tools, only 7 CLI tools have documented input schemas in the source code; 14 API tools have schemas inferred from backend code but no explicit MCP tool registration visible. Descriptions exist but are inconsistent in depth and clarity. Critical gaps: (1) no tool descriptions contain explanation of WHEN to use each tool vs alternatives; (2) parameter descriptions are sparse or missing depth guidance; (3) output schemas are completely undocumented, no tool declares what it returns, forcing LLMs to guess response structure; (4) error handling lacks recovery guidance; (5) no tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. The server appears to be a CLI tool with a REST API bolted on, not a native MCP server, tool definitions are inferred from code structure rather than explicitly registered with an MCP transport.
Get security alerts via HTTP API based on current scan results
Get scan duration metrics via HTTP API
Apply security fixes via HTTP API
Get scan history via HTTP API
Get metrics data via HTTP API
Get network scan details via HTTP API
Get detailed process scan information via HTTP API
WebSocket endpoint for real-time scan updates
Execute a security scan via HTTP API endpoint
No documented output schemas for any tool, LLMs cannot infer response structure, forcing them to guess which fields to extract and pass to downstream tools.
14 of 21 tools are API endpoints inferred from HTTP code, not explicitly registered with MCP tool definitions. Tool existence cannot be verified from the source code, only inferred from backend HTTP routes.
Naming exposes transport layer, 'api_*' prefix is implementation detail, not action verb. Breaks naming convention 'verb_noun'. Should be 'scan', 'get_processes', 'apply_fixes', not 'api_scan', 'api_processes', 'api_fix'.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 31 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 38 | - | v1 |
Get scan rate metrics via HTTP API
Get system status via HTTP API
Get historical trend data via HTTP API
Get trend data via HTTP API
Get vulnerability metrics via HTTP API
Apply security fixes automatically based on scan results. Supports auto-fix mode with confirmation prompts, dry-run to preview changes, category-specific fixing, and safe remediation with rollback options
Initialize AgentGuard configuration. Create a default policy configuration file in your home directory or current directory. This configuration file defines security policies for sandbox execution
Generate and display security reports based on scan results. Reports can be output in various formats for different use cases
Execute a command in a sandboxed environment with restricted permissions. This helps prevent AI agents from executing dangerous operations
Scan the current environment or a specific target for security risks. Supports scanning local environment permissions (filesystem, shell, network, secrets), plugin directories for risky code, and specific tools or agents
Analyze historical scan data to show risk trends. Compares current scan results with historical data from audit logs to show improvements or deteriorations in security posture. Supports time range selection, category-specific trends, JSON output for automation, and visual trend indicators
Display the version number of AgentGuard
CLI and API tools are duplicated (scan + api_scan, fix + api_fix, report + api_report missing, trend + api_trends). LLMs waste reasoning cycles choosing between nearly identical tools.
Descriptions lack 'WHEN to use' context and dependency guidance. No description explains when to call 'fix' after 'scan', or that 'run' requires sandboxing to be configured via 'init'. Missing pattern:recovery-guide.
No tool annotations despite explicit risk levels (READ_ONLY, REVERSIBLE, IRREVERSIBLE, WRITE). Missing readOnlyHint, destructiveHint, idempotentHint, LLMs cannot distinguish safe from dangerous tools.
Irreversible operations (run, fix with auto=true, delete-like operations) lack dry-run or confirmation support in descriptions. Pattern:confirmation-request not followed.
Parameter descriptions often missing or too generic. 'category' parameter appears in multiple tools (scan, fix, trend) but is described identically, missing context-specific guidance.
No error recovery guidance. Descriptions don't explain what to do if scan fails, if fix cannot auto-fix, or if run rejects input due to prompt injection. Missing pattern:recovery-guide.
Source code does not show explicit MCP server implementation or tool registration. Only HTTP routes and CLI commands visible. Cannot verify that tools are registered as MCP resources with proper metadata.