Loading ToolBench
Security scanner for AI agent packages — CLI + MCP server. Scans MCP tool definitions for hidden instructions, unicode tricks, obfuscated payloads, and manipulation patterns.
AgentAudit is a security scanner for MCP servers packaged as an MCP server itself. The 4 tools are explicitly defined in index.mjs with schemas and descriptions. However, definitions have significant gaps: descriptions lack LLM-optimized specificity, parameter descriptions are minimal, output schemas are not documented, and error handling guidance is absent. The tools are READ_ONLY or WRITE operations on security audit data, but descriptions do not explain failure modes or recovery steps. Most critically, parameter types are present but descriptions are sparse or missing context needed for LLM reasoning. This server lands in the 'fair to poor' range due to incomplete schemas and insufficient description depth relative to the patterns baseline (94% of A+ tools have full param descriptions; this server averages 1-2 per tool).
Clone a repo, return source code + audit prompt for LLM analysis
Look up a package in the AgentAudit registry
Find locally installed MCP servers + check registry status
Upload a completed audit report to agentaudit.dev
discover_servers has an empty input schema (no properties). The description 'Find locally installed MCP servers + check registry status' does not explain what the tool returns or when the LLM should invoke it (discovery pattern unclear). Missing output schema documentation.
audit_package description 'Clone a repo, return source code + audit prompt for LLM analysis' does not specify: (1) what 'source code + audit prompt' means in structured form, (2) the format/size of returned data, (3) error cases (invalid repo, clone failure, timeout), (4) how long this is expected to take. LLMs need explicit output schema and failure guidance.
submit_report 'report' parameter has type 'object' with description 'Completed audit report object' but NO schema for the object structure. LLMs cannot validate inputs without knowing required/optional fields, format, or constraints. This violates the 'constrained-input' pattern.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 47 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
All four tools lack documented return/output schemas. The rubric baseline shows 100% of A+ tools document return types. Agents cannot plan downstream operations or handle responses without knowing field names, types, and structure. E.g., does discover_servers return [{name, version, status}] or {servers: [...]}?
Descriptions are 30-80 characters; rubric baseline for A+ tools is 194 chars average (p10=34, p90=392). These descriptions are at the lower end and lack actionable detail. E.g., 'Find locally installed MCP servers + check registry status' does not explain what 'registry status' means or why the LLM would call this tool instead of another discovery method.
No error handling guidance in any tool description. The rubric requires error responses to tell the LLM what to do next ('User not found. Try search_users() first.'). For example, audit_package may fail if a repo is invalid, private, too large, or unreachable, but the description offers no recovery hints.
check_package description 'Look up a package in the AgentAudit registry' does not specify what 'look up' returns (e.g., audit results, security score, list of issues, metadata). This is vague and forces LLMs to guess the output format.
audit_package's repository_url parameter has minimal description: 'Git repository URL to clone and audit'. Should specify: 'Must be a valid Git HTTPS or SSH URL (e.g., https://github.com/user/repo.git). Cloning will fail for private repos without credentials.'