Loading ToolBench
Security scanner for AI agent tooling — MCP servers, tool definitions, and agentic pipelines
Three tools with well-structured schemas and comprehensive descriptions. All tools have proper JSON Schema definitions with typed parameters and clear descriptions (194 - 280 chars, within baseline 34 - 392 range). Tool names follow verb_noun pattern (audit_*, scan_*). Parameters include enums, type constraints, and descriptions. However, output schemas are not documented, LLMs cannot predict response structure. Error handling guidance is absent. No tool annotations (readOnlyHint, destructiveHint). Risk declarations present but not integrated into schema.
Scan all detected MCP configuration files (Claude Desktop and Cursor) for security vulnerabilities in a single operation. Automatically discovers config files in standard OS locations, scans all configured MCP servers in each file, and combines the results into one report. Each finding includes severity, the vulnerable location, a code snippet, and remediation guidance. Use this for a comprehensive security audit across all MCP configurations without specifying individual paths. Use audit_config instead when you only need to scan a specific file or a non-standard config location.
Scan an MCP server configuration file for security vulnerabilities. Use this tool to audit a specific config file path, or let it auto-detect the Claude Desktop configuration when no path is provided. Checks each configured MCP server for: prompt injection patterns in tool descriptions, command injection in server source code, hardcoded secrets in environment variables, and excessive permissions (shell execution, filesystem access). Returns a structured security report with findings grouped by severity (critical/high/medium/low/info). Each finding includes the vulnerable location, a code snippet, OWASP category, and remediation guidance. Use this when auditing a single config file. Use audit_all_configs instead when you want to scan all detected configs at once.
Scan a single MCP server definition for security vulnerabilities without requiring a full configuration file. Provide the server's command, arguments, and environment variables directly to evaluate it before adding it to your MCP config. Performs the same security checks as audit_config but for a single inline server definition. Source file scanning is skipped for speed. Use this when evaluating an unfamiliar MCP server before trusting it, or when testing a server definition that is not yet in a config file. Use audit_config or audit_all_configs when the server is already in a config file.
Output schemas not documented. LLMs cannot predict response structure (severity levels, finding fields, remediation format). Requires reverse-engineering from tool behavior.
No error handling guidance. Tool descriptions do not explain what happens on failure (file not found, permission denied, parse error). LLM cannot self-correct or retry intelligently.
Missing tool annotations. No readOnlyHint, destructiveHint, or idempotentHint in schema. LLMs cannot determine if tools are safe to retry or have side effects.
scan_server parameter 'env' lacks format constraints. Accepts arbitrary object; no guidance on secret detection, validation, or safe values. LLM may pass sensitive data.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
No pagination or result limits documented. audit_all_configs could return hundreds of findings; no guidance on max results, truncation, or chunking strategy.