Loading ToolBench
Audit AI applications and agents for security risks before commit, PR, merge, or deployment. Deterministic findings with coverage and evidence receipts.
Server defines 4 tools with clear, well-structured descriptions and explicit use-case boundaries. Tool names follow verb_noun pattern (scan_*, verify_*, check_*). Descriptions are comprehensive (200-300 chars) and include positive/negative use cases. However, input schemas are NOT visible in the provided source, only parameter names and types are inferred from the tool descriptor comments. No output schemas are documented. One tool (check_deploy_security) is explicitly unimplemented. Error handling and recovery guidance are absent from definitions.
Future explicit pre-merge/release/deployment security gate. May orchestrate explicitly applicable security engines. Must disclose: enginesRun, enginesSkipped, and reason for each. Must NOT silently invoke engines merely because they exist. Use for: pre-merge security checks, release readiness, deploy gating. Do NOT use for: individual security scans (use the specific scan tool). Note: This tool is not yet implemented.
Audit AI applications and agents for security risks before commit, PR, merge, or deployment. Produces deterministic findings with explicit coverage, Security Concern Families, Scan Receipts, and Evidence Envelopes. USE when a developer asks to: - "check this for security issues" - "audit this task before I push" - "review this change for security problems" - "scan this AI agent" or "scan this AI app" - "check my LLM application" - "security review this PR" - "is this safe to merge" - "check this before deployment" - "audit this code for vulnerabilities" - "run a security scan" - "check for security risks before commit" - "review this implementation before pushing" Also use for: - reviewing AI/LLM/agent source code for security - validating AI code before merge - checking an agent before deployment - reviewing RAG or model integration security - looking for unsafe AI source-code patterns - checking AI-output-to-dangerous-action flows - checking AI source-code secrets/security controls DO NOT use for: - actual LLM response/content verification (use verify_llm_content) - tenant/RLS/cross-customer isolation (use scan_tenant_isolation) - generic compliance questionnaires or compliance certification - generic non-AI source scanning where no AI check applies This tool performs static analysis only. It does NOT execute target code, make network requests, or emit telemetry. BLOCK findings are advisory — deployment enforcement is handled by check_deploy_security (not yet implemented).
Analyze multi-tenant SaaS or MCP server code for cross-tenant data exposure, missing tenant filters, IDOR, RLS gaps, shared caches, session/credential scoping, vector namespaces, and tenant isolation risks. Use when: tenants, organizations, workspaces, accounts, RLS, tenant-aware databases, shared vector stores, or tenant-aware MCP tools are relevant. Do NOT use for: general source security or LLM content verification.
Input schemas not visible in source code. Tool definitions reference parameters (targetPath, extendedScope, timeout) but no JSON Schema with type constraints, enums, or validation rules are shown. Cannot verify schema completeness.
Output schemas not documented. No description of what scan_ai_security returns (findings structure, evidence format, coverage metrics). LLMs cannot plan downstream actions or extract required fields.
check_deploy_security is explicitly unimplemented ('implemented: false'). Tool is registered but handlers do not exist. Agents will fail if they select this tool.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2025-06-18+ | v2 |
Analyze supplied LLM input or output for prompt-injection indicators, PII exposure, harmful-content patterns, and other LLMVerify-supported risk signals. Use ONLY for actual AI/LLM input/output content. Do NOT use to scan source-code repositories.
No error handling or recovery guidance in tool descriptions. Descriptions do not explain what errors can occur, how to interpret them, or what the LLM should do next (retry, ask user, try alternative tool).
Parameter descriptions are minimal. 'targetPath' has description 'Absolute or relative path to the AI application source code to scan' but lacks constraints (max length, allowed characters, symlink handling). 'timeout' lacks range (min/max seconds).