Loading ToolBench
Model Context Protocol server exposing the surface-audit web security scanner to LLM agents. Provides tools to run security audits against URLs with configurable checks, list available checks and report formats, and render scan reports in multiple formats.
Four tools with complete JSON schemas and descriptions. Naming follows verb_noun pattern (scan, list_*, render_*). Descriptions are adequate (50-150 chars) but lack LLM-optimized guidance on when to use each tool vs alternatives. Parameter descriptions present but sparse, many lack format/constraint details. Output schemas not documented; LLMs cannot predict response structure. Error handling mentions allowlist denial but lacks recovery guidance. No tool annotations (readOnlyHint present in metadata but not in schema). Composition is sound: tools are single-responsibility and chainable (scan → render_report). Security: allowlist gating is good, but no audit logging visible.
List every check registered in this environment.
List every report format registered in this environment.
Render an existing scan-report dict in the requested format. Useful when the LLM has the scan JSON and wants Markdown/SARIF/HTML output without re-running the scan.
Run a surface-audit scan against ``url`` and return the report. Findings map to OWASP Top 10 (2021) categories where applicable. Args: url: Target URL. Must resolve to a hostname on the server's allow-list. enabled_checks: Optional list of check IDs; when set, only these run. disabled_checks: Optional list of check IDs to skip. timeout: Per-request timeout in seconds. max_concurrency: Maximum number of concurrent HTTP requests. Returns: A JSON-serializable ``ScanReport.to_dict()`` payload, or an ``{"error": ...}`` object if the request was denied.
Output schemas not documented. LLMs cannot predict response structure from scan, list_checks, list_formats, or render_report. Downstream tool calls (e.g., passing scan result to render_report) rely on implicit knowledge.
list_checks and list_formats have minimal descriptions (under 50 chars). LLMs cannot determine when to call these discovery tools vs proceeding directly to scan/render. Add guidance: 'Call this first to see available checks before running scan with enabled_checks/disabled_checks.'
Parameter descriptions lack format/constraint details. 'enabled_checks' and 'disabled_checks' accept arrays of check IDs but do not specify valid ID format, length, or examples. 'timeout' and 'max_concurrency' have defaults but no min/max bounds documented.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
Error handling is minimal. scan returns {'error': ...} on allowlist denial but does not guide recovery. No guidance on retryable vs fatal errors, or how to proceed if a URL is denied.
No tool annotations in schema. readOnlyHint is declared in metadata but not propagated to tool definitions. LLMs cannot infer that all four tools are read-only and safe to call without confirmation.