Loading ToolBench
MCP server for OAuth/authorization probing and authentication testing of HTTP endpoints. Supports automated OAuth discovery, device flow registration, and credential-assisted scanning.
AuthProbe MCP server has significant definition quality gaps. While tool names follow a clear verb_noun pattern (authprobe_scan_http, authprobe_scan_resume, etc.), descriptions are inconsistent in depth and quality. Input schemas are present for all tools but lack type definitions for several parameters, and output schemas are not documented. Parameter descriptions are often minimal or missing context about valid ranges and constraints. The server lacks proper error handling guidance, confirmation patterns for destructive operations, and clear output schema documentation. Tool names use a prefixed convention (authprobe_*) which is non-standard but acceptable. The tools exhibit moderate composition concerns: authprobe_scan_http and authprobe_scan_http_with_credentials overlap in functionality, differing only in credential handling, which could confuse LLM tool selection.
Bundle scan evidence and outputs into a ZIP file. Returns base64-encoded ZIP if <400KB; otherwise returns file path on server.
Render a stored scan report or provided report JSON as markdown.
Run unauthenticated scan first. Output defaults to markdown. auth_assist defaults to auto (DCR + OAuth device flow); set auth_assist=off to disable.
Scan with pre-provided OAuth credentials or authorization header. Use credential_ref to reference MCP host credentials; falls back to raw authorization_header if provided.
Resume an auth_assist scan. Returns awaiting_user_auth until authorization succeeds, then finishes scan.
Missing output schema documentation. Tools return complex nested objects (status, scan_id, summary, auth_request, auth_assist, etc.) but no schema is documented for LLM consumption. LLMs cannot reliably extract fields or plan downstream calls without knowing expected response structure.
Parameter type definitions incomplete. Headers parameter is typed as array of strings but lacks format/constraint details. timeout_seconds, mcp_mode, and rfc_mode accept string enums but are not declared as such in the schema, they use generic type 'string' instead of enum constraints. This forces LLMs to guess valid values.
authprobe_scan_http and authprobe_scan_http_with_credentials are functionally similar (both scan, only credential input differs). This violates single-responsibility composition. LLMs must reason about when to use each, increasing token waste and error likelihood. Consider merging with optional credential_ref parameter.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | F | 49 | 2025-06-18+ | v2 |
| 2026-03-09 | C | 60 | 2025-11-25+ | v1 |
Descriptions lack actionable context. 'Resume an auth_assist scan' does not explain when to call it, what triggers the need to resume, or what awaiting_user_auth means. Descriptions should state WHAT, WHEN, and WHY for LLM tool selection. authprobe_scan_http description mentions 'auth_assist=auto (DCR + OAuth device flow)' but DCR is undefined in the description.
No error handling guidance. Tools can fail (credential_ref unresolved, scan_id not found, scan failures) but descriptions do not explain how to recover. No indication of which errors are retryable vs. user-fixable. Code shows error returns (e.g., 'unknown scan_id', credential resolution errors) but LLMs have no guidance on recovery.
Parameter descriptions lack constraint details. timeout_seconds has no min/max; headers array format undefined; mcp_mode and rfc_mode accept 'best-effort', 'strict', 'off' but this is only visible in code, not in schema. Descriptions do not state expected formats, ranges, or allowed values. Baseline: 100% of A+ tools document param constraints explicitly.
authprobe_scan_http_with_credentials requires credential_ref OR authorization_header but the relationship is ambiguous in the schema. The code shows credential_ref is 'preferred' and authorization_header is 'fallback', but this dependency is not documented in parameter descriptions. LLMs may pass both or neither without clear guidance.
authprobe_bundle_evidence description states 'Returns base64-encoded ZIP if <400KB; otherwise returns file path on server' but does not explain why size matters or how the client should handle both response types. Inconsistent output structure (base64_zip encoding vs. file path with redacted flag) lacks schema definition.