Loading ToolBench
MCP server gateway wrapping F*-extracted security tools with Tailscale identity-based authentication, policy enforcement, metering, and credential brokering
Critical gaps across all dimensions. Tool definitions exist but lack rigor. Descriptions are present but minimal (10-60 chars, well below 50-200 char baseline for LLM optimization). Input schemas visible for 3/5 tools but severely incomplete: credential_scan and tls_check have single string params with no type constraints; port_scan has two params but no validation guidance; network_posture and container_vuln have EMPTY input schemas ({}), violating the requirement that every parameter have a description. No output schemas documented. No error handling guidance. No tool annotations. Naming is reasonable (verb_noun pattern) but descriptions fail to explain WHEN to use each tool or what it returns. No pagination, no result limits stated. Security tools (credential_scan, port_scan) lack warnings about prerequisites, rate limits, or legal/ethical constraints.
Check for known container vulnerabilities (basic checks)
Scan a GitHub repository for exposed credentials
Assess network security posture of a K8s namespace
Run an nmap port scan on a target
Check TLS certificate for a target host
Two tools (network_posture, container_vuln) have empty input schemas ({}). These tools are unusable without parameter documentation.
Descriptions are 10-60 characters, far below the 50-200 char baseline for LLM optimization. Examples: 'Scan a GitHub repository for exposed credentials' (52 chars) lacks WHEN to use it, prerequisites, or what it returns. LLMs cannot reliably select these tools.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 34 | 2026-07-28+ | v2 |
Security tools (credential_scan, port_scan) lack ethical/legal warnings, rate limits, or prerequisites. No guidance on when scanning is authorized or what to do if credentials are found.
No error handling guidance. Tools return READ_ONLY risk but no documentation of failure modes, retryability, or recovery steps. LLMs cannot self-correct on failures.