Loading ToolBench
Open-source MCP server that lets AI agents call APIs without ever seeing the credentials. Per-secret policy-gated credential broker.
SecretProxy demonstrates solid tool design with clear, action-oriented names (list_secrets, scan_env_requirement, http_request, run_command) and comprehensive descriptions (150-250 chars each). All four tools have explicit input schemas with typed parameters and descriptions. The server correctly avoids exposing secrets as parameters, instead using server-side injection, a critical security pattern. However, output schemas are not documented in the visible code, and error handling guidance is minimal. Parameter descriptions lack format constraints (e.g., URL validation rules, binary path format). The tool composition is clean (each tool has one responsibility), but the http_request and run_command tools lack explicit error recovery guidance for policy violations.
Make an HTTP(S) API call with a secret injected into the Authorization header or other headers. The vault injects the real secret value at call time and scrubs it from responses. Supports Bearer tokens, API keys, and custom header templates. Policy enforces which hosts are allowed per secret.
List all available secrets and their allowed scopes. Returns secret names, creation/update timestamps, and policy summaries (host counts, command counts, env var counts). Does NOT reveal secret values.
Execute a CLI binary with a secret injected into environment variables. The vault injects the real secret value at call time. Policy enforces which binaries and argument patterns are allowed per secret. Supports capturing stdout/stderr.
Scan a CLI binary's environment variable requirements and suggest matching vault secrets. Helps discover which vault secret satisfies an env var a script expects.
Output schemas not documented. LLMs cannot plan downstream operations or extract required fields (e.g., what fields does list_secrets return? Is there pagination?). Agents must guess at response structure.
Error handling lacks recovery guidance. When http_request or run_command returns POLICY_DENIED, the description says 'tell the user, do not retry' but the tool response itself does not include actionable error messages. LLMs need explicit guidance in the error response, not just the description.
Parameter format constraints missing. http_request's 'url' parameter lacks validation rules (HTTPS-only enforcement, allowed hosts). run_command's 'binary' parameter lacks format guidance (absolute path vs name lookup). Descriptions should state: 'Full HTTPS URL (http rejected unless policy allows)' and 'Path or name of CLI binary (e.g., gh, aws, psql)'.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
http_request headers parameter lacks clarity on Authorization template syntax. Description mentions '{{value}} placeholder' but does not show concrete examples (e.g., 'Bearer {{value}}' vs 'Token {{value}}' vs 'X-API-Key: {{value}}'). LLMs will guess and fail.
scan_env_requirement's output is not specified. Does it return a list of matching secrets? A single best match? What fields are included? Without documented output schema, agents cannot chain this tool into http_request or run_command calls.