Loading ToolBench
MCP server for scanning codebases and git history for hardcoded secrets with privacy-preserving redaction
Two well-named tools with comprehensive descriptions and solid input schemas. Both tools follow verb_noun naming (scan_for_secrets, scan_git_history) and include detailed descriptions (180+ chars each) explaining what they do, when to use them, and what they return. Input parameters are typed and described. However, output schemas are documented only in prose within descriptions, not as formal JSON Schema structures. Error handling is present but minimal, no recovery guidance or actionable error messages visible in the code. Security is strong (PII redaction enforced, path validation, no secrets in params). Composition is clean: each tool has one responsibility. Missing: enum constraints for severity parameter (free-form string), explicit output schema documentation, and per-item error reporting for batch-like operations.
Scan a working tree for hardcoded secrets. Reports location only — never values. Returns: schemaVersion, truncated, filesSkipped, findings[]. Each finding carries file, line, column, secretType, severity, hint (masked field indicator), and suggestedFix. PII fields are stripped at the wire boundary.
Scan a git repository's commit history for hardcoded secrets. Reports the earliest introduction of each unique secret along with whether it is still present in the working tree. Returns: schemaVersion, truncated, commitsWalked, findings[]. PII fields are stripped at the wire boundary.
Severity parameter accepts free-form string instead of enum. LLMs may hallucinate invalid values like 'urgent' or 'severe' instead of the documented low|medium|high|critical.
Output schemas documented only in prose descriptions, not as formal JSON Schema. LLMs cannot parse the structure of findings[] or extract field names reliably without explicit schema.
Error responses lack recovery guidance. ScanForSecretsResponse.Failure() returns a plain error string with no hint about what the agent should do next (e.g., 'Try a different path' or 'Check workspace configuration').
No tool annotations (readOnlyHint, idempotentHint) declared. Both tools are read-only and idempotent, but this is not signaled to the MCP client or LLM.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
No pagination or result limits documented in tool descriptions. If findings exceed a reasonable threshold, the response could bloat the context window. Truncation is implemented but not advertised.