Loading ToolBench
Expose AgentLeak privacy-leakage testing to coding agents over the Model Context Protocol. Scans source code for secrets and PII, analyzes agent execution traces for privacy violations, and provides preflight checks before shipping.
Four tools with reasonable descriptions (60-120 chars each) but significant schema gaps. privacy_preflight and scan_code have minimal parameter descriptions. check_trace and redact lack visible input schema validation. No error handling guidance, no output schema documentation, and no tool annotations. Naming is clear (verb_noun pattern), but parameter constraints are underdocumented. This is a typical community-grade server with functional tools but incomplete LLM-facing definitions.
Score one agent run across all eight channels.
Check a project before shipping it, and say what changed since last time. Scans source code for secrets and PII, optionally scores a supplied agent execution trace, and identifies new findings since the last check.
Strip sensitive values out of text before it goes anywhere.
Static privacy scan of source: secrets, PII in logs, unsafe sends.
No output schema documentation. LLMs cannot plan downstream calls or extract results without knowing what fields to expect.
Parameter descriptions are minimal or missing. 'path' in scan_code lacks detail on what 'defaults to current directory' means in practice. 'trace' in check_trace has no guidance on structure or required fields.
No error handling guidance. If scan_code encounters a permission error or invalid path, the LLM has no recovery hint. No categorization of retryable vs fatal errors.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All four tools are read-only, but this is not declared in the schema, forcing LLMs to infer safety.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
privacy_preflight 'trace' parameter accepts 'object' with no schema constraints. LLMs cannot validate structure before calling. Should define required fields (e.g. 'events' array) and types.