Loading ToolBench
Single tool 'do-smuggler' has a clear action verb and structured schema, but suffers from critical gaps: (1) Tool description is 89 chars, below the 194-char baseline and lacks WHEN/WHY context for LLM selection. (2) Parameter 'smuggler_args' description is 400+ chars of raw CLI help text, not LLM-optimized guidance. (3) No error handling guidance, tool rejects non-zero exit codes with generic 'Smuggler exited with code X' but does not tell the LLM what to do next (retryable? user-fixable?). (4) Output schema documents metadata.findings structure but lacks guidance on interpreting severity levels or when to escalate findings. (5) No input validation, url parameter uses z.string().url() but tool spawns external Python process without sanitization, risking command injection if smuggler_args are not validated. (6) Tool is READ_ONLY but description does not state this, missing the pattern:command-tool requirement to declare state-modifying intent.
Run Smuggler to detect HTTP Request Smuggling vulnerabilities
Tool description is 89 characters, below 194-char baseline. Lacks WHEN/WHY context for LLM selection. Does not explain when to use this tool vs. other security scanners or what prerequisites exist.
Parameter 'smuggler_args' description is 400+ chars of raw CLI help text (method, vhost, len, configfile, etc.). Not LLM-optimized. LLMs cannot parse unstructured CLI docs, should be structured enum or separate typed parameters.
Error handling is generic. Tool rejects non-zero exit codes with 'Smuggler exited with code X' but does not tell LLM what to do next: is it retryable? Should user check URL? Is smuggler.py missing? No recovery guidance.
No input validation on smuggler_args array. Tool spawns Python process with user-supplied args without sanitization. LLM could inject shell metacharacters (e.g., '; rm -rf /') via smuggler_args, risking command injection.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Output schema documents findings structure but lacks interpretation guidance. What does severity 'high' vs 'medium' mean for agent decision-making? Should agent escalate? Retry with different args? No guidance.