Find it. Break it. Fix it. The hacker's toolkit for AI agents. Security scanning and hardening tool for AI agent projects.
Three tools with clear security-focused purposes and well-structured schemas. All tools have descriptions (194 - 250 chars, within baseline 34 - 392 range) and input schemas with typed parameters. However, descriptions lack LLM-optimized guidance on WHEN to use each tool vs. alternatives, and parameter descriptions are minimal. Output schemas are not documented. Error handling is present but generic. No tool annotations (readOnlyHint) despite all being read-only. Naming is clear and verb-driven (scan, deep_scan, benchmark).
Run OASB-1 (Open Agent Security Benchmark) compliance assessment. Evaluates the project against security controls at L1 (Essential), L2 (Standard), or L3 (Hardened) level.
Deep security scan: runs pattern + structural analysis, then returns security-relevant file contents for you (the LLM) to analyze for threats that automated tools miss. YOU should reason about the returned files and identify: credentials in ANY format, dangerous MCP configurations, overprivileged permissions, attack chains, and prompt injection vectors.
Scan a directory inside this server's allowed roots for AI agent security issues. Runs 100+ security checks across multiple categories: credentials, MCP configs, permissions, git security, dependencies, and more. Read-only: this tool never modifies files. Each finding carries its fix command. Fixes are applied from a terminal with `hackmyagent secure --fix <directory>`, so the person running it sees the change before it is written.
Output schemas not documented. LLMs cannot plan downstream actions or extract structured data from scan results. Baseline: 100% of A+ tools document return types.
Parameter descriptions are minimal. 'directory' param lacks guidance on path format, max length, or what happens if path is outside roots. Baseline: 100% of A+ tool params have descriptions; avg 72 chars.
Tool descriptions lack WHEN-to-use guidance. 'hackmyagent_scan' vs 'hackmyagent_deep_scan' distinction is unclear, when should an LLM choose one over the other? Descriptions should state prerequisites and selection criteria.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | 2026-07-28+ | v2 |
No tool annotations despite all tools being read-only. Missing readOnlyHint in schema would help LLMs understand these tools are safe to call without side effects.
'ignore' parameter in hackmyagent_scan lacks format guidance. What is the exact syntax for check IDs? Are they case-sensitive? What happens if an invalid ID is passed?