Loading ToolBench
The independent security agent for AI-written software. A deterministic engine finds issues across application code, AI agents, MCP servers, skills, dependencies, CI/CD, and secrets. An investigation layer traces whether each one is real and reports the evidence as JSON and SARIF.
Ship Safe exposes 4 security-focused tools with clear, domain-specific descriptions (avg 180 chars). All tools have input schemas with typed parameters and enums. However, output schemas are not documented in the source, parameter descriptions lack actionable constraints (e.g., path format, file size limits), and error handling guidance is absent. Tools follow verb_noun naming (ship_safe_*) but descriptions could better explain when to use each tool vs. alternatives. No tool annotations (readOnlyHint, destructiveHint) despite all being read-only operations.
Run a Ship Safe security audit on a local codebase directory. Returns a findings report with severity-graded issues, CWE/OWASP mappings, and remediation guidance. Use before deploying any code or merging PRs.
Retrieve findings from the last saved Ship Safe scan report for a project. Optionally filter by minimum severity. Returns an array of findings with rule, title, severity, file, line, and remediation guidance.
List all entries in the Ship Safe security memory for a project. The memory stores learned false positives that are automatically filtered from future scans. Returns each entry with its rule, file pattern, and the snippet that was suppressed.
Analyze an MCP server manifest (URL or local file path) for security issues before connecting. Checks for prompt injection in tool descriptions, credential harvesting patterns, Hermes function-call poisoning, schema bypass (additionalProperties: true), and known-malicious server hashes. Returns per-tool findings.
Output schemas not documented. LLMs cannot plan downstream calls or extract required fields (e.g., what fields does ship_safe_audit return? Is there a findings array? What are the field names?). Agents must guess structure.
Parameter descriptions lack actionable constraints. 'path' is described as 'Absolute path to the project root directory' but does not specify: must exist? must be readable? max length? Does it accept symlinks? LLMs cannot validate inputs without these details.
No error handling guidance. If ship_safe_audit fails (e.g., path not found, permission denied, scan timeout), the tool provides no recovery hint. LLMs cannot self-correct or decide whether to retry.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | 2026-07-28+ | v2 |
No tool annotations despite all tools being read-only. Adding readOnlyHint=true to each tool's definition enables agents to safely parallelize calls and avoid unnecessary confirmation steps.
Parameter 'severity' enum is present but descriptions do not explain the difference between thresholds or when to use each. E.g., does 'critical' return only critical findings, or critical+higher? Does 'medium' include high and critical?