Loading ToolBench
Pre-launch risk scanner for AI-built apps. MCP server exposing static risk detectors (exposed secrets, copyleft license contamination, unauthenticated data endpoints, missing PII/consent artifacts, client-side payment handling) to any MCP-compatible coding agent.
ShipCheck defines 2 tools with clear, well-structured schemas and strong descriptions. Both tools follow verb_noun naming (scan_repo, scan_diff) and include comprehensive parameter documentation with enums and defaults. Descriptions are detailed (180 - 220 chars) and explain WHAT the tool does, WHEN to use it, and important caveats (e.g., 'not a legal compliance certification'). Input schemas use Zod with proper types, enums for categories, and sensible defaults. Output is structured JSON. Error handling returns actionable messages (e.g., path validation with clear guidance). Main gaps: no output schema documentation in code comments, no per-parameter type hints in descriptions (e.g., 'path: string, absolute or relative'), and no explicit idempotency or retry guidance. Both tools are READ_ONLY, which is good for safety, but this is not annotated in the schema (toolAnnotations feature is absent).
Scans only what changed since a git ref (default HEAD, i.e. staged + unstaged work plus new untracked files) with the same five detectors as scan_repo. Built for the "check it before you commit" loop — fast and cheap where scan_repo is thorough. Same caveat: risk-pattern findings, not a pass/fail or compliance verdict.
Runs static risk detectors over a codebase: exposed secrets, copyleft-licensed dependencies in closed-source use, unauthenticated data endpoints, PII collection without a privacy policy, and client-side payment handling. Returns structured findings, never a pass/fail verdict — this is not a legal compliance certification.
Output schema not documented in code. Callers must infer the structure from buildScanResult() and JSON.stringify(result). LLMs cannot plan downstream operations without knowing what fields to expect.
Tool annotations (readOnlyHint, idempotentHint) not declared. Both tools are READ_ONLY and idempotent, but this is not signaled to the MCP client. Agents cannot infer safety properties.
Parameter descriptions lack explicit type and format hints. 'path' is described as 'Path to the repo root' but does not state 'absolute or relative path accepted' or 'must be a directory'. LLMs may pass invalid types.
Error handling in detectors is generic. When a detector fails (e.g., malformed package.json), the tool returns a medium-severity finding instead of distinguishing retryable vs. fatal errors. LLMs cannot determine if re-running will help.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | 2026-07-28+ | v2 |
No pagination or result limits documented. If a repo has thousands of findings, the entire list is returned as JSON text. Large responses waste tokens and risk context window exhaustion.