A safety layer for AI coding agents before they touch your repo. Provides runtime guardrails via MCP server exposing file access checks, command validation, protection listing, and repo safety scanning.
Four tools with clear safety-focused purpose and reasonable descriptions (avg 150 chars). All have input schemas with type definitions. However, critical gaps: (1) check_file_access lacks 'operation' in required array despite being essential for distinguishing read/write/delete intent; (2) descriptions are functional but lack LLM-optimized guidance on WHEN to call each tool vs alternatives; (3) no output schemas documented, LLMs cannot predict response structure; (4) error handling descriptions missing, no guidance on what 'ask' vs 'block' means operationally or how to recover; (5) no parameter descriptions for 'operation' enum values. Naming is strong (verb_noun pattern). Composition is good, each tool has single responsibility. Security-focused design is appropriate for the domain.
Check whether a shell command is safe to run in this repo. Returns safe | warn | block. Call this BEFORE executing any shell command.
Check whether the AI agent is allowed to read/write a given file path. Returns one of: allow, ask, block. Call this BEFORE editing or creating any file. If "block" is returned, do not proceed. If "ask", request explicit human approval first.
List all protected files, approval-required globs, and blocked commands configured in .repo-seatbelt.json.
Run a full repo safety scan and return the score plus a list of risks.
check_file_access: 'operation' parameter not in required array, but description implies it is essential for distinguishing read/write/delete decisions. LLMs may omit it, causing ambiguous results.
No output schemas documented for any tool. LLMs cannot predict response structure (e.g., does check_file_access return {decision, reason} or {status, message}?). Agents must infer from examples.
Descriptions lack recovery guidance. 'Returns safe | warn | block' tells LLM the values but not what to do when block is returned. Should say: 'If block, do not execute. If warn, log and ask user.'
list_protections and scan_repo descriptions are generic (60 chars). Missing context on when to call them vs check_* tools. Should explain: 'Call this first to understand repo constraints before using check_* tools.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
check_file_access 'operation' enum values lack descriptions. LLM must infer that 'delete' is distinct from 'write'. Add descriptions: operation enum with 'read: File read access', 'write: File creation/modification', 'delete: File deletion'.