Audit a GitHub org's security posture (branch rules, 2FA, secrets) from an export
Single tool 'ghaudit_scan' has a reasonable description (97 chars) but critical gaps in schema completeness and parameter documentation. Input schema shows only 'target' parameter with basic type and description, but lacks validation constraints (format, pattern, min/max length). No output schema is documented, LLMs cannot infer the structure of returned JSON findings. The tool description does not explain when to use it, what prerequisites exist (e.g., 'requires GitHub org export file'), or how to interpret results. No error handling guidance provided. Tool naming is acceptable (verb_noun pattern) but the implementation conflates two concerns: scanning a file/directory AND auditing GitHub security posture, which should be clearer in the description.
Audit a GitHub org's security posture (branch rules, 2FA, secrets) from an export. Returns JSON findings.
No output schema documented. LLMs cannot infer the structure of the JSON findings returned by ghaudit_scan. The description says 'Returns JSON findings' but does not specify fields (id, severity, where, etc.) or data types.
Input parameter 'target' lacks validation constraints. No format, pattern, min/max length, or enum specified. Description does not clarify whether it accepts file paths, directory paths, or both, or what happens if the path does not exist.
Tool description does not explain prerequisites or error conditions. It should state: 'Requires a GitHub organization export JSON file. Scans for branch protection rules, 2FA enforcement, and secrets exposure. Returns findings with severity levels.'
No error handling guidance. If the file is not found, malformed, or inaccessible, the tool should return a clear, actionable error message (e.g., 'File not found at /path/to/export.json. Ensure the path is correct and readable.').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
STDIO transport only. Server is not remotely accessible and cannot be used by hosted MCP clients. This is a hard architectural limitation.