A Python-based toolchain for building, verifying, and packaging the Agentic SAMM (Software Assurance Maturity Model) framework documentation. Includes PDF generation, consistency verification, secret scanning, and reviewer pack creation.
ASAMM tools lack critical definition quality. All 6 tools have descriptions (positive), but input schemas are either missing or severely incomplete. Most tools expose only 1-2 boolean/string parameters with minimal type information. No output schemas are documented. Tool names are action-oriented (generate_*, build_*, check_*, verify_*) but descriptions are technical and lack LLM-optimized guidance on WHEN to call each tool or WHAT happens on failure. No error handling guidance, no parameter constraints (enums, ranges), no recovery paths. Schemas visible in source show basic structure but lack depth: e.g., build_pdf has 4 params but no output schema, no documented return structure, no error classification. This is typical of internal build-tool wrappers, not production agent tools.
Build and verify the English ASAMM review PDF from Markdown source using Pandoc and WeasyPrint
Build and verify a deterministic ASAMM reviewer pack as a ZIP archive with manifest, checksums, and verification report
Fail when current repository artifacts contain an unreviewed secret using detect-secrets hook
Validate the ASAMM control catalog and render its publication views (control cards, index, JSON/CSV exports)
Assemble the English ASAMM monolith from its normative source parts (part0-3 and frontmatter)
Run repository-wide ASAMM consistency and publication checks including generation, catalog validation, links, controls, taxonomy, figures, and metadata
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract structured data. build_pdf returns pages and digest but this is inferred from code, not declared in schema.
Descriptions lack LLM-optimized guidance. No explanation of WHEN to call each tool, what prerequisites exist, or what happens on failure. E.g., 'Validate the ASAMM control catalog...' does not explain when an agent should invoke this vs verify_consistency.
Input schemas lack constraints and enums. Boolean flags (check, verify_only, reproducibility_check) have no guidance on mutual exclusivity or side effects. No min/max for string paths. No enum for output file formats.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 38 | 2026-07-28+ | v2 |
No error handling guidance. Tools raise PdfError, PackError, RuntimeError but descriptions do not explain recovery paths. An LLM calling build_pdf with invalid output path gets a generic OSError with no hint to validate the path first.
Tool names do not clearly distinguish responsibilities. generate_controls, generate_monolith, and verify_consistency all perform validation and generation. Descriptions do not explain which to call first or in what order.