MCP server for verifying declared safety invariants before finalizing structured output in high-stakes AI applications
Two tools with basic descriptions but significant schema and parameter documentation gaps. check_invariants_tool has a complex 6-parameter interface with minimal per-parameter descriptions (only 1-2 sentences each). list_invariants_tool is simpler but both lack output schema documentation. No error handling guidance, no enum constraints, no validation rules stated in descriptions. Parameter relationships (suite_path vs policy mutual exclusivity) are undocumented. Naming is adequate (verb_noun pattern) but descriptions lack LLM-optimization depth (194-char baseline for A+ tools; these average ~120 chars). No tool annotations or security considerations visible.
Verify declared safety invariants before finalizing structured output. Provide suite_path to an existing suite.yaml, or an inline policy dict with field_policies and assertions. Returns passed/invariant_passed and failures.
Read-only: list field policies, equivalence classes, and per-case assertions.
Parameter descriptions are minimal (1-2 sentences) and lack format/constraint guidance. E.g., 'suite_path' has no description of expected file format, path resolution rules, or error behavior if file missing. 'policy' dict structure is undocumented, what keys/types does it require?
No output schema documented. check_invariants_tool returns a dict with 'passed/invariant_passed and failures' but structure is opaque, what are the exact keys, types, and nesting? LLMs cannot plan downstream calls without knowing return shape.
Undocumented parameter relationships. check_invariants_tool accepts both 'suite_path' and 'policy' as optional, but their mutual exclusivity or precedence is not stated. LLMs will pass both or neither, causing silent failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
No error handling guidance. Descriptions do not explain what happens if suite.yaml is malformed, policy dict is invalid, or invariants fail. No recovery hints (e.g., 'If invariant fails, review field_policies in suite.yaml').
No enum constraints or validation rules in descriptions. 'human_confirmed' is a dict but its expected structure (field names, boolean values?) is not documented. LLMs will guess and pass invalid structures.