Automated quality drift detection for AI-assisted development sessions
drift-guard has 5 tools with complete input schemas and descriptions, but quality is uneven. All tools have descriptions (10 - 200 chars, within baseline), and schemas are properly typed with required fields. However, parameter descriptions are sparse or generic. drift_guard_check has an optional nested object (evaluationResult) with undocumented sub-properties (score, violations array items). Output schemas are not documented anywhere, LLMs cannot predict what these tools return. Error handling is present but generic (catch-all try/catch returning JSON errors). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels (WRITE vs READ_ONLY). Naming is verb-noun compliant (drift_guard_*), but descriptions lack WHEN/WHY context and dependency hints. Parameter descriptions are minimal: 'Absolute path to the project root' appears 4 times verbatim, and 'File paths to track' lacks format guidance. No pagination, no result limits documented, no chaining IDs in responses. Composition is reasonable (each tool has one job), but no evidence of idempotency or batch operations.
Run quality checks. Stage 1 uses rule engine; if any llm_eval promises exist and rules pass, returns an evaluation prompt for Stage 2. If evaluationResult is provided, computes the final score.
Initialise drift-guard for a project: create .drift-guard/, auto-extract quality promises from CLAUDE.md, run baseline check, and restore previous session context.
Generate a session report from check history: start/end score, drift, violations.
Save a session context summary to .drift-guard/context.md for future restoration.
Track file metadata (path, line count, size, sections for .md). Saves a snapshot to history.
Output schemas not documented. LLMs cannot predict return types, forcing them to guess what fields are available for downstream tool calls or user display.
drift_guard_check has optional nested object 'evaluationResult' with undocumented sub-properties (score, violations). Sub-object properties lack descriptions and type constraints.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite explicit risk levels (WRITE vs READ_ONLY). Agents cannot determine which tools are safe to retry.
Parameter descriptions are generic or missing context. 'Absolute path to the project root' repeated 4 times; 'File paths to track' lacks format guidance (relative vs absolute, glob patterns, file extensions).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Error handling is generic (catch-all try/catch). Errors return JSON with 'error' field but no recovery guidance, error classification (retryable vs fatal), or actionable next steps.