An autonomous ML lineage guardian that watches DataHub for silent upstream changes, detects impact on production models, and remediates via fail-loud guards and write-back to the graph.
DriftGuard exposes 11 tools with basic schemas and descriptions, but significant gaps prevent production use. All tools have descriptions (10-200 chars, within baseline), but parameter documentation is sparse. Input schemas are present but minimal: most parameters lack type constraints, enums, or validation guidance. No output schemas documented. Error handling is absent, tools provide no recovery guidance. The server conflates multiple concerns (e.g., trace_lineage + measure_impact should be separate). STDIO transport caps protocol readiness at 50, further limiting score.
Add owners to DataHub entities.
Add tags to DataHub entities.
Get lineage information for a DataHub entity.
Read the training metrics (e.g. AUC) recorded on an at-risk ML model in DataHub.
List schema fields for a DataHub dataset.
Measure the model's performance drop caused by the change (AUC before vs after).
Remove owners from DataHub entities.
measure_impact has NO input schema (empty object {}). Tool is unusable without parameters defining what to measure.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., what does trace_lineage return? URNs? Metadata?). Violates pattern:tool and mxe:include-chaining-ids.
No error handling or recovery guidance. Tools provide no actionable error messages. If search fails or a URN is invalid, LLM has no guidance on what to try next.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Remove tags from DataHub entities.
Save a document to DataHub.
Search DataHub for entities matching a query.
Trace DataHub lineage from a changed column to the ML features and production models it affects.
Destructive tools (add_tags, remove_tags, add_owners, remove_owners, save_document) lack confirmation/dry-run support. No idempotent hints. Agents can accidentally bulk-modify DataHub without safeguards.
Parameter descriptions lack format/constraint details. E.g., 'entity_urns' array has no description of URN format, length, or valid prefixes. 'ownership_type' has a default but no enum of valid values.