YAML-first, domain-driven data governance for AI agents
Four tools with basic verb-noun naming (lookup_*, trace_*, run_query). All have descriptions (10-150 chars, within baseline range). Input schemas present with type and description for each parameter. However, critical gaps: no output schemas documented anywhere; descriptions lack WHEN/WHY context and dependency hints; no error handling guidance; no pagination for list-like operations; no parameter constraints (enums, ranges); descriptions are minimal (avg ~80 chars vs baseline 194). Tools appear to be read-only (good for safety) but lack idempotency/retry guidance. No tool annotations (readOnlyHint present in metadata but not in schema). Schema quality is basic, parameters are typed but lack format/pattern/enum constraints. Overall: functional but well below production baseline.
Look up a domain by name to retrieve its description and associated metrics
Look up a metric by name to retrieve its definition, domain, and governance rules
Execute a SQL query against the governed data warehouse with contract validation and access control
Trace the impact graph of a metric, showing upstream drivers or downstream dependents with confidence levels (verified/correlated/hypothesized)
No output schemas documented. LLMs cannot plan downstream tool calls or extract required fields. All four tools lack return type documentation.
Descriptions lack WHEN/WHY context and dependency hints. E.g., 'Look up a domain by name' does not explain when to call this vs trace_metric_impacts, or what prerequisites exist. Baseline: 194 chars; actual: ~80 chars.
run_query accepts free-form SQL with no input validation, sanitization, or error guidance. No mention of SQL injection prevention, query timeouts, or result limits. Critical for security.
trace_metric_impacts 'direction' parameter lacks enum constraint. Should be enum: ['upstream', 'downstream']. Currently free-form string invites hallucinated values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
No error handling guidance. Tools do not document what errors are retryable, user-fixable, or fatal. No recovery hints (e.g., 'If metric not found, try search_metrics() first').