Embedding / vector-store drift and poisoning audit exposed as MCP tools
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. Both tools accept file paths and numeric thresholds with defaults, but lack detailed guidance on expected formats, error conditions, and output structure. Descriptions are present but generic (under 100 chars). No parameter descriptions visible in schema. No error handling guidance. Tools are read-only (good) but lack idempotent/readonly annotations.
Audit a JSONL embedding snapshot for drift/poisoning. Returns JSON.
Compare a baseline JSONL snapshot to a current one. Returns JSON.
Parameter descriptions missing from schema. snapshot_path, dup_threshold, domination_share, baseline_path, current_path, drift_threshold have no descriptions visible in the JSON Schema input objects.
Output schema not documented. Both tools return JSON strings but no schema is provided describing the structure of the returned JSON (fields, types, nested objects). LLMs cannot plan downstream operations without knowing output shape.
Tool descriptions lack actionable context. 'Audit a JSONL embedding snapshot for drift/poisoning. Returns JSON.' does not explain WHEN to use this tool, what 'drift/poisoning' means, or what the LLM should do with the returned JSON. Descriptions should be 50-200 chars with clear intent.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
No error handling guidance. Tools accept file paths but do not document what happens if the file does not exist, is malformed, or is unreadable. No recovery hints provided to the LLM.
No tool annotations. Tools are read-only but lack readOnlyHint annotation in the MCP schema. This prevents clients from optimizing caching or safety policies.