Agent oversight/compliance gateway on MCP: audit + PII masking + policy
Four tools with basic verb-noun naming (get_, list_, search_, update_). All have descriptions (10-60 chars, below baseline 194 avg). Input schemas present with type definitions but parameter descriptions are minimal or absent. No output schemas documented. No error handling guidance. Oversight/audit/masking applied but not explained in tool descriptions. Missing critical context for LLM selection and recovery.
Retrieve a single employee by ID with oversight/audit/masking applied
List all employees with oversight/audit/masking applied
Search employees by query string with oversight/audit/masking applied
Update an employee's salary with oversight/audit/masking applied
Tool descriptions are too short (10-60 chars vs baseline 194 avg) and lack context. 'Retrieve a single employee by ID with oversight/audit/masking applied' does not explain WHEN to use this tool, what oversight means, or what masking does to the output.
No output schemas documented. LLMs cannot plan downstream calls or extract fields without knowing what get_employee returns (e.g., does it include salary, email, role?). Baseline: 100% of A+ tools document return types.
Parameter descriptions missing or trivial. 'emp_id' has description 'Employee ID' but no guidance on format, range, or what happens if ID does not exist. 'q' in search_employees has no description at all. Baseline: 100% of A+ tools have parameter descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2026-07-28+ | v2 |
No error handling guidance. If get_employee(999) fails, LLM has no recovery path. No indication whether error is retryable, user-fixable, or fatal. Baseline: A+ tools include recovery hints.
Oversight/masking behavior not explained in tool descriptions. LLM does not know that 'salary' is redacted for 'viewer' role or that 'update_employee' is denied for 'viewer'. This critical permission logic must be in the description.