MCP Gateway/Proxy with OIDC authentication, access control (RBAC/ABAC/ReBAC), and monitoring
MCP Gateway exposes 8 tools with basic schemas and descriptions, but falls short of production quality. Tool names lack consistent verb-noun structure (e.g., 'virtual-tool list' vs 'list_virtual_tools'). Descriptions are present but generic (10-50 chars for most tools), providing minimal context for LLM selection. Input schemas are visible and typed, but parameter descriptions are sparse or missing entirely. No output schemas documented. Error handling is absent, no recovery guidance, no categorization of retryable vs fatal errors. Security considerations (permission gates, audit trails) are not evident in tool definitions. Composition is reasonable (tools are single-purpose), but chaining support is unclear due to missing output documentation.
Set cache flags on a tool
Create a virtual tool from a plan JSON file
Delete a virtual tool
List virtual tools
Show virtual tool detail
Dry-run a virtual tool with given args
Update a virtual tool plan from a JSON file
Validate a plan JSON file without persisting
Tool names do not follow verb_noun convention. 'virtual-tool list' should be 'list_virtual_tools'; 'virtual-tool create' should be 'create_virtual_tool'. Hyphenated subcommand style is CLI-idiomatic but confuses LLMs expecting action-first naming.
Descriptions are too brief (10-50 chars) and lack context for LLM selection. 'List virtual tools' does not explain when to call it, what it returns, or how it differs from 'show'. Descriptions should be 50-200 chars and answer: what, when, why.
No output schemas documented. LLMs cannot infer what fields 'list' returns, whether 'show' includes nested metadata, or what 'test' produces. Without documented return types, agents cannot plan downstream tool calls or extract required data.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | <=2025-11-25 | v2 |
Parameter descriptions are minimal or absent. 'config' is described as 'Path to plan JSON file' but does not explain the expected JSON structure, required fields, or validation rules. 'args' in 'virtual-tool test' is 'JSON string of input arguments', what arguments? What format?
No error handling guidance. Destructive tools like 'virtual-tool delete' lack confirmation or dry-run support. No recovery hints (e.g., 'If validation fails, check the JSON schema at /docs/plan-schema.json'). Agents cannot self-correct on failures.