Loading ToolBench
MCP Gateway has 9 tools with complete input schemas and descriptions. Naming follows verb_noun convention (create_key, list_keys, revoke_key, proxy_mcp_request, get_logs). Descriptions are present but brief (10-100 chars for most tools). Parameters have types and constraints (enums for period, min/max for integers). However, output schemas are not documented in the source code provided, critical for LLM planning. Error handling and recovery guidance are absent. Tool composition is reasonable (separate key management, stats, logging, proxy), but descriptions lack context on when to use each tool vs. alternatives. No tool annotations (readOnlyHint/destructiveHint) visible. Baseline: 194 chars avg description; these average ~60 chars.
Create a new API key. Returns the raw key once — store it securely.
Query proxied call logs with filtering and pagination.
Gateway health check.
List all API keys (without raw key values).
Aggregate stats for the given period.
Proxy a JSON-RPC request to the named upstream MCP server. The request body is forwarded as-is. Every call is authenticated, rate-limited, and logged.
Revoke an API key.
Output schemas not documented. LLMs cannot plan downstream tool calls or extract required fields (e.g., what fields does create_key return? Does get_logs return pagination metadata?). This violates pattern:tool and pattern:response-shaper.
Descriptions are too brief (avg 60 chars vs. baseline 194 chars). Missing context on WHEN to use each tool and what it returns. E.g., 'List all API keys (without raw key values)' lacks guidance on when to call this vs. stats_by_key.
No error handling or recovery guidance. If proxy_mcp_request fails (upstream server down, auth error, rate limit), the tool provides no actionable next steps. Pattern:recovery-guide requires error responses to tell the LLM what to do.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
Per-API-key usage breakdown.
Per-server stats breakdown.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). LLMs cannot distinguish safe reads from destructive writes. create_key and revoke_key should be marked destructive; health_check and list_keys should be marked read-only.
proxy_mcp_request accepts 'authorization' as a parameter. This violates pattern:secret-injection, credentials must never be tool parameters. Use server-side secret injection via environment or vault.