Loading ToolBench
Federated MCP gateway — one spec-compliant MCP server (Streamable HTTP + OAuth 2.1) in front of N plain-HTTP backends
Cortex-gateway demo backend exposes 11 tools with consistent naming (verb_noun pattern) and reasonable descriptions (avg ~100 chars). All tools have input schemas visible in source. However, parameter descriptions are sparse or missing entirely for most tools, only 'get_time' has a fully documented enum parameter. Output schemas are undocumented. Error handling is minimal (no recovery guidance). Security model relies on scope-based filtering but lacks explicit permission declarations per tool. Composition is sound (single responsibility per tool), but several tools lack the depth of documentation expected for production use.
Deletes a demo note by id. Requires the write scope.
Echoes the message back. Smoke-test tool.
Returns the structured documentation of the demo backend (workflows, conventions, examples).
Aggregates backend snapshot data including uptime and status metrics.
Returns the current server time (ISO 8601).
Lists the shared demo notes (in-memory, reset on restart).
Lists available prompts from the backend catalog.
Output schemas undocumented. Tools return structured data (e.g., get_snapshot returns {backend, generatedAt, title, headline}, whoami returns {email, role, capabilities}) but LLMs have no schema to parse results or plan downstream calls.
Parameter descriptions missing or minimal. Most tools accept parameters (e.g., get_help accepts 'topic', save_note accepts 'text', delete_note accepts 'id') but descriptions are absent or trivial. LLMs cannot infer parameter semantics from names alone.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
Lists available resource templates from the backend catalog.
Lists available tools from the backend catalog.
Saves a shared demo note. Requires the write scope — read-only callers do not even see this tool.
Returns the authenticated user's identity, role, and capabilities based on their scopes.
No error recovery guidance. Tools lack descriptions of failure modes, retryability, or next steps. E.g., delete_note could fail if id is invalid, but no guidance on how to recover or what to try next.
Scope-based access control implicit in tool definitions. Tools declare 'scope' field (e.g., 'mcp:demo:write') but tool definitions lack explicit permission declarations. LLMs cannot reason about which tools require elevated privileges.
List tools lack pagination parameters. list_tools, list_prompts, list_resource_templates return unbounded results with no limit, offset, or cursor. If catalogs grow, responses could exhaust context windows.