Loading ToolBench
A Model Context Protocol server that exposes OpenAPI schema information to Large Language Models
This OpenAPI schema inspector offers 9 well-named tools with mostly complete input schemas, but suffers from critical parameter description gaps (rubric baseline: 100% of A+ tools have param descriptions; this server has 0). All 9 tools lack parameter descriptions entirely, violating pattern:tool-description. Tool descriptions are present but brief (average ~60 chars; rubric baseline 194 chars). No output schemas are documented. Error handling returns plain text but does not guide recovery or categorize errors. The server is STDIO-only, which hard-caps protocol readiness at 50. Given these gaps, the server functions as a basic read-only explorer but lacks the rigor for production agent deployment.
Gets detailed definition for a specific component
Gets detailed information about a specific API endpoint
Gets examples for a specific component or endpoint
Gets the parameters for a specific path
Gets the request body schema for a specific endpoint
Gets the response schema for a specific endpoint, method, and status code
Lists all schema components (schemas, parameters, responses, etc.)
CRITICAL: All 9 tools lack parameter descriptions entirely. Tools with parameters (get-endpoint, get-request-body, get-response-schema, get-path-parameters, get-component, get-examples) expose bare Zod schemas with no descriptions for 'path', 'method', 'statusCode', 'type', 'name', or 'component'. Rubric baseline: 100% of A+ tools have param descriptions. LLMs cannot infer what 'path' means (URL path? file path?) or what values are valid without explicit descriptions.
CRITICAL: No output schemas documented for any tool. Rubric rule: 100% of A+ tools have documented return types. Tools return YAML-formatted text in a single content block with type='text'. LLMs cannot plan downstream calls or extract structured data without knowing what fields are returned. For example, get-endpoint returns path/method/summary/description/tags/parameters/requestBody/responses/security/deprecated as YAML, but this structure is not declared in the schema.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 52 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 33 | - | v1 |
Lists all API paths and their HTTP methods with summaries, organized by path
Lists all available security schemes
HIGH: Tool descriptions are generic and do not explain WHEN to use each tool or what the LLM should do next. Example: 'Gets detailed information about a specific API endpoint' does not mention that the LLM should first call list-endpoints to discover paths. Descriptions should follow the pattern: WHAT + WHEN + prerequisite. Rubric baseline: average description length 194 chars; this server averages ~60 chars.
HIGH: Error handling does not guide recovery. When a path is not found, the tool returns plain text: 'Path {path} not found'. The LLM receives no suggestion to call list-endpoints first or try alternate path syntax. Error should read: 'Path "/users/{id}" not found. Call list-endpoints to discover available paths.' Rubric pattern:recovery-guide requires actionable error messages.
MEDIUM: Parameter 'statusCode' in get-response-schema has a default value of '200', but the description does not explain this. LLMs may not realize the parameter is optional and safe to omit. The code shows z.string().default('200'), but the description field is null. Similarly, 'method' in get-path-parameters is marked optional in the schema (z.string().optional()) but the description does not say so.
MEDIUM: No pagination or limits enforced on list-endpoints, list-components, and list-security-schemes. If an OpenAPI spec defines hundreds of paths or components, the YAML output could be extremely large, wasting tokens and risking context overflow. Rubric pattern:paginated-result and mxe:enforce-result-limits require pagination and result limits.
MEDIUM: get-examples tool has an enum constraint on 'type' (request|response) and optional 'path'/'method'/'component' params, but the description is missing. The logic requires either (path+method) OR component, but this mutual exclusivity is not documented. LLMs may pass invalid combinations (e.g. type='request' with only 'component').