Loading ToolBench
Turn any OpenAPI specification into a Model Context Protocol (MCP) server with a single command.
Three well-named, verb-starting tools with clear descriptions and partial schema visibility. The server exposes OpenAPI specs as MCP tools via three meta-tools: list_operations (discovery), get_operation (schema inspection), and call_operation (invocation). Descriptions are present and substantive (50-150 chars), exceeding the 10-char minimum. get_operation and call_operation have documented input schemas with typed parameters. However, output schemas are not documented in the visible source code, only the tool descriptions hint at return structure ('{operations: [...]}', '{name, description, input_schema}'). This is a moderate gap for LLM planning. Tool composition is sound: list → get → call forms a logical discovery chain. The 'call_operation' tool accepts a generic 'arguments' object, which trades explicitness for flexibility but mirrors OpenAPI's variable schema requirements. Risk annotations (READ_ONLY, WRITE) are declared but not formally reflected in ToolAnnotations in the visible code snippet. Error handling and validation logic are not visible in the provided excerpt, limiting assessment of recovery guidance.
Invoke an operation by name with a JSON object of arguments. Argument keys must match the keys in `get_operation(name).input_schema.properties`.
Return the input schema for one operation. Returns `{name, description, input_schema}`. Use the JSON Schema to construct the `arguments` payload for `call_operation`.
List every operation available on this server. Returns `{operations: [{name, description}, ...]}`. Call `get_operation(name)` next to fetch the input schema for one specific operation.
Output schemas not documented. Tool descriptions state return structure informally ('{operations: [...]}', '{name, description, input_schema}') but no formal JSON Schema or structured documentation of response fields is visible. LLMs cannot predict response structure for planning or field extraction.
list_operations has no input schema visible. The tool accepts no parameters but the absence of an explicit empty schema definition may confuse clients expecting schema documentation.
call_operation uses a generic 'arguments' object with no schema constraint. Clients cannot validate payloads before sending; the tool must accept any JSON object, then validate and report errors at runtime. Error messages are not documented, leaving LLMs without recovery guidance.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 67 | <=2025-11-25 | v2 |
Risk annotations (READ_ONLY, WRITE) declared in metadata but not formalized in ToolAnnotations. The code shows derive_tool_annotations() derives read_only_hint and destructive_hint from HTTP method, but it is unclear whether list_operations and get_operation actually register these hints, they are discovery/read tools but no annotation override evidence visible.
No pagination guidance for list_operations. If the OpenAPI spec exposes many operations, a large flat list could exhaust context. Description does not hint at limits, ordering, or pagination options.