Loading ToolBench
MCP Server that parses OpenAPI 3.0 specifications and generates autonomous tool methods for agent execution. Translates OpenAPI endpoint paths, query/body schemas, and HTTP methods into self-contained, type-annotated Python client methods with parameter validation.
Critical gaps across all dimensions. Tool descriptions are present but minimal (18-25 chars, well below 50-200 char baseline for LLM optimization). Input schemas lack proper JSON Schema structure, parameters are documented as bare object types with no field-level type definitions or constraints. No output schemas documented. Parameter descriptions are missing entirely (e.g., 'spec' param has only 'OpenAPI 3.0 specification dictionary' with no format/constraint guidance). No error handling guidance. No tool annotations (readOnlyHint/destructiveHint). invoke_tool accepts arbitrary 'params' object with no validation schema visible. This server would require substantial rework before production use.
Parse OpenAPI specification dictionary and extract tool methods from paths, methods, operationIds, summaries, and parameters
Execute a generated tool by name with supplied parameters, validating required arguments and returning execution result with status, method, path, and supplied arguments
Input schemas lack proper JSON Schema structure. 'spec' parameter documented as bare object type with no field-level type definitions, constraints, or format guidance. 'params' in invoke_tool is completely untyped, accepts any object with no validation schema.
Tool descriptions are too short (18-25 chars) and lack LLM-optimization guidance. 'Parse OpenAPI specification dictionary...' does not explain WHEN to use this tool, what prerequisites exist, or what the agent should do with the returned tool_names. Baseline: 50-200 chars for A+ tools.
No output schemas documented. Callers cannot predict response structure (e.g., does invoke_tool return execution result, HTTP status, error details?). LLMs cannot plan downstream tool calls without knowing what fields to extract.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 31 | 2026-07-28+ | v2 |
No error handling guidance. execute_tool_call returns {'error': '...'} with no recovery hints. LLM receives 'Tool not found' with no suggestion to call import_spec first or list available tools. Missing actionable error classification (retryable vs user-fixable vs fatal).
No tool annotations. invoke_tool is marked WRITE risk but has no destructiveHint in schema. import_spec is READ_ONLY but has no readOnlyHint. Missing idempotentHint guidance for retry safety.