Loading ToolBench
TypeScript-based MCP server that integrates with Swagger/OpenAPI specifications to expose API endpoints as tools for LLMs
The server defines 4 tools with explicit schemas and descriptions visible in src/server.ts. However, multiple critical gaps prevent a higher score: (1) Parameter descriptions are present but minimal (10-30 chars on average, well below the 72-char baseline); (2) Output schemas are completely undocumented, no structured response format is specified, forcing LLMs to guess what fields are returned; (3) Tool names lack action verbs for some tools (e.g., 'get_cache_information' should clarify what 'information' contains); (4) No error handling guidance is documented, errors are caught but responses don't guide recovery; (5) All tools accept swaggerUrl as a parameter, creating a potential security issue if URLs are untrusted; (6) No pagination guidance for list_services and get_all_service_information despite their potential for large result sets. The schemas themselves are well-formed JSON Schema objects with proper typing, but descriptions are too terse to meet LLM-optimized standards (rubric baseline: 50-200 chars for optimal; these are 20-80 chars).
Get detailed information about all API services from a Swagger/OpenAPI specification including parameters, request/response schemas, and examples
Get comprehensive information about the cache status including cached specifications, expiration times, and performance metrics
Get comprehensive information about a specific API service including parameters, request/response schemas, and example usage
List all available API services from the Swagger/OpenAPI specification and prepare for caching
No output schemas documented for any tool. LLMs cannot predict response fields, forcing trial-and-error tool chaining and reducing multi-step plan reliability.
Parameter descriptions are terse (10-45 chars), well below the 72-char baseline. For example, swaggerUrl descriptions are 35-43 chars and omit format guidance (must be HTTP/HTTPS URL? must be accessible? JSON vs YAML?). serviceId description (42 chars) doesn't explain how to discover valid IDs.
get_all_service_information returns 'all' services but has no pagination parameters (limit, offset, page_size) and no documented result cap. Returns 500+ services could exhaust context window.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 39 | - | v1 |
No error handling guidance in tool descriptions. What happens if swaggerUrl is invalid, unreachable, or returns malformed OpenAPI spec? What if serviceId doesn't exist? Descriptions don't guide recovery, and calling code (src/server.ts) catches errors but doesn't provide actionable error messages.
All tools accept swaggerUrl as untrusted user input. No validation visible in schema or description to prevent SSRF attacks (internal URLs), malicious payloads, or DoS via extremely large specs. Parameter description should document constraints (e.g., 'HTTPS only', 'max 50MB').
get_cache_information naming is ambiguous. 'cache_information' could mean 'get stats about the cache' or 'get the cached data'. Consider renaming to 'get_cache_status' or 'describe_cache_metrics' for clarity. Per pattern:tool, names alone should convey intent.
No distinction between get_service_information and get_all_service_information in descriptions. When would an LLM choose one over the other? 'Get comprehensive information about a specific API service' vs. 'Get detailed information about all API services', both claim to be comprehensive. Descriptions should clarify use cases.