Loading ToolBench
A lightweight, high-performance protocol conversion gateway that transforms OpenAPI specifications into MCP (Model Context Protocol) format for AI agents.
Synapse MCP Gateway has significant definition quality gaps. Tool definitions are partially present but lack consistent, comprehensive documentation. Naming is inconsistent (handle_* prefix for protocol handlers is non-standard). Parameters lack descriptions in critical tools. Input/output schemas are incompletely documented. Error handling guidance is minimal. Only 4 of 11 tools have adequate descriptions; 7 tools lack parameter documentation in the source. The server conflates protocol-level message handlers (initialize, tools/list, tools/call) with user-facing tools, which violates the single-responsibility principle. Average tool score: 42/100.
Creates a new MCP server.
Deletes an MCP server.
Fetches an OpenAPI 3.0 specification and returns a simplified list of its endpoints.
Retrieves a single MCP server by ID.
Retrieves all registered MCP servers.
Exposes converted MCP Tools from an OpenAPI 3.0 specification.
Handles initialize request - returns protocol version, server capabilities, and server information.
Protocol handler tools (handle_initialize, handle_tools_list, handle_tools_call) violate single-responsibility principle. These are MCP protocol message handlers, not user-facing tools. They should not be exposed as top-level tools in the tool registry.
Non-standard naming convention. Tools prefixed 'handle_' are not action verbs. MCP protocol specifies tool names should follow verb_noun pattern (get_, list_, create_, update_, delete_, search_). 'handle_tools_call' should be 'call_tool' or 'execute_tool'.
Descriptions for core tools are missing or minimal. 'handle_tools_list' has only 38 chars. 'handle_initialize' has only 33 chars. Rule baseline: descriptions must be 10 - 1024 chars with sufficient context for LLM selection. These fail to explain WHEN to call vs alternative tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 11 | - | v1 |
Handles tools/call request - executes actual HTTP API calls to backend services with path parameter substitution, request body handling, query parameters, and authentication support.
Handles tools/list request - returns all available tools for an MCP server aggregated from active combinations.
Toggles MCP server status (enable/disable).
Updates an existing MCP server information.
Parameter documentation incomplete. 'handle_tools_call' arguments object has properties ('_method', '_path', '_serviceUrl', '_authType', '_authConfig', 'body') with minimal or missing descriptions. LLMs cannot infer meaning from '_method' alone, is this HTTP method, workflow method, or something else?
Output schemas not documented. No tool specifies what structure is returned. LLMs cannot plan downstream tool chains without knowing response format. E.g., 'get_mcp_servers' returns list but no documentation of list item structure (fields, types).
Error handling lacks guidance. Tools describe HTTPException with generic status codes but no recovery hints. E.g., 'get_api_endpoints' raises 404/500 with detail only. Should state: 'If URL invalid, check OpenAPI spec host. If timeout, try shorter spec.'
Destructive operations lack confirmation/dry-run pattern. 'delete_mcp_server' has no dry-run option or confirmation request. Agents can be tricked into deleting servers. Should support 'dry_run=true' parameter or explicit confirmation step.
Parameter value constraints not fully specified. 'toggle_mcp_server_status' has enum ['active', 'inactive'] but no description clarifying what state transitions are valid or what happens on toggle. Is 'active' idempotent if already active?
Authentication handling opaque. 'handle_tools_call' accepts '_authType' and '_authConfig' as user-provided parameters. No description of what config structure is expected for each auth type (api_key, basic, oauth2). This invites hallucinated payloads.