Loading ToolBench
A comprehensive MCP (Model Context Protocol) gateway server that manages tool discovery, registration, and execution across multiple MCP servers with database persistence, WebSocket support, and multi-transport capabilities
MCP Gateway is a Go-based tool management gateway with 8 tools covering tool CRUD, discovery, and refresh operations. While tools are present with descriptions and parameter documentation, several critical quality gaps significantly limit production readiness. Tool naming lacks consistent verb-prefix conventions (DiscoverServerTools, RefreshServerTools follow imperative style but are verbose). Descriptions are present but generic, they state WHAT but lack WHEN context and clear dependencies. Output schemas are entirely undocumented; the code shows handlers but no explicit schema definitions in responses. Parameter descriptions exist but lack constraint documentation (e.g., valid category enum values are mentioned in description text but not as formal JSON Schema enums). Error handling and recovery guidance are absent from tool definitions. Security considerations (required permissions, sensitive fields) are not documented. Most critically, the output shapes for all 8 tools are inferred from handler code rather than explicitly documented in tool definitions, which caps per-tool schema scores at 30-50.
Creates a new MCP tool with schema validation, category validation, and duplicate checking
Soft-deletes (deactivates) an MCP tool
Discovers and registers tools from an MCP server using transport layer connections
Retrieves all discovered tools for a specific MCP server
Retrieves a specific MCP tool by ID
Lists all tools for an organization with support for filtering by category, search term, popularity, and public tools
Refreshes tools for a specific server by deleting existing discovered tools and re-discovering
Output schemas are completely undocumented. No tool defines what it returns to the LLM. Handler code exists (tool.go, tool_discovery_service.go) but response shapes are inferred, not explicitly documented. This prevents LLMs from knowing what fields to expect and blocks effective chaining.
Parameter constraints are documented in description text but not as formal JSON Schema constraints. Example: CreateTool lists valid category values ('general, data, file, web, system, ai, dev, custom') in the description string, not as an enum. Timeout and retry bounds are stated in text (timeout default 30 max 600, retries default 3 max 10) but not in schema minima/maxima. LLMs cannot parse these from free-text descriptions reliably.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 30 | 2024-11-05+ | v1 |
Updates an existing MCP tool with field validation
Descriptions lack WHEN context and dependencies. Example: DiscoverServerTools says 'Discovers and registers tools from an MCP server using transport layer connections' but does not explain when to call it (is it a one-time setup? periodic refresh?), what server_id format is expected, or what happens if the server is unreachable. Descriptions are WHAT-oriented, not LLM-optimized.
No error handling or recovery guidance in tool definitions. No description mentions what errors might occur, whether they are retryable, or what the LLM should do next. Example: DeleteTool says 'Soft-deletes (deactivates) an MCP tool' but does not describe failure modes (tool not found, permission denied, etc.) or recovery steps.
Tool names lack consistent verb-prefix convention. DiscoverServerTools and RefreshServerTools are verbose imperative forms; GetTool, UpdateTool, DeleteTool, ListTools follow get/update/delete/list convention; CreateTool breaks to create (not add). Inconsistency adds cognitive load for LLM tool selection. Recommend standardize to: list_tools, get_tool, create_tool, update_tool, delete_tool, discover_server_tools, refresh_server_tools, get_discovered_tools_for_server.
Security and permission documentation absent. No tool describes what permissions are required (e.g., is CreateTool limited to org admins? does DeleteTool require write:tools scope?). No mention of authentication method or user context passed to tools. Agents cannot reason about least-privilege or permission errors.
Parameter identification ambiguities. CreateTool requires both 'name' (unique within org) and 'function_name' (unique function identifier) but descriptions do not clarify the distinction or interaction. Are these independent or must they match? Can you create two tools with different names but same function_name? This ambiguity will cause misuse.
Pagination and result limits not clearly specified. ListTools accepts limit and offset but no description states default limit, maximum, or total result count in response. Large tool lists could exhaust context; LLMs need explicit cardinality guidance.