Loading ToolBench
An intelligent agent communication platform based on LLM with MCP server integration for tool management and execution
AgentChat MCP server has significant quality gaps. Tool definitions are present with basic descriptions and schemas, but fall short of production standards. 11 of 15 tools have input schemas, but many lack parameter descriptions and proper type definitions. Descriptions exist but are generic and lack LLM-optimization guidance. No evidence of error handling patterns, recovery guidance, or idempotency safeguards. The server exposes dangerous operations (delete_mcp_server, delete_user_defined_tool, execute_http_tool) with minimal validation hints. Several tools conflate multiple responsibilities (e.g., create_mcp_server combines registration, configuration, and discovery). Parameter naming is inconsistent: some use suffixes (_id, _name) while others omit them. No evidence of permission gates, audit trails, or secret injection patterns. This is a C-grade server with gaps typical of internal tool integration layers but unsuitable for production agent use.
Create and register an MCP server with tools, parameters, and configuration
Create a user-defined tool with optional OpenAPI schema validation and authentication configuration
Delete an MCP server by ID with permission verification
Delete a user-defined tool with permission verification
Execute an HTTP-based tool registered as an MCP tool by making HTTP requests based on tool API configuration
Get all MCP servers visible to the user, including personal and system servers
execute_http_tool accepts arbitrary HTTP tool definitions and parameters without validation guidance. No schema documentation for the 'tool' or 'arguments' objects. LLM cannot reason about parameter mapping or request construction safety.
Destructive operations (delete_user_defined_tool, delete_mcp_server) lack confirmation/dry-run patterns. No recovery guidance in descriptions. Agents could permanently delete resources without safeguards.
Parameter descriptions are minimal or absent in many tools. 'update_data' in update_mcp_server is a generic object with only 'Dictionary of fields to update', LLM cannot determine valid fields, constraints, or dependencies. Violates pattern:tool-description.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 62 | 2025-06-18+ | v2 |
| 2026-03-09 | D | 56 | - | v1 |
Get all visible tools for the current user including personal tools and system default tools
Get the default logo URL for tools
Get MCP server details by server ID
Get detailed information about all tools in an MCP server including schemas and descriptions
Get user-defined tools created by the current user
Register and import an MCP server by discovering its tools and creating server configuration
Update MCP server configuration and settings
Update a user-defined tool with optional OpenAPI schema validation and permission verification
Validate MCP server configuration before import
Tools expose user/permission context via parameters but lack explicit permission-gate documentation. verify_user_permission logic in ToolService checks AdminUser and user_id ownership, but tool descriptions do not state permission requirements or error conditions when unauthorized.
No output schema documentation. Tools like get_all_tools, get_all_mcp_servers, get_mcp_tools_info return lists, but descriptions do not specify field structure, pagination support, result limits, or chaining IDs needed for downstream calls. Violates pattern:response-shaper.
create_mcp_server conflates multiple responsibilities: server creation, tool registration, configuration validation, and import. Should split into separate tools (create_mcp_server, register_mcp_tools, configure_mcp_server) so agent can compose them independently.
Parameter naming inconsistent. Some tools use suffixes (user_id, mcp_server_id, server_id) while others omit them (payload, tool, arguments). Ambiguous names like 'tool' and 'arguments' in execute_http_tool force LLM guessing.
No evidence of error handling patterns. Descriptions lack recovery guidance (e.g., 'If tool not found, call get_user_defined_tools() first'). No indication of retryable vs fatal errors or what invalid input looks like.
Opaque object parameters lack field documentation. 'openapi_schema' and 'auth_config' in create_tool/update_user_defined_tool are undefined objects. LLM cannot infer required fields, valid structure, or constraints.
No idempotency guarantees documented. Agents will retry on ambiguous failures, create_tool, create_mcp_server, execute_http_tool could cause duplicate tool registrations, duplicate requests, or side effects.