Loading ToolBench
A modular, scalable Model Control Protocol client with FastAPI web server for managing MCP servers and LLM providers
MCP-Hive suffers from severe definition quality issues across all dimensions. While 7 tools are declared, only partial source code is visible (web_server.py truncated mid-function). Tool descriptions are minimal (10-50 chars, well below the 194-char baseline for A-grade tools). Parameter schemas are underspecified: most lack detailed type information, validation constraints, and contextual descriptions. The `chat` and `websocket_endpoint` tools define parameters but lack descriptions for critical fields like 'broadcast' (what happens when true vs false?). The `call_tool` accepts freeform 'tool_args' as an object with no shape specification, violating the constrained-input pattern. No output schemas are documented. Error handling is not visible in the provided code excerpt. Security concerns: `call_tool` accepts arbitrary tool names and arguments with no validation, this is a significant attack surface. The server is HTTP-based (not STDIO-capped), which is positive for protocol readiness, but definition quality is the primary bottleneck here.
Call a tool on an MCP server
Process a chat message
List available LLM providers
List connected MCP servers
Health check endpoint
Set active LLM provider
WebSocket endpoint for real-time chat
Critical: `call_tool` accepts unvalidated 'tool_args' as generic object. No schema constrains what keys/values are acceptable. This is an injection vector, malicious 'tool_name' values or nested 'tool_args' can trigger unintended operations. Violates pattern:constrained-input and pattern:tool-gateway.
High: Tool descriptions are drastically below baseline. 'Health check endpoint' (24 chars), 'List available LLM providers' (30 chars), 'WebSocket endpoint for real-time chat' (38 chars). Baseline is 194 chars; these fail to explain WHEN to use the tool, prerequisites, or what the result enables. LLMs cannot reliably select the right tool.
High: Parameter descriptions are missing or trivial. 'chat' tool has 'broadcast' parameter (boolean) with no description of what happens when true vs false. 'conversation_id' labeled 'Optional conversation ID for context' lacks detail on format, required structure, or how prior conversations are retrieved. LLMs guess.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 36 | - | v1 |
High: No output schemas documented for any tool. Web API responses are implicit. If 'chat' returns a string, object, array, or structured message, that's unknown to the LLM. Agents cannot plan downstream tool calls or extract fields reliably.
High: No error handling guidance visible. HTTP error codes from FastAPI endpoints (e.g. 404, 400) are standard, but tool definitions do not explain recovery. If 'set_provider' receives an invalid provider name, what should the LLM do? Retry? Ask the user? No guidance.
Medium: 'chat' and 'websocket_endpoint' are semantically overlapping. Both accept 'query', 'conversation_id', and 'broadcast'. The distinction is transport (HTTP POST vs WebSocket) not semantic, violates pattern:tool composition rule. Consider merging into a single 'send_chat_message' tool.
Medium: 'call_tool' bypasses LLM tool selection entirely, the LLM names an arbitrary tool and passes arbitrary args. This is a meta-call pattern that disables safety, auditability, and composition. If the goal is to dynamically invoke MCP server tools, enumerate those tools directly and let the LLM select them.