Loading ToolBench
A multi-service financial MCP server consisting of MCP Servers registry, MCP Hosts agent, and Yahoo Finance real-time price service
This server exhibits severe definition quality issues across nearly all dimensions. Tool definitions are largely inferred from API endpoint documentation rather than explicitly registered with proper schemas. Of 8 tools listed, only 2 have even basic parameter documentation visible in source (interact, get_realtime_price). The majority of tool schemas are missing or completely undocumented. Descriptions are generic, non-instructive, and fail to guide LLM decision-making. Parameter documentation is almost entirely absent. No error handling guidance is visible. The 'dynamic_tools_from_registry' tool is particularly problematic, it admits to having 'Variable schema based on discovered tool specifications' with no fixed contract. Three tools are duplicates (health endpoints in different services). The codebase shows HTTP API implementations but tool registration follows no MCP standard pattern, tools appear to be Flask/FastAPI endpoints, not properly defined MCP tool specifications. Code review of tools_discovery.py shows dynamic tool creation but with no guarantee of quality schemas for discovered tools.
Dynamically discovered tools from MCP Servers registry. Tools are created at runtime by fetching specifications from the registry endpoint and converting them to LangChain StructuredTools
Provide a list of all available provider configurations
Retrieve real-time stock prices from Yahoo Finance
Provide a list of all available tool specifications. Agent hosts call this endpoint when starting up to discover capabilities
Health check endpoint. Returns a simple status response to indicate the application is running
Health check endpoint for MCP Servers. Returns a simple status response to indicate the application is running
Health check endpoint for Yahoo Finance service. Returns a simple status response to indicate the application is running
No explicit MCP tool registration found in source code. Tools appear to be HTTP API endpoints, not formally registered MCP tools with proper schemas, descriptions, and metadata. This violates the fundamental MCP tool definition pattern.
Input schemas completely missing or undocumented for 5 of 8 tools (dynamic_tools_from_registry, get_tools, get_providers, health endpoints). Even tools with parameters lack formal JSON Schema definitions with type constraints and descriptions.
Three duplicate 'health' tools across different services (mcp_hosts, mcp_servers, yf). Duplicate tool names cause LLM disambiguation failures. Each service should expose a single health check or tools should have unique names (health_mcp_hosts, health_mcp_servers, health_yf).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 26 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 15 | - | v1 |
Main endpoint for interacting with the financial agent. Receives user messages and processes them through the agent, which may use various tools to gather information and formulate responses
'dynamic_tools_from_registry' tool admits to 'Variable schema based on discovered tool specifications from registry endpoint'. This is not a proper tool definition, it cannot provide a fixed input schema, making it impossible for clients to invoke correctly. Violates tool definition contract.
Parameter descriptions missing or trivial for most tools. 'session_id' in 'interact' tool described only as 'Unique identifier for the conversation session', no guidance on format, length, or generation. LLMs cannot infer correct usage.
Tool descriptions are generic and fail to answer key questions: What does it do? When should I call it vs a similar tool? What does it return? Example: 'Provide a list of all available tool specifications' gives no context on why this discovery matters or when to invoke it.
No output schema documentation visible. Tools do not document what they return, forcing LLMs to guess at response structure. Tools like 'get_tools' and 'get_providers' offer no schema for their list responses.
No error handling guidance visible in any tool definition. No indication of what errors can occur, how to recover, or what the LLM should do next. Tools provide no recovery patterns.
Tool naming is inconsistent and ambiguous. 'interact' is vague, does it search, retrieve, analyze, or execute? 'get_realtime_price' is clear but stands alone. No consistent verb_noun pattern across the toolkit.