Loading ToolBench
A multi-MCP framework usecase demonstrating FastMCP server with tools, resources, and prompts
This server demonstrates moderate definition quality with significant gaps. Tools use consistent verb-noun naming (get_*, list_) and most have basic descriptions, but parameter documentation is sparse and output schemas are poorly documented. 5 of 7 tools have input schemas with parameter descriptions, but 2 tools (get_config, list_cities) lack proper schema definition. Descriptions are brief (10-40 chars typical) and lack context on WHEN to use each tool or what happens on error. No parameter constraints (enums, ranges), no error guidance, and no tool annotations present. The codebase shows good intent (structured return types via Pydantic, TypedDict, basic classes) but the MCP registration layer doesn't expose these schemas to clients in a machine-readable way. Overall this reads like an early-stage demonstration rather than production-ready tooling.
This returns unstructured output - no schema generated
Get location coordinates
Get various statistics
Get temperature as a simple float
Get user profile - returns structured data
Get weather for a city - returns structured data.
Get a list of cities
get_config returns unstructured output with no schema. Tool cannot be composed with downstream tools; LLM cannot extract fields or plan chaining. Return type is an untyped Python class (UntypedConfig) which violates pattern:response-shaper.
Output schemas not documented in tool definitions. Tools return structured types (WeatherData, UserProfile, LocationInfo) but the MCP tool registration does not expose these schemas to clients. Clients cannot know what fields to expect without reading source code. This blocks downstream tool composition.
Parameters lack constraints and enums. 'city', 'data_type', 'user_id', 'address' parameters accept free-form strings with no validation. No enum values, no format constraints, no length limits. LLM must guess valid values and may pass hallucinated data.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2025-06-18+ | v2 |
| 2026-03-09 | F | 44 | - | v1 |
Descriptions are too brief and lack context. Most tool descriptions (26-48 chars) state WHAT the tool does but not WHEN to use it or how it differs from similar tools. Descriptions should answer: What does it do? When use this vs alternatives? What does it return? Example: 'get_temperature' and 'get_weather' both take a city, documentation doesn't clarify the difference.
No error handling guidance. Tools provide no recovery paths. Example: get_user with invalid user_id returns nothing; description doesn't suggest 'Call search_users() to find the ID first.' Pattern:recovery-guide requires actionable error messages.
list_cities lacks pagination. Returns hardcoded 3-city list with no limit, offset, or pagination support. Real deployments would fail if city list grows. Per pattern:paginated-result, list tools must support pagination even for small result sets.
Duplicate or overlapping tools not clarified. Both get_weather and get_temperature accept a 'city' parameter. Descriptions don't explain when to call each, LLM reasoning overhead and risk of wrong tool selection.
Parameter descriptions are generic or circular. Example: 'data_type' parameter has description 'Type of statistics to retrieve' but no valid enum values listed. Parameter must include valid types or link to discovery tool.