A collection of multiple MCP servers including weather, trading, filesystem, and demo applications
Scoring was not performed
Output schemas completely undocumented. No tool shows what it returns or what fields the LLM should expect. (e.g., calculate_bmi returns a float with no context; fetch_weather returns raw response.text; query_db returns untyped string). LLMs cannot plan downstream tool calls or extract required data without knowing response structure.
Parameter descriptions missing or generic. 'ctx' parameter in query_db and long_task documented as 'Request context with lifespan context' and 'Request context for progress reporting', these tell the LLM nothing about what value to pass or why. Parameters 'weight_kg', 'height_m', 'city', 'a', 'b' have minimal descriptions that lack usage context or constraints.
No error handling or recovery guidance in any tool. fetch_weather calls httpx without try/catch or timeout; it will fail silently or hang. No tool returns actionable error messages. No tool categorizes errors as retryable vs fatal. Agents have no path to self-correct on failure.
Vague tool descriptions that fail to distinguish tools or guide LLM selection. 'Add two numbers' (add tool) lacks context on when to use it vs other numeric tools. 'Tool that uses initialized resources' (query_db) is uninformative, what does it query? What does it return? 'Process multiple files with progress tracking' (long_task) does not explain what processing happens. Description lengths (add, query_db, long_task) are 17, 35, 41 chars, below the 50-200 char production baseline.
Ambiguous or overloaded parameters. 'ctx' parameter (query_db, long_task) is opaque, LLMs cannot know how to construct or populate a Context object. It appears to be a framework-injected parameter, not one the agent should pass. If so, this should be hidden from the tool interface or explicitly marked as internal.
No input validation or constraint documentation. Numeric parameters (weight_kg, height_m, a, b) have no min/max bounds. 'files' in long_task is a list with no size limit. fetch_weather accepts 'city' as a free string with no format constraint. LLMs can pass invalid or nonsensical values; tools do not guide correction.
Generic naming for some tools reduces discoverability. 'add' is extremely generic, in a multi-tool environment, an LLM might confuse it with API operations or misuse it. Consider 'add_integers' or 'calculate_sum'. 'query_db' is vague, what database? What query language? What data is returned?
Destructive tool (create_thumbnail) lacks confirmation step or dry-run support. The tool modifies the filesystem by creating an image file. No warning in the description. No permission check or confirmation before execution. Agents should be gated on write operations.
No pagination support for list operations. long_task accepts an unbounded 'files' array. If an agent passes 1000 files, processing will explode. No limit stated in description. No offset/limit parameters to handle large results gracefully.
Unresolvable external dependency in fetch_weather. URL 'https://api.weather.com/{city}' does not appear to be a real service (no authentication, no standard weather API format). Tool will fail on any invocation. No error handling to communicate failure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 0 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 33 | - | v1 |