Loading ToolBench
Single tool with clear naming (getWeatherForecast) and reasonable description (73 chars). Input schema is well-defined with Zod types, proper descriptions, and constraints (min/max for days). However, output schema is not documented, the tool returns text + UI resource but LLMs cannot see the expected structure. Error handling is present but generic (no recovery guidance). Tool is read-only and idempotent, which is good, but lacks explicit annotations. Missing output documentation is the primary gap preventing a higher score.
Give a weather forecast for a city with interactive UI visualization
Output schema not documented. Tool returns text + UIResource but LLMs cannot see the expected structure (fields, types, format). This forces LLMs to guess what data is available for downstream reasoning.
Error messages lack recovery guidance. Errors like 'API Error: {statusText}' or 'An unexpected error occurred: {message}' do not tell the LLM what to do next (retry, ask user, try different params). Per pattern:recovery-guide, errors should be actionable.
Tool lacks explicit idempotent/read-only annotations. The tool is read-only and safe to retry, but this is not declared via toolAnnotations (readOnlyHint). Modern MCP clients use these hints for optimization and safety.
Prompt template is generic and does not guide LLM toward meaningful weather queries. 'Please provide a friendly weather forecast' lacks context (no city, no timeframe). Should suggest concrete examples or ask for required parameters.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
API key exposed in error message path. While the key itself is not logged, the code path that constructs URLs with apiKey could leak it if exceptions are logged with full stack traces. Use server-side secret injection pattern.