Loading ToolBench
A demonstration MCP server that provides weather tools using the NOAA Weather API
Two tools with minimal schema documentation and generic descriptions. Tool definitions are visible in WeatherService.cs with C# attributes, but descriptions lack specificity for LLM selection. Parameters have type information via method signatures, but descriptions are terse (10-30 chars). No output schema documentation. No error handling guidance. No composition with other tools evident. Naming follows verb_noun pattern (GetAlerts, GetForecast) which is correct, but descriptions do not explain WHEN to use each tool or differentiate from similar weather services.
Get weather alerts for a US state.
Get weather forecast for a location.
Output schemas are not documented. Tools return string responses with no defined structure (GetAlerts returns multi-line alerts joined by '--\n', GetForecast returns formatted periods). LLMs cannot parse these outputs reliably or chain results to downstream tools.
Descriptions are under 50 characters and lack context for LLM selection. 'Get weather alerts for a US state.' does not explain when to prefer this over a general weather API, what 'alerts' means (tornado, flood, winter storm?), or what data structure the LLM should expect.
Parameter descriptions are minimal. 'The US state to get alerts for.' (39 chars) and 'Latitude of the location.' (25 chars) lack format hints, valid value constraints, or examples. No mention of coordinate range (latitude -90..90, longitude -180..180), state code format (2-letter abbr or full name?), or required precision.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 35 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 36 | - | v1 |
No error handling guidance. What happens if state='XX' (invalid)? If lat/lon resolve to international waters? Tools return unstructured strings; no recovery hints for LLMs if the API fails or returns empty results.
No pagination or result limiting. GetAlerts calls .EnumerateArray() on all features without a limit; if a state has 500+ active alerts, the response explodes token usage. GetForecast returns all periods without truncation. Both tools should cap results and document the limit in descriptions.
Parameter naming for GetForecast uses 'latitude' and 'longitude' but does not clarify format. Are these decimal degrees? Do they accept string coords? No validation rules documented. LLM might pass 'north 45 degrees' literally instead of 45.0.