Loading ToolBench
Two tools with minimal descriptions (both ~40 chars, baseline 194 chars avg). Input schemas present but extremely sparse: single string parameter with basic description, no enums/constraints/validation. No output schema documented. No error handling guidance. Parameter descriptions lack format/range constraints. Tools lack actionable context for LLM selection. No pagination, no structured output guidance.
Get hourly weather forecast for a location.
Get current weather forecast for a location.
Descriptions are too short (40 chars vs 194 baseline). 'Get current weather forecast for a location' lacks WHEN to use, WHAT it returns, and prerequisites. LLMs cannot distinguish between get_current_weather and get_7day_weather without reading code.
Input schema lacks constraints. 'location' parameter accepts any string with no enum, pattern, or format guidance. No validation hints (e.g., 'City, Country format' or 'Nominatim geocoding required'). LLMs will pass invalid location strings without guidance.
No output schema documented. get_7day_weather returns dict of records with fields (date, temperature_2m, relative_humidity_2m, precipitation); get_current_weather returns JSON string with 7 fields. LLMs cannot plan downstream operations or extract fields without seeing the schema.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 42 | 2026-07-28+ | v2 |
No error handling or recovery guidance. If geocoding fails (location not found), get_lat_long returns None silently. No error message tells the LLM to retry with a different location string or call a search_locations tool. Agents will see cryptic failures.
Output format inconsistency. get_7day_weather returns list of dicts (structured); get_current_weather returns JSON string (unstructured). LLMs must parse JSON string manually, wasting tokens and inviting parse errors. Both should return structured objects.