Loading ToolBench
Keyless directory of breweries worldwide — search by city, state, name, type, or postal code; look up a single brewery; find breweries near a lat/lng point; or free-text search across name and city. Data from api.openbrewerydb.org (v1).
Four well-named, read-only tools with complete input schemas and clear descriptions. All tools follow verb_noun naming (search_*, get_*, breweries_near). Descriptions are concise (80 - 180 chars) and explain what each tool does. Input schemas include type definitions and parameter descriptions. However, output schemas are not formally documented, and error handling lacks recovery guidance. No tool annotations (readOnlyHint, idempotentHint) despite all being read-only. Parameter descriptions could be more prescriptive about constraints (e.g., latitude/longitude ranges, limit bounds).
Find breweries near a latitude/longitude point, sorted by distance. Keyless. Returns the origin point and nearby compact brewery records.
Get a single brewery by its Open Brewery DB id (uuid or slug). Keyless. Returns the full compact brewery record.
Search the Open Brewery DB directory by city, state, name, type, or postal code. At least one filter is required. Keyless. Returns compact brewery records (name, type, address, geo, phone, website).
Free-text search across brewery name and city in the Open Brewery DB directory. Keyless. Returns compact brewery records.
Output schemas not documented. LLMs cannot plan downstream calls or extract fields without knowing what each tool returns. search_breweries returns {count, breweries[]} but this is inferred from code, not declared in the tool definition.
No tool annotations despite all tools being read-only and idempotent. Adding readOnlyHint and idempotentHint would signal to agents that these tools are safe to call repeatedly without side effects.
Error responses lack recovery guidance. When search_breweries returns {error: 'provide at least one filter...'}, the LLM knows what went wrong but not what to do next. Should suggest 'Try again with by_city, by_state, by_name, by_type, or by_postal.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
Parameter descriptions lack explicit constraints. 'latitude' and 'longitude' should state valid ranges (e.g., -90 to 90, -180 to 180). 'limit' should explicitly state 'default 20, max 50' in the description, not just in the code.
search_breweries requires 'at least one filter' but the schema does not enforce this. No parameters are marked required, so an LLM could call with no filters and receive an error. Schema should mark at least one filter as required or use oneOf constraint.