Loading ToolBench
Town of Oro Valley GIS — Oro Valley, Arizona open geospatial data (ArcGIS). Provides access to parcels, zoning, public works & city services datasets via ArcGIS Feature Services.
Three well-named, read-only tools with clear descriptions and complete input schemas. All tools follow verb_noun naming (search_, query_, layer_info). Descriptions are substantive (120 - 180 chars) and explain what each tool does and when to use it. Input schemas include type definitions and parameter descriptions. However, output schemas are not documented, LLMs cannot predict what fields to expect from responses. Parameter descriptions lack explicit constraints (e.g., 'limit 1 - 50' is stated but not in schema format). Error handling is minimal, no recovery guidance or actionable error messages. No tool annotations (readOnlyHint, idempotentHint) despite all being read-only. Missing documentation of response structure and field mappings between tools (e.g., search_datasets returns 'url' which query_layer expects).
Get an ArcGIS Feature/Map Service layer's schema by url: fields (name + type), geometry type, total record count, and capabilities.
Query an ArcGIS Feature Service / Map Service layer by its url (from search_datasets). SQL-like `where`, comma-separated `out_fields`, `order_by`, `limit`, `offset`. Returns attribute rows (and geometry). Use where="1=1" + out_fields="*" to sample.
Search Town of Oro Valley GIS open geospatial datasets (parcels, zoning, public works & city services) by keyword. Returns each dataset's name, summary, record_count, owner/org, and its Feature Service `url` — pass that url to query_layer / layer_info.
Output schemas not documented. LLMs cannot predict response structure, field names, or types. Agents cannot plan downstream calls or extract required data reliably.
No tool annotations despite all tools being read-only. Missing readOnlyHint and idempotentHint metadata prevents agents from optimizing retry logic and understanding safety guarantees.
Parameter constraints stated in descriptions but not formalized in schema. 'limit 1 - 50' is prose; should be minValue/maxValue in JSON Schema. LLMs cannot parse prose constraints reliably.
Error handling lacks recovery guidance. Errors are thrown as generic messages (e.g., 'ArcGIS Hub: 404'). No actionable next steps or error classification (retryable vs. user-fixable).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
Tool chaining not documented. search_datasets returns 'url' field; query_layer and layer_info expect 'url' parameter. No explicit guidance on how to chain these tools or what fields flow between them.