Loading ToolBench
City of Phoenix GIS — Phoenix, Arizona open geospatial data (ArcGIS). Provides tools to search datasets, query layers, and retrieve layer schema from City of Phoenix GIS open geospatial data.
Three well-named, read-only tools with clear descriptions and complete input schemas. All tools follow verb_noun naming (search_, query_, layer_info). Descriptions are concise (100 - 200 chars) and explain WHAT, WHEN, and WHAT-RETURNS. Input schemas include types and descriptions for all parameters. However, output schemas are NOT documented, LLMs cannot predict response structure. Parameter descriptions lack format constraints (e.g., 'url must be https and end in /FeatureServer/<n>' is in code validation, not schema). No error handling guidance in descriptions. Missing pagination documentation for search_datasets despite returning a 'total' field.
Get an ArcGIS Feature/Map Service layer's schema by url: fields (name + type), geometry type, total record count, and capabilities.
Query an ArcGIS Feature Service / Map Service layer by its url (from search_datasets). SQL-like `where`, comma-separated `out_fields`, `order_by`, `limit`, `offset`. Returns attribute rows (and geometry). Use where="1=1" + out_fields="*" to sample.
Search City of Phoenix GIS open geospatial datasets (parcels, zoning, public works & city services) by keyword. Returns each dataset's name, summary, record_count, owner/org, and its Feature Service `url` — pass that url to query_layer / layer_info.
Output schemas not documented. LLMs cannot predict response structure (fields, types, nested objects). search_datasets returns {total, count, datasets[{id, name, summary, org, org_id, record_count, type, url}]} but this is invisible in the tool definition.
Parameter format constraints embedded in code validation (layerUrl regex, clamp bounds) but NOT in schema descriptions. LLMs cannot read code, they rely on description text. E.g., 'url' param lacks 'must be https and match /FeatureServer/<n> or /MapServer/<n>' in description.
search_datasets returns 'total' count but no pagination guidance. Description does not explain how to iterate through large result sets or whether limit/offset are supported for pagination.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
Error handling not documented. Code throws errors like 'ArcGIS Hub: 404' or 'url must be https' but tool descriptions do not explain what errors are possible, whether they are retryable, or what the LLM should do next.
query_layer 'where' parameter accepts arbitrary SQL but description does not warn about injection risk or document supported SQL syntax. LLMs may pass malicious or unsupported clauses.