Loading ToolBench
City of Fairfield GIS — Fairfield, California open geospatial data (ArcGIS). Provides access to parcels, zoning, public works & city services datasets through ArcGIS Feature Services.
Three well-named, read-only tools with clear descriptions and complete input schemas. All tools follow verb_noun naming (search_, query_, layer_info). Descriptions are concise (100 - 200 chars) and explain WHAT, WHEN, and WHAT-TO-DO-NEXT. Input schemas include types and descriptions for all parameters. However, output schemas are NOT documented, LLMs cannot predict response structure. Parameter descriptions lack format constraints (e.g., 'url must be https and end in /FeatureServer/<n>' is stated in code but not in schema descriptions). No error handling guidance in descriptions. Missing pagination guidance for search_datasets despite returning a 'total' count.
Get an ArcGIS Feature/Map Service layer's schema by url: fields (name + type), geometry type, total record count, and capabilities.
Query an ArcGIS Feature Service / Map Service layer by its url (from search_datasets). SQL-like `where`, comma-separated `out_fields`, `order_by`, `limit`, `offset`. Returns attribute rows (and geometry). Use where="1=1" + out_fields="*" to sample.
Search City of Fairfield GIS open geospatial datasets (parcels, zoning, public works & city services) by keyword. Returns each dataset's name, summary, record_count, owner/org, and its Feature Service `url` — pass that url to query_layer / layer_info.
Output schemas not documented. LLMs cannot predict response structure (fields, types, nested objects). search_datasets returns {total, count, datasets[{id, name, summary, org, org_id, record_count, type, url}]} but this is invisible in the tool definition.
Parameter descriptions lack actionable format constraints. 'url' param says 'Feature/Map Service layer url' but does not state 'must be https, must match /FeatureServer|MapServer/\d+, no IP addresses or localhost'. Constraints are enforced in code (layerUrl function) but invisible to LLM.
No error handling guidance in descriptions. query_layer and layer_info can fail with 'ArcGIS query: 404' or 'ArcGIS layer_info: 403' but descriptions do not explain what to do on failure (e.g., 'If layer not found, try search_datasets to find valid urls').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
search_datasets returns 'total' count but no pagination guidance. Description does not explain how to iterate through large result sets (use limit + offset pattern or cursor). Agents may assume all results fit in one call.
query_layer 'where' parameter accepts arbitrary SQL but description does not warn about injection risk or document escaping rules. LLMs may pass unescaped user input directly.