Loading ToolBench
City of Falls Church GIS — Falls Church, Virginia open geospatial data (ArcGIS). Provides access to parcels, zoning, public works & city services datasets via ArcGIS Feature Services.
Three well-named, read-only tools with clear descriptions and complete input schemas. All tools follow verb_noun naming (search_, query_, layer_info). Descriptions are substantive (120 - 180 chars) and explain what each tool does and when to use it. Input schemas include types and descriptions for all parameters. However, output schemas are not documented, LLMs cannot predict what fields to expect from responses. Parameter descriptions lack format constraints (e.g., 'url must be https and end in /FeatureServer/<n>' is stated in code but not in schema descriptions). No error handling guidance or recovery paths documented. Missing pagination documentation for search_datasets despite returning a 'total' count.
Get an ArcGIS Feature/Map Service layer's schema by url: fields (name + type), geometry type, total record count, and capabilities.
Query an ArcGIS Feature Service / Map Service layer by its url (from search_datasets). SQL-like `where`, comma-separated `out_fields`, `order_by`, `limit`, `offset`. Returns attribute rows (and geometry). Use where="1=1" + out_fields="*" to sample.
Search City of Falls Church GIS open geospatial datasets (parcels, zoning, public works & city services) by keyword. Returns each dataset's name, summary, record_count, owner/org, and its Feature Service `url` — pass that url to query_layer / layer_info.
Output schemas not documented. LLMs cannot predict response structure (fields, types, pagination). search_datasets returns {total, count, datasets[]} but this is invisible in the schema.
Parameter descriptions lack actionable format constraints. 'url' param says 'Feature/Map Service layer url' but does not state 'must be https, end in /FeatureServer/<n> or /MapServer/<n>, and not contain IP/localhost'. Constraints are enforced in code but invisible to LLMs.
No error handling guidance. If a query fails (invalid where clause, missing layer), the tool throws a generic error. LLMs receive no recovery path (e.g., 'Try layer_info() to inspect schema first').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
search_datasets returns 'total' count but no pagination guidance. If total > limit, LLMs cannot fetch remaining results, no offset/page parameter documented in response or tool description.
query_layer 'where' parameter accepts arbitrary SQL. No validation examples or common patterns documented (e.g., 'Use where="1=1" to fetch all records'). LLMs may construct invalid SQL.