Loading ToolBench
St. Louis County GIS — St. Louis County, Missouri open geospatial data (ArcGIS). Provides access to parcels, addresses, zoning & public works datasets via ArcGIS Feature Services.
Three well-named, read-only tools with clear descriptions and complete input schemas. All tools follow verb_noun naming (search_, query_, layer_info). Descriptions are substantive (150 - 250 chars) and explain purpose, prerequisites, and output. Input schemas include types and descriptions for all parameters. However, output schemas are completely undocumented, LLMs cannot predict what fields to expect from query_layer or layer_info responses. Parameter constraints (e.g., limit ranges 1 - 50, 1 - 2000) are documented in descriptions but not enforced via JSON Schema enums/min/max. No error handling guidance, tools throw generic errors without recovery hints. Missing tool annotations (readOnlyHint) despite all three being read-only. Composition is sound: search_datasets returns Feature Service URLs that feed directly into query_layer and layer_info.
Get an ArcGIS Feature/Map Service layer's schema by url: fields (name + type), geometry type, total record count, and capabilities.
Query an ArcGIS Feature Service / Map Service layer by its url (from search_datasets). SQL-like `where`, comma-separated `out_fields`, `order_by`, `limit`, `offset`. Returns attribute rows (and geometry). Use where="1=1" + out_fields="*" to sample.
Search St. Louis County GIS open geospatial datasets (parcels, addresses, zoning & public works) by keyword. Returns each dataset's name, summary, record_count, owner/org, and its Feature Service `url` — pass that url to query_layer / layer_info.
Output schemas completely undocumented. LLMs cannot predict response structure for query_layer (returns ArcGIS JSON with features, attributes, geometry) or layer_info (returns fields, geometry type, capabilities). Forces agents to guess field names and risk extraction errors.
Parameter constraints (limit 1 - 50 for search_datasets, 1 - 2000 for query_layer) documented only in descriptions, not in JSON Schema. LLMs cannot parse natural-language constraints reliably. Should use minValue/maxValue in schema.
No error handling guidance. Tools throw generic errors ('ArcGIS Hub: 404', 'ArcGIS query: 500') without recovery hints. LLMs cannot determine if error is retryable, user-fixable, or fatal.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Missing tool annotations. All three tools are read-only but lack readOnlyHint annotation. Agents cannot infer safety properties without explicit hints.
query_layer and layer_info accept arbitrary ArcGIS Feature/Map Service URLs. While layerUrl() validates HTTPS and blocks IPs/localhost, no rate limiting or quota enforcement. Agents could hammer external Esri services.