Loading ToolBench
Linn County GIS — Linn County, Iowa open geospatial data (ArcGIS). Provides access to parcels, addresses, zoning & public works datasets via ArcGIS Feature Services.
Three well-named, read-only tools with clear descriptions and complete input schemas. Naming follows verb_noun pattern (search_datasets, query_layer, layer_info). Descriptions are substantive (120 - 180 chars) and explain purpose and prerequisites. All parameters have type definitions and descriptions. However, output schemas are not documented, LLMs cannot predict response structure for downstream planning. Error handling is minimal (generic HTTP status errors). No parameter constraints (enums, ranges) despite opportunities (e.g., limit 1 - 50 for search_datasets is stated in description but not enforced in schema). Missing output documentation is the primary gap preventing a higher score.
Get an ArcGIS Feature/Map Service layer's schema by url: fields (name + type), geometry type, total record count, and capabilities.
Query an ArcGIS Feature Service / Map Service layer by its url (from search_datasets). SQL-like `where`, comma-separated `out_fields`, `order_by`, `limit`, `offset`. Returns attribute rows (and geometry). Use where="1=1" + out_fields="*" to sample.
Search Linn County GIS open geospatial datasets (parcels, addresses, zoning & public works) by keyword. Returns each dataset's name, summary, record_count, owner/org, and its Feature Service `url` — pass that url to query_layer / layer_info.
Output schemas not documented. LLMs cannot predict response structure (fields, types, nested objects) for downstream tool calls or data extraction. search_datasets returns {total, count, datasets[{id, name, summary, org, org_id, record_count, type, url}]} but this is invisible to the agent.
Parameter constraints stated in descriptions but not enforced in schema. 'limit' (1 - 50 for search_datasets, 1 - 2000 for query_layer) should use minimum/maximum in JSON Schema. Descriptions alone are not machine-parseable; LLMs may pass invalid values.
Error handling is generic. HTTP status errors ('ArcGIS Hub: 404', 'ArcGIS query: 500') do not guide recovery. No distinction between retryable (network timeout) vs user-fixable (invalid URL) vs fatal errors. LLM cannot determine next action.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
query_layer and layer_info accept arbitrary ArcGIS URLs. While layerUrl() validates HTTPS and blocks IPs/localhost, it does not validate that the URL actually belongs to Linn County GIS or a trusted org. An agent could be tricked into querying a malicious ArcGIS server.
No pagination support in query_layer response. If a query matches 10,000 features but limit=2000, the agent receives only the first 2000 with no indication of total count or next_cursor. Large result sets risk context window exhaustion.