Loading ToolBench
Delaware County, Indiana open geospatial data (ArcGIS) — parcels, addresses, zoning & public works. Search datasets, query layers, and retrieve layer schemas from ArcGIS Feature Services.
Three well-named, read-only tools with complete input schemas and clear descriptions. All tools follow verb_noun naming (search_*, query_*, layer_info). Descriptions are substantive (120 - 180 chars) and explain purpose, prerequisites, and data flow. Input schemas include types and required fields. However, output schemas are not documented, callers cannot predict response structure. Parameter descriptions lack format constraints (e.g., 'url must be https' is enforced in code but not stated in schema). Error handling is minimal (generic HTTP status errors). No pagination guidance for search_datasets despite returning potentially large result sets.
Get an ArcGIS Feature/Map Service layer's schema by url: fields (name + type), geometry type, total record count, and capabilities.
Query an ArcGIS Feature Service / Map Service layer by its url (from search_datasets). SQL-like `where`, comma-separated `out_fields`, `order_by`, `limit`, `offset`. Returns attribute rows (and geometry). Use where="1=1" + out_fields="*" to sample.
Search Delaware County GIS open geospatial datasets (parcels, addresses, zoning & public works) by keyword. Returns each dataset's name, summary, record_count, owner/org, and its Feature Service `url` — pass that url to query_layer / layer_info.
Output schemas not documented. Callers cannot predict response structure (e.g., search_datasets returns {total, count, datasets[]} but this is not declared in tool definition). LLMs must infer output shape from trial, wasting tokens and risking misinterpretation.
Parameter format constraints enforced in code (e.g., layerUrl() validates https, hostname, pathname) but not declared in schema descriptions. LLMs cannot see these constraints and may pass invalid URLs, causing runtime errors instead of self-correcting.
search_datasets returns potentially large result sets (limit up to 50) but no pagination guidance. If total > limit, LLM has no way to fetch next page. Description should mention 'Use limit and offset for pagination' or similar.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
Error messages are generic HTTP status codes ('ArcGIS Hub: 404', 'ArcGIS query: 500'). No recovery guidance. LLM cannot determine if error is retryable, user-fixable, or fatal.
query_layer 'where' parameter accepts arbitrary SQL. No validation or injection guards documented. Description should warn: 'SQL injection risk, sanitize user input before passing.'