Loading ToolBench
City of Sacramento GIS — Sacramento, California open geospatial data (ArcGIS). Provides access to parcels, zoning, public works & city services datasets through ArcGIS Feature Services.
Three well-named, read-only tools with clear descriptions (avg 180 chars, within baseline 194). All tools have input schemas with typed properties and required fields. However, parameter descriptions lack actionable constraints (e.g., 'limit' has no min/max enforcement guidance), output schemas are undocumented, and error handling is minimal. No tool annotations (readOnlyHint, idempotentHint). search_datasets lacks 'required' array; query_layer and layer_info properly declare required params. Composition is sound, tools chain logically (search → query → info). Security: SSRF guard in layerUrl() is excellent; no secrets exposed.
Get an ArcGIS Feature/Map Service layer's schema by url: fields (name + type), geometry type, total record count, and capabilities.
Query an ArcGIS Feature Service / Map Service layer by its url (from search_datasets). SQL-like `where`, comma-separated `out_fields`, `order_by`, `limit`, `offset`. Returns attribute rows (and geometry). Use where="1=1" + out_fields="*" to sample.
Search City of Sacramento GIS open geospatial datasets (parcels, zoning, public works & city services) by keyword. Returns each dataset's name, summary, record_count, owner/org, and its Feature Service `url` — pass that url to query_layer / layer_info.
Output schemas not documented. LLMs cannot predict response structure (fields, types, pagination). search_datasets returns {total, count, datasets[]} but this is inferred from code, not declared in tool definition.
Parameter constraints lack enforcement guidance. 'limit' accepts 1-50 (search_datasets) and 1-2000 (query_layer) but descriptions do not state min/max or what happens if violated. LLMs may pass invalid values.
No tool annotations. Tools are read-only and idempotent but lack readOnlyHint and idempotentHint. Agents cannot infer safety properties without explicit hints.
Error handling is minimal. callTool() throws generic errors ('ArcGIS Hub: 404', 'Unknown tool') without recovery guidance. LLMs cannot self-correct or retry intelligently.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
search_datasets inputSchema missing 'required' array. 'query' is the primary search term but is not marked required, allowing empty searches that may return irrelevant results.