Loading ToolBench
City of Coquitlam GIS — Coquitlam, BC (Canada) open geospatial data (ArcGIS). Provides access to parcels, zoning, parks & city services datasets via ArcGIS Feature Services.
Three well-named, read-only tools with clear descriptions and complete input schemas. All tools follow verb_noun naming (search_, query_, layer_info). Descriptions are substantive (100 - 200 chars) and explain purpose, prerequisites, and data flow. Input schemas include types and descriptions for all parameters. However, output schemas are completely undocumented, LLMs cannot predict what fields to expect from query results, limiting downstream tool composition. Error handling is minimal (generic HTTP status errors with no recovery guidance). No pagination guidance for search_datasets despite potentially large result sets. Missing parameter constraints (e.g., limit ranges are documented in descriptions but not as JSON Schema constraints). No tool annotations (readOnlyHint, idempotentHint) despite all three being read-only and idempotent.
Get an ArcGIS Feature/Map Service layer's schema by url: fields (name + type), geometry type, total record count, and capabilities.
Query an ArcGIS Feature Service / Map Service layer by its url (from search_datasets). SQL-like `where`, comma-separated `out_fields`, `order_by`, `limit`, `offset`. Returns attribute rows (and geometry). Use where="1=1" + out_fields="*" to sample.
Search City of Coquitlam GIS open geospatial datasets (parcels, zoning, parks & city services) by keyword. Returns each dataset's name, summary, record_count, owner/org, and its Feature Service `url` — pass that url to query_layer / layer_info.
Output schemas completely undocumented. LLMs cannot predict response structure (fields, types, nesting) for any of the three tools. This breaks downstream tool composition and forces agents to guess at field names.
No tool annotations despite all three tools being read-only and idempotent. Missing readOnlyHint and idempotentHint metadata prevents agents from understanding retry safety and side-effect profile.
Error handling provides no recovery guidance. Generic 'ArcGIS Hub: 404' or 'ArcGIS query: 500' errors give agents no actionable next steps. Should include suggestions like 'Try search_datasets() with different keywords' or 'Verify the Feature Service URL is correct.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
search_datasets lacks pagination guidance. Description mentions 'limit (1-50, default 20)' but does not explain how to fetch additional results or whether a total_count is returned. Large result sets could exhaust context.
Parameter constraints (limit ranges, required fields) are documented in descriptions but not formalized in JSON Schema (no minimum/maximum, no enum). LLMs cannot parse constraint text reliably.