Loading ToolBench
MCP server for spatial analysis over remote GeoParquet files, read in place by DuckDB.
Strong spatial domain focus with well-structured tool definitions. All 8 tools have clear descriptions (avg 85 chars) and complete input schemas with proper types. Naming follows verb_noun pattern consistently (geoparquet_*). However, output schemas are not documented in the visible code, parameter descriptions lack format/constraint details (e.g., WKT format, SQL injection warnings), and error handling guidance is absent. No tool annotations (readOnlyHint, etc.) despite all being READ_ONLY. Missing recovery guidance for common failures (invalid geometry, SQL errors, missing sources).
Aggregate statistics (counts, averages) computed remotely over filtered features.
Count features from one dataset that fall within polygons from another dataset.
One dataset's schema, CRS and extent.
Features inside a rectangle or WKT geometry, as GeoJSON.
Closest features to a point, with distances.
The first rows of a dataset.
Execute arbitrary SQL query over GeoParquet sources for complex joins and window functions.
Spatial density summary using H3 hexagonal grid.
Output schemas not documented. LLMs cannot plan downstream operations or extract required fields (e.g., geometry format, coordinate system in results). Rubric baseline: 100% of A+ tools document return types.
geoparquet_run_sql accepts arbitrary SQL with no validation or injection warnings. Parameter description lacks guidance on safe query patterns, supported functions, or error recovery. Agents may pass malicious or malformed SQL without guidance.
No tool annotations (readOnlyHint, idempotentHint). All 8 tools are READ_ONLY and idempotent, but this is not declared in the schema. Agents cannot infer safety properties and may over-retry or apply unnecessary confirmation logic.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Spatial parameter descriptions lack format guidance. 'WKT geometry' and 'bounding box' are mentioned but no examples or validation rules (e.g., 'WKT must be valid OGC format; POLYGON((lon lat, ...))' or 'bbox: min_lon < max_lon, -180 ≤ lon ≤ 180'). Agents may pass invalid geometries.
No error recovery guidance. If a source is not found, SQL fails, or geometry is invalid, the tool description does not hint at recovery steps (e.g., 'Call geoparquet_describe_source to list available sources' or 'Ensure WKT is valid OGC format'). Agents dead-end on errors.