Loading ToolBench
MCP server for managing city assessments across multiple dimensions (policing, health, economy, government). Resolves cities to OpenStreetMap boundaries, imports facilities, manages indicator submissions, and provides verdicts based on published data.
Groundtruth demonstrates strong naming conventions (verb-first: create_city, get_template, list_gaps, submit_value, assess) and comprehensive parameter descriptions. All 5 tools have detailed, context-rich descriptions (150-400+ chars) that explain WHAT, WHEN, and WHY. Input schemas are fully specified with types, enums, and constraints. However, output schemas are not documented in the visible code, responses are inferred from tool logic rather than explicitly declared. Error handling is present but inconsistent: create_city returns structured {ok, error, candidates} for ambiguity, but other tools throw exceptions that server.ts must catch. The submit_value tool has exceptionally complex nested schema (derivation object with inputs array) that is well-documented but could benefit from clearer examples of valid payloads.
The verdict for a layer: each dimension against its own benchmarks, the worst level across them, and how many dimensions are sourced. Nothing is weighted or averaged, so every part of the verdict traces back to one published value and one benchmark with a source. Only published values count — pending submissions do not move it.
Resolve a city to an OpenStreetMap administrative boundary and import its facilities. If the name is ambiguous the tool creates nothing and returns the candidate relations — call it again with osm_relation_id to choose. Never guess which candidate is right; a city in the wrong country silently poisons every assessment built on it.
The research protocol for a layer: every dimension, the unit, the geography it must be measured at, which source tiers are acceptable, a plausible value range, and the authoritative source to use. Read this before gathering anything. Following it is what makes the resulting numbers defensible enough to put to an official body.
The worklist: dimensions in this layer with no published value, or whose newest value is too old to rely on. Each gap carries its own template guidance, so it can be worked straight from this response. Gaps already carrying a pending submission are marked — do not gather those again.
Output schemas not documented. Tool descriptions explain inputs but do not specify what fields/structure the response contains. LLMs cannot plan downstream calls without knowing response shape.
Inconsistent error handling. create_city returns structured {ok: false, error, candidates} for validation/ambiguity, but other tools throw exceptions. Agents cannot distinguish retryable errors from fatal ones without catching and parsing exceptions.
submit_value derivation object is deeply nested with conditional requirements (required only when kind='estimated'). No validation guidance in description for when/how to populate nested inputs array. LLMs may omit required fields.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 69 | 2026-07-28+ | v2 |
Record one value for one dimension. Always lands as pending and is never published directly. Rejected without a source_url, at the wrong geography, outside the dimension's plausible range, or as an estimate with no derivation. Rejections come back as a list of what to fix — read them, correct the submission, and resubmit rather than working around them.
No pagination support documented. list_gaps and assess may return large result sets (all dimensions in a layer). No limit, offset, or cursor parameters visible. Large responses risk context window exhaustion.
create_city accepts osm_relation_id as optional number but does not validate it exists or belongs to the requested name. Invalid IDs silently fail or create wrong boundaries. Should validate and return actionable error.