Loading ToolBench
MCP server providing tools to query OpenStreetMap data including amenities, buildings, emergency services, and geographic information using Overpass API
The puca server provides 9 tools with mixed quality. All tools have descriptions and basic input schemas visible in the source code. However, there are significant gaps: (1) Descriptions are generic/copy-pasted (tools 7-8 claim to show 'parking' instead of toilets/post offices), (2) No output schema documentation, tools return free-form strings without structured fields, (3) Limited parameter validation, distance parameters lack bounds, no enum constraints where applicable, (4) Error handling is minimal, most error messages are identical generic strings, (5) No idempotency guarantees or confirmation steps for tools that query external services. Tool naming is clear (verb_noun pattern, action-oriented), but parameter descriptions lack detail. The tools are read-only (low risk) but would benefit from stricter input validation and richer error recovery guidance. Average tool score: 52/100.
Get the coordinates of a given address
Get locations of Coffee Shops/Cafes within a radius around an address
Get the coordinates of a given address
Get locations of defibrillators/AEDs within a radius around a given address
Get the distance in metres between two addresses. The addresses are converted to coordinates and OSRM is queried to get a route between the points and the route distance.
Get the distance in metres between two sets of coordinates. OSRM is queried to get a route between the points and the route distance.
Copy-pasted descriptions with incorrect text. Tools 7 (get_toilets) and 8 (get_post_offices) claim to show 'parking' instead of their actual amenity type. This confuses LLMs about what each tool does.
No output schema documentation. All tools return free-form strings (e.g., 'Latitude: {coords.lat}, Longitude: {coords.lon}'). LLMs cannot parse these reliably or chain outputs to downstream tools. Expected: structured JSON with typed fields (latitude: number, longitude: number, address: string, etc.).
Unbounded distance parameter. All *_radius tools (get_defibrillators, get_parking, get_toilets, etc.) accept 'distance: int' with no documented min/max. LLMs could pass absurd values (0, 999999999) causing timeouts or API errors. Should specify e.g., 'distance: int (1-5000 metres)'.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 67 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 38 | - | v1 |
Get details about parking available within a radius around an address. This will only show results of parking that has been added by volunteers to OpenStreetMap.
Get details about post offices available within a radius around an address. This will only show results of parking that has been added by volunteers to OpenStreetMap.
Get details about public toilets available within a radius around an address. This will only show results of parking that has been added by volunteers to OpenStreetMap.
Identical generic error message ('Unable to get valid coordinates for the address requested. Please check the spelling of the address.') across tools. Does not help LLMs recover. Missing actionable guidance: should suggest search_* tools or explain why validation failed.
Parameter get_address_by_coordinates has docstring that says 'address: A valid address' but the actual parameter is (lat, lon). Docstring is incorrect and confuses developers.
No pagination or result limits documented. Tools like get_parking or get_cafes could return hundreds of results as unstructured text, blowing context windows. Should cap results (e.g., 20-50) and offer pagination (limit, offset/cursor).
No input validation beyond basic type checking. Coordinates (-90 to 90 for lat, -180 to 180 for lon) are documented in one tool but not validated at runtime. Addresses are free-form strings with no length limits or injection protection.