Loading ToolBench
Singapore open data + real-time environment/transport feeds
Server defines 9 tools with basic schemas and descriptions. All tools have names starting with action verbs (search_, get_, query_, weather_, air_quality_, uv_, taxi_, traffic_). Descriptions are present but brief (10-80 chars, below the 194-char baseline). Input schemas are visible and typed, but lack depth: parameter descriptions are minimal, no enums for constrained inputs, no output schemas documented. Three tools (weather_now, air_quality_psi, air_quality_pm25, uv_index, taxi_availability, traffic_incidents) have empty input schemas with no parameters, acceptable for read-only feeds but descriptions could explain what data is returned. query_dataset accepts a filters object with additionalProperties but no guidance on valid column names or filter semantics. No error handling guidance, no recovery hints, no pagination documentation for search_datasets despite returning paginated results.
Current PM2.5 µg/m³ readings by region.
Current Pollutant Standards Index (PSI) by region (north, south, east, west, central).
Dataset metadata + column schema.
Fetch rows from a dataset. Supports limit, offset, and filter map (column → value).
Browse / search the data.gov.sg dataset catalog.
Live taxi positions across Singapore.
Current incidents on expressways and major roads.
No output schemas documented. LLMs cannot plan downstream calls or extract fields without knowing what each tool returns. query_dataset, weather_now, and real-time feeds lack return type documentation.
Descriptions are too brief (10-80 chars vs. 194-char baseline). weather_now, uv_index, taxi_availability, traffic_incidents lack context on what data is returned and when to use them. LLMs cannot distinguish between similar real-time feeds.
query_dataset filters parameter lacks guidance. No documentation of valid column names, filter syntax, or examples. LLMs will guess at filter structure, causing failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Current UV index.
Current temperature, humidity, wind, rain across Singapore weather stations.
No error handling guidance. callTool throws generic errors ('Unknown tool', 'data.gov.sg: not found', 'rate-limit') without recovery hints. LLMs cannot self-correct or know whether to retry.
search_datasets pagination not documented. Tool accepts page and page_size but description does not explain pagination behavior, total count, or how to iterate. LLMs may not know pagination is available.