Loading ToolBench
datos.gov.co — Colombia's national open-data portal (Socrata platform).
Three well-named, read-only tools with complete input schemas and solid descriptions. All tools follow verb_noun naming (search_, dataset_, query_). Descriptions are 150 - 250 chars, within the 10 - 1024 baseline. All parameters have type definitions and descriptions. However, output schemas are not documented, LLMs cannot predict response structure for downstream chaining. No error handling guidance. No tool annotations (readOnlyHint). Missing pagination metadata (total count, next_cursor) in search_datasets despite returning lists.
List the columns of a datos.gov.co dataset: field name (used in SoQL $select/$where), human label, and data type. Call this before query_dataset so you know which fields exist. datasetId is the Socrata 4x4 code from search_datasets.
Query rows from a datos.gov.co dataset using SoQL. datasetId is the Socrata 4x4 code from search_datasets. SoQL clauses: $select (columns / aggregates like "count(*)"), $where (SQL-like filter, e.g. "departamento_nom='BOGOTA' AND edad > 60"), $order ("count desc"), $group, $q (full-text across the row). Use dataset_columns first to learn field names. Returns an array of row objects keyed by field name. Default $limit is 50 (Socrata max per page is 50000).
Search datasets published on Colombia's open-data portal datos.gov.co. Returns each dataset's Socrata 4x4 id (e.g. "gt2j-8ykr"), name, and description (Spanish). Use the id with dataset_columns and query_dataset. Query terms can be Spanish or English (e.g. "covid", "presupuesto", "contratos").
Output schemas not documented. LLMs cannot predict response structure (fields, types, nesting) for search_datasets, dataset_columns, or query_dataset. This breaks downstream tool chaining and forces agents to guess field names.
search_datasets returns a list but lacks pagination metadata. No 'total_count' or 'next_cursor' in response. Agents cannot determine if results are truncated or plan multi-page queries. Default limit=20 is reasonable but undocumented in output.
No error handling guidance. If a dataset ID is invalid, the tool throws a raw HTTP error. No recovery hints (e.g., 'Try search_datasets() first to find valid IDs'). Agents cannot self-correct.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
No tool annotations. All three tools are read-only and idempotent, but lack readOnlyHint and idempotentHint annotations. Agents cannot infer safety properties without explicit hints.
query_dataset description includes example SoQL syntax ('departamento_nom=\'BOGOTA\'') which LLMs may reuse literally. Replace with formal constraints or a link to SoQL documentation.