Loading ToolBench
Query IMF SDMX 3.0 macroeconomic data — 193 dataflows, WEO, BOP, CPI, exchange rates, 190 countries.
Strong tool definitions with comprehensive descriptions (avg 250+ chars), well-structured schemas, and clear parameter documentation. All 6 tools have explicit input schemas with typed parameters and descriptions. Tool names follow verb_noun convention (imf_list, imf_get, imf_query, imf_dataframe_*). Descriptions are LLM-optimized, explaining WHAT, WHEN, and dependencies. Risk annotations (READ_ONLY, DESTRUCTIVE) are present. Minor gaps: output schemas not formally documented in source; some parameter descriptions could be more concise; no explicit error recovery guidance in descriptions.
List DataCanvas tables and columns staged by a prior imf_query_dataset call. Returns each table's name, row count, and column schema (name + DuckDB type). Required before imf_dataframe_query to discover the table and column names for SQL.
Remove one DataCanvas table or view staged by imf_query_dataset without affecting other tables on the same canvas. Use imf_dataframe_describe to copy the exact table name. A repeated or absent drop returns dropped=false.
Run a read-only SQL SELECT against a DataCanvas table staged by imf_query_dataset. Supports multi-country comparisons, time-series aggregation, and cross-indicator joins. Requires imf_dataframe_describe first to discover table and column names. One SELECT statement per call; a leading WITH … SELECT (CTE) is accepted. DML and DDL are rejected.
Fetch a dataflow's dimension list with a codelist preview for each dimension. Resolves human-readable terms to SDMX codes (e.g. "United States" → USA, "Constant prices" → NGDP_RPCH). Required before imf_query_dataset — SDMX keys are opaque without codelist lookups. Each codelist is capped at the first 50 entries by default, including previews filtered by codelist_filter. Set dimension_id to retrieve one codelist with bounded limit/offset paging after the optional substring filter. Set available_only=true to page codes the dataflow actually publishes, with series and time coverage metadata; availability filtering happens before codelist_filter and paging. The imf://database/{dataflow_id} resource provides the same bounded discovery summary. Country codes are ISO 3-letter (USA, GBR, DEU), not ISO 2-letter (US, GB, DE). The key_format field shows the exact dimension order required by imf_query_dataset. Note: codelists enumerate the code universe, not actual coverage — valid codes can still return no_data if the combination has no series in this dataflow.
Output schemas not formally documented in source code. Tool descriptions explain what is returned (e.g., 'returns each table's name, row count, and column schema') but JSON Schema output definitions are not visible in the provided source.
imf_dataframe_drop lacks explicit error guidance. Description states 'A repeated or absent drop returns dropped=false' but does not explain what the LLM should do if a table is not found or if the drop fails, should it retry, ask the user, or proceed?
imf_query_dataset 'key' parameter description is dense and technical. While accurate, it could benefit from a simpler example and clearer guidance on when to use wildcards vs. explicit codes.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 65 | 2026-07-28+ | v2 |
List IMF SDMX dataflows available on the portal. Entry point for every query: imf_get_database and imf_query_dataset both require a dataflow id obtained here. Vintage (historical snapshot) dataflows such as WEO_2025_OCT_VINTAGE are excluded by default; set include_vintages=true to include them. Results are paged — 50 per call by default, adjustable with limit and offset — and total_count reports how many dataflows matched. Descriptions are shortened here; imf_get_database returns the full text for a single dataflow.
Query a dataflow by dimension key over a time range. Periods are compared as the date spans they name, so a bound coarser than the observation frequency covers every sub-period inside it. A key resolving to several series carries each one's own unit/scale/decimals, inline and on the canvas. Large result sets spill to DataCanvas for SQL analysis.
No explicit idempotency guarantees documented. Tools like imf_dataframe_drop and imf_query_dataset should clarify whether repeated calls with identical parameters are safe to retry without side effects.
imf_get_database 'limit' and 'offset' parameters are only valid with 'dimension_id' but this constraint is documented in descriptions rather than enforced via schema (e.g., conditional required fields). LLMs may pass limit/offset without dimension_id.