Loading ToolBench
BPstat — Banco de Portugal statistics API (Portuguese central bank). Keyless.
Four well-named tools with clear, detailed descriptions (avg 180 chars) and complete input schemas. All parameters are typed and described. Tool names follow verb_noun pattern (list_*, get_*). Descriptions explain WHAT, WHEN, and dependencies. However, output schemas are not documented, LLMs cannot predict response structure for planning. No error handling guidance. No pagination support despite potential for large datasets. Missing tool annotations (readOnlyHint). Composition is sound: tools chain logically (list_domains → list_datasets → get_dataset; get_series_metadata → get_dataset). All tools are read-only and stateless.
Fetch a dataset as JSON-stat — THIS is how you get actual data/observations. Returns the value[] and status[] arrays plus the dimension objects; observations align to dimension.reference_date (the time axis, ISO dates). extension.series[] lists every series (id + Portuguese label) the dataset contains. Requires both the parent domain_id and the dataset id (from list_datasets). Large datasets can be sizeable. lang defaults to PT.
Look up metadata for one or more series by numeric series id. Returns label, short_label, description (Portuguese by default), dataset_id, domain_ids, and dimension_category. Use this to identify a series and find which dataset_id holds its observations, then call get_dataset. NOTE: this endpoint returns metadata only — it does NOT return the numeric values. lang defaults to PT.
List the datasets within a statistical domain (use a domain id from list_domains). Returns a JSON-stat collection; each item has extension.id (the dataset id, a hex string) and extension.num_series. The dataset id feeds get_dataset. lang defaults to PT.
List Banco de Portugal's statistical domains (the full subject tree). Each domain has id, parent_id (tree links; null = top level), label/short_label/description (Portuguese by default), num_series, num_datasets, and has_series. Start here, then use a domain id with list_datasets. lang defaults to PT; pass "EN" for English.
Output schemas not documented. LLMs cannot predict response structure (JSON-stat format, value[]/status[] arrays, dimension objects, extension.series[]). This forces agents to call tools blindly and parse responses reactively, increasing errors and token waste.
No pagination support. get_dataset and list_datasets may return large results (e.g. 'Large datasets can be sizeable'). Without limit/offset and total_count, agents cannot handle multi-page results, risking context window exhaustion.
No error recovery guidance. callTool() throws generic errors ('BPstat: 404 ...', 'Unknown tool: ...', 'Required argument missing'). LLMs receive no actionable next steps, should suggest alternatives (e.g. 'Domain not found. Call list_domains() to see available domains').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
Tool annotations missing. All four tools are read-only (no side effects), but readOnlyHint is not declared in schemas. This prevents MCP clients from optimizing caching, retry logic, or safety checks.
Parameter 'lang' is optional but behavior differs (defaults to PT). Description says 'defaults to PT' but schema does not declare a default value. LLMs may not realize PT is the fallback and pass 'lang' unnecessarily.