Loading ToolBench
Server has 4 well-named, read-only tools with complete input schemas and descriptions. Naming follows verb_noun pattern (search_stats, get_metadata, get_data, list_data_catalog). All tools have descriptions (avg 95 chars, within 10-1024 range). Input schemas are properly typed with required fields declared. However, output schemas are completely undocumented, LLMs cannot predict response structure for downstream chaining. Parameter descriptions lack detail on constraints (e.g., 'limit: 1-100000' stated but no guidance on typical values or performance impact). Error handling is basic (rate-limit detection, status code checking) but lacks recovery guidance. No tool annotations (readOnlyHint present implicitly via risk field, but not in schema). API key injection via _apiKey parameter violates secret-injection pattern, should use server-side credential management.
Fetch observations from a stats table. Optionally filter by dimension codes.
Fetch dimensions and code lists for a stats table.
Browse the high-level data catalog (table groupings).
Search e-Stat statistical tables. Returns IDs and names for tables matching the query. Use the IDs with get_data / get_metadata.
Output schemas completely undocumented. LLMs cannot predict response structure, field names, or types. Breaks tool chaining, agent cannot extract stats_data_id from search_stats to pass to get_data without trial-and-error.
API key exposed as tool parameter (_apiKey). Violates secret-injection pattern, credentials in parameters are logged in traces and prompt history. Should use server-side environment variable or vault injection.
Parameter descriptions lack actionable constraints. 'limit: 1-100000 (default 20)' stated but no guidance on typical values, performance impact, or when to paginate. 'filters' parameter example uses opaque codes (cdCat01, cdTime) without explaining how to discover valid codes.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Error handling lacks recovery guidance. Rate-limit (429) and API status errors are caught but responses do not suggest next steps (e.g., 'Retry after 60 seconds' or 'Call search_stats() to find valid table IDs'). LLM cannot self-correct.
No tool annotations. Tools are read-only but schema lacks readOnlyHint. Helps LLM understand safety and retry semantics.