Loading ToolBench
Three tools with basic schemas and descriptions, but significant gaps in parameter documentation, output schema specification, and error handling guidance. Tool names lack action verbs (subjects, table_meta, query_table are noun-heavy). The 'body' parameter in query_table has only a description string with no type validation or nested schema. No output schemas documented. Error handling returns raw HTTP status + truncated text with no recovery guidance. Missing parameter descriptions for critical fields (e.g., what constitutes a valid 'path' format beyond examples). No pagination, rate limiting, or result limits documented despite potential for large result sets.
Pull data from a ".px" table. body is a PxWeb query object. path e.g. "khi/statfin_khi_pxt_11xs.px".
Navigate the subject tree. Entries with type "l" are folders (drill in with their id); type "t" are tables (id ends in ".px", use with table_meta / query_table).
Table definition (dimensions, valid values). path must point at a ".px" table, e.g. "khi/statfin_khi_pxt_11xs.px".
Tool names lack action verbs. 'subjects', 'table_meta', 'query_table' are noun-heavy and do not clearly signal intent. Should be 'list_subjects', 'get_table_metadata', 'fetch_table_data' to match LLM expectations.
Output schemas are not documented. LLMs cannot plan downstream calls or extract fields without knowing what each tool returns. Add explicit 'returns' documentation with field names, types, and descriptions.
query_table 'body' parameter has type 'object' with only a description string '{query: [...], response: {...}}', no nested schema. LLMs cannot validate structure. Define a proper JSON Schema with required fields, nested properties, and type constraints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Error handling returns raw HTTP status codes and truncated response text with no recovery guidance. 'StatFin: 404 ...' tells the LLM nothing about what to do next. Errors should suggest alternatives (e.g., 'Path not found. Call subjects() to discover valid paths.').
No pagination, result limits, or result count documented. StatFin API may return large datasets; without pagination guidance, LLMs risk context window exhaustion. Add limit/offset parameters and document max result size.