Loading ToolBench
Three tools with complete schemas and descriptions, but descriptions are minimal (10-50 chars) and lack LLM-optimized guidance. Tool names follow verb_noun pattern (list_, query_, search_) which is good. Parameters have types and descriptions, but descriptions are terse and lack format/constraint details. No error handling guidance, no output schema documentation, no parameter validation rules stated. Schemas are present but sparse. Composition is reasonable, three distinct concerns, but parameter relationships (e.g., article_no vs from_no/to_no mutual exclusivity) are undocumented.
列出所有可查詢的國際公約。
查詢國際公約條文。可查單條、範圍、或全文。
在國際公約中搜尋關鍵字。
Descriptions are under 50 characters and lack LLM-optimized guidance. 'list_treaties' (16 chars), 'query_treaty' (30 chars), 'search_treaty' (20 chars) do not explain WHEN to use each tool or what distinguishes them. LLMs cannot reliably select the right tool without richer context.
Parameter relationships are undocumented. query_treaty accepts article_no, from_no, and to_no, but the mutual exclusivity (single article vs range) is not stated in parameter descriptions. LLMs will pass conflicting combinations without guidance.
Output schemas are not documented. Tools return dicts with 'success', 'error', 'available', 'treaties', etc., but the response structure is not formally specified. LLMs cannot plan downstream calls or extract fields reliably without schema documentation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
No error handling guidance. When a treaty is not found, the tool returns {'success': False, 'error': '...', 'available': [...]}. The description does not explain this error format or guide the LLM on recovery steps (e.g., 'Try list_treaties() to see available options').
Parameter descriptions lack format/constraint details. 'article_no' is described as '條號（如「38」「287」）' but does not state: accepted formats (Arabic, Chinese numerals), range (1 - 1914), or behavior if out of range. LLMs will guess and pass invalid values.