Loading ToolBench
Two tools with basic definitions and schemas. Both tools have descriptions (50-60 chars, below the 194-char baseline) and input schemas with typed parameters. However, descriptions lack actionable context (WHEN to use, prerequisites, return details). No parameter descriptions beyond the schema type. No output schema documentation. No error handling guidance. Missing tool annotations (readOnlyHint). Naming is clear (search, fetch) but descriptions are minimal.
Fetch a cupcake order by ID. Returns a FetchResult model.
Search for cupcake orders – keyword match. Returns a SearchResultPage containing a list of SearchResult items.
Descriptions lack actionable context. Both tools have 50-60 char descriptions (below 194-char baseline). Missing WHAT, WHEN, and return details. E.g., 'search' should explain: 'Search cupcake orders by keyword. Returns up to 20 results. Use when you need to find orders by name, flavor, or customer. Returns SearchResult items with id, title, text.'
No parameter descriptions in schema. The 'query' and 'id' parameters have type but no description explaining format, constraints, or examples.
Output schemas not documented in tool descriptions. SearchResultPage and FetchResult are Pydantic models but LLMs cannot infer structure from code. Descriptions should state: 'Returns SearchResultPage with results array (id, title, text fields).'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
No error handling guidance. fetch() raises ValueError('unknown id') but description does not explain what happens on invalid ID or how to recover. Per pattern:recovery-guide, errors must tell LLM what to do next.
Missing tool annotations. Neither tool declares readOnlyHint=true, though both are read-only. Per current spec (2026-07-28), tool annotations enable better agent planning and safety.