Loading ToolBench
Two tools with complete input schemas and descriptions, but descriptions are minimal (under 100 chars) and lack actionable context for LLM selection. search_literature has good parameter constraints (limit 1-100, offset ≥0) and clear pagination support. read_paper_text similarly well-constrained. However, both descriptions omit WHEN to use each tool, prerequisites, or what downstream actions they enable. No output schemas documented. Error handling returns structured errors but lacks recovery guidance. Tool composition is sound (search then read), but descriptions don't guide this workflow.
Fetch and convert full arXiv paper HTML to plain text for evidence-backed implementation
Search papers with searchthearxiv relevance ranking
Tool descriptions lack actionable context. 'Search papers with searchthearxiv relevance ranking' does not explain WHEN to call this vs other discovery tools, what the LLM should do with results, or how it chains to read_paper_text. Descriptions should be 50-200 chars and answer: what, when, and what's next.
Output schemas not documented. LLMs cannot plan downstream calls or extract fields without knowing the response structure. search_literature returns papers array with paperId, score, title, authors, abstract, year, month, authorsParsed, and authors array, none of this is visible in tool definition.
Error handling returns structured errors but lacks recovery guidance. toolError() returns status, message, and details, but does not tell the LLM what to do next (retry? ask user? try different query?). Errors should categorize as retryable, user-fixable, or fatal.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2025-06-18+ | v2 |
Parameter descriptions are minimal. 'Search query' and 'Maximum number of papers to return' lack format guidance. Should specify: query syntax (free text? boolean operators?), typical length, examples of good vs bad queries. 'Maximum characters to return' (maxChars) lacks context on why pagination is needed or how to use offset.
read_paper_text accepts 'identifier' as arXiv ID or URL but description does not clarify accepted formats. Code shows regex extraction (\d{4}\.\d{4,5}(?:v\d+)?) but LLM cannot see this. Should document: 'arXiv ID (e.g. 2401.12345 or 2401.12345v2) or full URL (https://arxiv.org/abs/2401.12345)'.