Loading ToolBench
Search a billion+ documents — papers, books, code, legal cases, forums, Wikipedia, and more.
Two tools with strong naming and comprehensive parameter schemas. search() has excellent descriptions (250+ chars) with clear usage guidance and strategy hints. Parameters are well-typed with enums and constraints. detail() is minimal but functional. Output schemas are not explicitly documented in code, requiring inference from API responses. Error handling is present but generic. No tool annotations (readOnlyHint/destructiveHint) despite both being read-only operations.
Get full metadata for a document. Use IDs from search results.
Search documents. Returns ranked results with title, author, year, description, url, id, score — not full content. Use URLs from results with fetch/browsing to read actual documents. Use when: user asks to research, find papers/books/articles, look up facts, find discussions, legal cases, or any "search for..." request. Strategy: use 2-4 keywords per query (English preferred). Pick the right type first. Try synonyms if few results. Search across multiple types to cross-reference. Use detail() for full metadata on promising results.
detail() tool has minimal description (11 words). LLMs cannot determine when to call it vs search(). Should explain: 'Retrieve full metadata and content for a document identified by ID from search results. Use after search() to get complete details, citations, or full text.'
Output schemas not documented in code. search() returns 'ranked results with title, author, year, description, url, id, score' but no formal schema visible. detail() output structure completely undocumented. LLMs cannot plan downstream operations without knowing response fields.
No tool annotations despite both tools being read-only. search() and detail() should declare readOnlyHint=true to signal safety to agents and prevent unnecessary confirmation prompts.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2025-06-18+ | v2 |
Error handling is generic. Code throws 'Invalid API key', 'Rate limit hit', 'API error {status}' but does not guide recovery. Should return: 'Rate limit hit (429). Get a Pro key at https://jail.li for higher limits, or retry in 60 seconds.'
search() limit parameter defaults to 10 but description says 'Trial max 10, Pro max 50'. No way for LLM to know if it has Pro access or should request higher limits. Should clarify: 'Default 10 (trial). With API key, up to 50.'