Loading ToolBench
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation, error handling, and composition. The 'search' tool lacks pagination despite potentially returning many results. The 'download_article' tool has a hardcoded default path that is user-specific (/Users/vivek/...), creating portability issues. Neither tool provides recovery guidance on errors. Parameter descriptions exist but are minimal (10-30 chars). No tool annotations (readOnlyHint/destructiveHint) despite clear semantic differences. Output schemas are defined via Pydantic models but not documented in tool descriptions.
Download an arXiv article PDF and return its saved path.
Search for arXiv articles matching the query and return the results as dict.
Hardcoded user-specific default path in download_article (/Users/vivek/Downloads/arxiv-researcher) breaks portability and will fail on any other system. Default paths must be system-agnostic or configurable.
search tool lacks pagination support. With max_results=5 default but no limit enforcement or next_cursor, agents cannot retrieve large result sets. Should accept limit (1-100) and offset/cursor parameters.
No error recovery guidance. download_article raises ValueError('No arXiv article found...') but provides no actionable next step. Should suggest: 'Try search() to verify the article ID exists.'
Missing tool annotations. download_article is destructive (writes files) and search is read-only, but neither declares readOnlyHint or destructiveHint. Agents cannot reason about side effects.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
Parameter descriptions are minimal (10-30 chars). 'Maximum number of results to return' and 'Directory path where the PDF will be saved' lack constraints (e.g., count range 1-100, directory must exist or be creatable). LLMs cannot infer valid ranges.