Loading ToolBench
MCP server for searching and extracting information from arXiv papers
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation, output schema clarity, and error handling. search_papers lacks description of max_results behavior and return value structure. extract_info has minimal parameter guidance and no documented output format. Neither tool includes error recovery hints or validation constraints. Naming is acceptable (verb_noun pattern), but descriptions are generic and under 200 chars. No tool annotations, no pagination support despite potential for large result sets, no idempotency guarantees.
Search for information about a specific paper across all topic directories.
Search for papers on arXiv based on a topic and store their information.
Output schemas not documented. search_papers returns List[str] (paper IDs) but LLM cannot infer structure of returned data or what fields are available for downstream use. extract_info returns JSON string but format is undocumented.
Parameter descriptions incomplete. max_results has description but no bounds (min/max), no guidance on performance impact. paper_id has no format hints, no guidance on where to obtain valid IDs.
No error recovery guidance. extract_info returns plain text error message 'There's no saved information related to paper {paper_id}' but does not suggest next steps (e.g., 'Try search_papers() first to find valid paper IDs').
search_papers has side effects (writes to disk) but description does not state this. LLM cannot determine if tool is safe to retry or has irreversible consequences.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
No pagination support. search_papers accepts max_results but does not document offset/page parameters or return total_count. If arXiv returns 1000+ results, LLM context window could be exhausted.