Loading ToolBench
Search arXiv, fetch paper metadata, and read full-text content via MCP. STDIO or Streamable HTTP.
The arxiv-mcp-server demonstrates strong definition quality with well-structured tool definitions, comprehensive parameter documentation, and clear descriptions. All 4 tools have explicit schema definitions with typed parameters and meaningful descriptions. Tool naming follows verb_noun conventions (search, get, read, list). Schemas are complete with proper JSON Schema format. Output documentation is present and adequate. However, there are minor gaps: parameter descriptions could be more tightly optimized for LLM selection patterns, and error handling guidance is not explicitly present in tool descriptions. The server accepts human-friendly identifiers (paper IDs in multiple formats, category names) which is excellent. Response schemas are well-documented inline via parameter descriptions. No critical definition issues detected.
Get full metadata for one or more arXiv papers by ID. Use when you have known IDs from citations, prior search results, or memory.
List arXiv category codes and names. Useful for discovering valid category filters for arxiv_search. Lists subject classes only; arxiv_search also accepts a bare archive code (the part before the dot, e.g. "astro-ph" or "cs") to search a whole archive at once.
Fetch the full text of an arXiv paper. Tries arxiv.org/html first, falls back to ar5iv.labs.arxiv.org, and falls back again to text extracted from the PDF when neither has an HTML render — check the source field to know which one answered. Page through long papers with start and max_characters, or pass max_characters null to get the entire body in one call.
Search arXiv papers by query with category and sort filters. Returns paper metadata including title, authors, abstract, categories, and links.
Parameter descriptions for arxiv_search could be more concise and LLM-optimized. The 'au:' and query syntax explanations, while comprehensive, exceed recommended 50-200 char range for parameter descriptions (currently ~500+ chars). Readability and LLM parsing would benefit from condensation.
Error handling and recovery guidance not explicitly documented in tool descriptions. Descriptions do not state what to do when a search returns no results, paper format is unavailable, or pagination limits are exceeded.
The 'start' parameter in arxiv_read_paper uses character offsets which may be non-intuitive for LLMs. No guidance on what happens when start exceeds body_characters, or how to detect end-of-content.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | A | 80 | 2026-07-28+ | v2 |
arxiv_get_metadata accepts union type (single ID or array of up to 10 IDs) but the description could be clearer about the upper limit enforcement and behavior when that limit is exceeded.