MCP server for searching and retrieving papers from arXiv with advanced query capabilities, paper details, trending analysis, and PDF content extraction
Arxiv Searcher has 4 tools with mixed definition quality. All tools have descriptions (50-150 chars each), which is adequate but brief. All tools have input schemas with type definitions and parameter descriptions. However, schemas lack output schema documentation, the rubric baseline shows 100% of A+ tools have documented return types. Parameter naming follows verb_noun conventions well (search_papers, get_paper_details, download_paper, analyze_trends). Tool names are action-oriented and clear. The main deficits are: (1) no documented output schemas visible in the code, (2) missing enum constraints on 'category' parameter despite being a known set of arXiv categories (defaults to 'cs.SE' but doesn't enumerate valid options), (3) no error handling guidance in tool descriptions (e.g., what happens if arxiv_id is invalid, or if PDF download fails), (4) no per-field output documentation. The 'category' parameter accepts a string with a default but no validation or enum list, inviting hallucinated category names. Average per-tool score: 58/100.
Analyze trends in papers based on a search query, showing most common keywords, authors, and categories.
Download a paper PDF from arXiv and save it locally.
Get detailed information about a specific paper by ArXiv ID.
Search for papers on arXiv. It can parse natural language queries, extracting keywords and years for filtering.
No output schemas documented. All 4 tools lack explicit return type definitions. The rubric baseline shows 100% of A+ tools document return types. LLMs cannot plan downstream tool calls or extract results without knowing what fields to expect.
Category parameter accepts free-form string with no enum constraint. The server code mentions arxiv_categories.json exists but tool schema does not reference it. All 4 tools allow 'category' input (default 'cs.SE'). LLMs will hallucinate invalid categories like 'ai.ml' or 'computer-science' instead of valid arXiv category codes (cs.AI, cs.CL, cs.SE, etc.).
No error handling guidance in tool descriptions. What happens if arxiv_id is malformed? If PDF download fails due to network error? If no papers match the query? Tool descriptions do not guide LLMs on recovery steps (e.g., 'If search returns 0 results, try broadening query or removing year filters').
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 13 | - | v1 |
Tool descriptions are under 150 characters, below the rubric baseline of 194 chars for A/A+ tools. Descriptions lack context on WHEN to use each tool. 'Search for papers on arXiv' alone doesn't explain when to prefer search_papers vs analyze_trends, or how they differ in intent.
download_paper tool has no guidance on where PDFs are saved or error cases. Description says 'Download a paper PDF from arXiv and save it locally' but omits: (1) what happens if the PDF is too large, (2) what happens if the directory doesn't exist, (3) what is returned (filepath? success message?), (4) whether downloads are retryable.
No pagination guidance in search_papers. With max_results default=10, what if the LLM wants 500 papers? The description should note that large requests may timeout and offer guidance on batch operations (e.g., 'For large result sets, call with smaller max_results and iterate over pages').