An MCP server for searching and extracting information from academic papers using arXiv API
Three tools are registered via @mcp.tool() decorators with names, input schemas, and descriptions. Schemas are properly typed with descriptions. However, output documentation is missing entirely, the rubric requires documented output schemas (DIMENSION 1.D critical check). Descriptions are adequate but brief (avg ~100 chars). Parameter naming follows verb_noun convention (search_*, extract_*), though parameter descriptions lack range/format guidance. No error handling guidance, no idempotent/destructive hints, no confirmation patterns for state-modifying operations. Resources and prompts are present but not core tools. Overall: decent foundation but significant gaps in output documentation and error recovery patterns.
Extract and return saved information about a specific paper by its ID
Generate a prompt for Claude to find and discuss academic papers on a specific topic
Search for academic papers on arXiv by topic with configurable result limit
Output schemas are not documented. Each tool description does not specify what the tool returns or the structure of the response. LLMs cannot plan downstream operations without knowing output shape.
search_papers modifies state (writes to filesystem) but description does not say so. Per pattern:command-tool, LLMs must know which calls have irreversible consequences.
No error handling or recovery guidance. If arxiv.Client() fails, if the paper_id is not found, or if filesystem write fails, tools return bare messages with no hint about retry strategy or next steps.
Parameter descriptions lack format and range guidance. E.g., 'max_results' and 'num_papers' have no stated range (min/max bounds). LLMs may pass absurd values (0, 1000000).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 58 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 58 | - | v1 |
No tool annotations. search_papers is destructive/mutating (writes to disk) and should have destructiveHint=true or idempotentHint=false. This guards against careless agent retry loops.
extract_info uses bare string return type. No field names, no structure. LLMs must parse JSON from a free-text string, which is error-prone and wastes tokens.