An MCP server for searching and managing academic papers from arXiv with Claude integration
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation, error handling, and output structure. search_papers has a WRITE risk but no confirmation pattern. extract_info returns unstructured strings instead of typed objects. Both tools lack actionable error messages and recovery guidance. Naming is clear (verb_noun), but descriptions are generic and under-optimized for LLM selection. No tool annotations (readOnlyHint, destructiveHint). Output schemas are not documented.
Search for information about a specific paper across all topic directories.
Search for papers on arXiv based on a topic and store their information.
extract_info returns unstructured JSON string instead of typed object. LLM cannot plan downstream operations or extract fields reliably. Should return {title, authors[], summary, pdf_url, published} as structured output.
search_papers marked WRITE risk but has no confirmation/dry-run pattern. Tool modifies filesystem (creates directories, writes JSON) without user confirmation. Agents may accidentally trigger searches with side effects.
Error handling is minimal. extract_info returns bare string 'There's no saved information related to paper {paper_id}' instead of actionable guidance. No recovery hints (e.g., 'Try search_papers first'). search_papers silently catches exceptions and prints to stdout instead of returning structured errors.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | 2026-07-28+ | v2 |
Tool descriptions are generic and under 100 chars. 'Search for papers on arXiv based on a topic and store their information' does not explain WHEN to use this vs. extract_info, what the side effects are, or what structure is returned. Baseline for A+ tools is 194 chars with clear WHAT/WHEN/RETURN.
No tool annotations. search_papers should have destructiveHint=true (modifies filesystem). extract_info should have readOnlyHint=true. Annotations help LLMs reason about safety and retry logic.