Loading ToolBench
Claude Code MCP server that fetches, analyzes, and ranks daily arxiv preprints personalized to your research profile
arxiv-scout has 5 tools with explicit schemas and descriptions, but significant gaps limit production readiness. Tool naming is clear (fetch_papers, get_user_profile, analyze_papers, rank_and_display, save_config), but descriptions lack actionable detail for LLM selection. Most parameters lack type definitions in schemas (e.g., analysis_json in rank_and_display has no type). Error handling is absent, no recovery guidance, validation errors, or categorization. The analyze_papers tool is conceptually problematic: it returns a prompt for Claude to execute rather than structured output, breaking the tool composition pattern. Output schemas are undocumented. No parameter descriptions exist for most inputs. Security: save_config writes to user home directory without validation, risking path traversal.
Prepare the analysis prompt. This tool does NOT call an LLM — it returns a structured prompt that YOU (Claude) should then execute to produce the analysis JSON. Takes no arguments; reads the profile internally.
Fetch today's arxiv preprints in the user's configured categories. Results are keyword-filtered and cached in SQLite so repeat calls within the same day are instant. Returns a JSON list of papers: {paper_id, title, authors, abstract, url, submitted}.
Read the user's research profile from config/user_profile.yaml and optionally fetch their Google Scholar page to extract past paper titles. Returns a JSON object with: categories, keywords, ranking_criteria, past_papers (list of titles), scholar_fetched (bool), top_n, max_papers_to_analyze.
Sort Claude's analysis by novelty score and render the top-N papers as a readable markdown report.
Save the user's arxiv-scout configuration to ~/.claude-plugin-config/arxiv-scout/. Accepts profile and/or settings fields and writes them as YAML. Call this from the /scout-setup skill after collecting the user's preferences.
rank_and_display analysis_json parameter has no type definition. Schema shows only description field, missing type, format, or structure. LLM cannot validate input shape.
analyze_papers returns a prompt string for Claude to execute, not structured output. This breaks tool composition, the LLM must manually parse and execute the prompt, defeating the purpose of tool abstraction. Should return structured analysis directly or accept papers as input.
No error handling or recovery guidance. Tools lack descriptions of failure modes, retryability, or next steps. E.g., fetch_papers may fail on network errors or invalid dates, but no guidance is provided.
save_config writes to ~/.claude-plugin-config/arxiv-scout/ without input validation. Accepts arbitrary field names and values. No sanitization against path traversal or injection attacks.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
Output schemas are not documented. Tools return JSON but LLM cannot know field names, types, or structure in advance. E.g., fetch_papers returns {paper_id, title, authors, abstract, url, submitted} but this is only in description text, not in a formal schema.