Loading ToolBench
MCP Research Server for arXiv Paper Search and Analysis. Exposes tools, resources, and prompts via FastMCP so that any MCP-compatible client (Claude Desktop, Cursor, etc.) can search, analyse, and synthesise academic literature from arXiv.
This arXiv research server has 11 tools with moderate definition quality. Most tools have descriptions (8-12 sentences, 100-300 chars), which meets baseline. However, 7 of 11 tools lack complete input schema visibility in the provided code snippet, only tool names and brief descriptions are shown, with parameter details inferred from docstrings rather than explicit schema registration. The code shows tool registration via @mcp.tool() decorator, but the actual schema objects are not fully visible. This forces schema scoring down significantly. Naming is generally good (verb-noun, action-oriented: search_papers, analyze_papers, compare_papers). Parameters that ARE visible (topic, max_results, paper_ids, comparison_criteria) have reasonable descriptions. However, error handling is minimal, most tools return plain-text error strings with no guidance on recovery steps or categorization (retryable vs fatal). No tool declares permissions, rates limits, or output schema documentation. The semantic_search and ask_papers tools have RAG-specific descriptions that are detailed, but lack formal schema definitions in visible code.
Generate a comprehensive literature review for a topic.
RAG-style question answering against the saved paper corpus. Retrieves the most relevant chunks and assembles a grounded context block that an LLM can use to answer the question.
Compare multiple papers across a chosen criterion.
Export saved papers as BibTeX references.
Export saved papers as formatted citations.
Find papers most similar to a given paper using TF-IDF cosine similarity.
Analyse research gaps across 9 dimensions (scalability, privacy, etc.). Scans all paper abstracts and classifies each dimension as covered, under-explored, or a gap.
Input schemas not fully visible for 7 of 11 tools. Code shows @mcp.tool() decorator and docstring parameters, but actual JSON Schema definitions not exposed in provided source. This prevents validation that schemas include all required type information and descriptions.
Error handling lacks recovery guidance. Most tools return plain strings like 'No papers for topic found. Run search_papers first.' without categorizing errors (retryable, user-fixable, fatal) or providing structured error responses with error codes and actionable next steps.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | <=2025-11-25 | v2 |
Search for academic papers on arXiv.
Search across ALL saved papers using TF-IDF retrieval at chunk level. Unlike keyword search, this finds semantically relevant passages even if the exact words differ.
Generate 5-7 original research questions based on the saved papers.
Track research trends over time for a topic.
No output schema documentation. Tools return free-text Markdown or plain strings without declaring the structure of returned data. LLMs cannot plan downstream calls or extract structured fields without explicit schema.
Stateful dependencies not documented. Multiple tools have implicit ordering requirements (e.g., 'must have searched first') documented only in descriptions. No mechanism to guide LLMs through multi-step sequences or enforce prerequisites.
No result pagination or limit enforcement. search_papers accepts max_results up to 20, but most tools (semantic_search, ask_papers) lack explicit result limits. Large result sets could exhaust LLM context windows.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All tools are read-only (Risk: READ_ONLY noted in metadata), but this is not declared in tool definitions themselves. Missing annotations prevent LLM safety reasoning.
No permission declarations. Tools have no scope documentation (e.g., 'read:arxiv', 'read:cache'). Unclear what privileges the agent needs and what blast radius would be if compromised.