Loading ToolBench
MCP server for finding relevant academic papers through multiple research APIs with semantic search and cross-encoder reranking
Three tools with complete input schemas and descriptions, but quality varies significantly. find_relevant_papers has a strong, detailed description (280+ chars) with clear WHEN/WHAT guidance. fetch_full_paper and fetch_paper_artifacts have good descriptions but lack parameter-level detail on expected formats and constraints. No error handling guidance, no output schema documentation, and no tool annotations (readOnlyHint). Parameter descriptions are present but generic, e.g., 'paper_id_or_url' lacks format examples or validation rules. All tools are READ_ONLY but this is not declared via annotations. Missing pagination guidance for find_relevant_papers despite returning lists.
Downloads and extracts the complete full-text content of a specific paper on the fly. Use this tool AFTER running `search_paper_abstracts` when you need granular technical details not available in the abstract—such as specific equations, detailed methodology, hyperparameter grids, or code implementations. Args: paper_id_or_url: The canonical paper identifier (e.g., arXiv ID '2301.12345', DOI) or the direct HTTP URL to the paper's PDF or HTML page. Returns: JSON string containing the full extracted plain-text body of the paper, section headers (if parsed), and fetch metadata.
Extracts software and code assets connected to a publication, including repositories and models. Use this tool when asked if official code exists, or when looking for pre-trained model weights, datasets, Google Colab notebooks, or project pages. Args: paper_id: The canonical paper identifier (e.g., arXiv ID '2301.12345', DOI '10.1038/s41586-020...', or Semantic Scholar ID). Returns: JSON object with lists for 'github_repos', 'huggingface_models', 'datasets', and 'project_urls'.
Finds the most relevant papers through multiple sources that could answer a natural language query. Args: query: The natural language query to search for. top_k: The maximum number of top matching paper abstracts to retrieve (default is 5). Returns: JSON string containing an object with two lists of papers, each paper carrying its paper ID, DOI, title, publication year, abstract, direct paper URL and license. `papers` holds up to top_k matches that come with an abstract — the ones worth reading now. `relevant_without_abstract` holds matches that ranked among the best but whose abstract the source does not publish, so they carry a title and little else. That often means the work is not openly available, though not always; some publishers restrict the abstract alone. They are still worth following by DOI or URL, and their `abstract` field is empty.
No output schema documentation. Tool descriptions state what is returned (JSON with papers, full-text, artifacts) but do not specify field types, required vs optional fields, or structure. LLMs cannot plan downstream calls or extract data reliably.
No error handling guidance. Tools do not document what happens on invalid input (e.g., malformed arXiv ID, unreachable URL, API rate limit). LLMs have no recovery path.
Parameter descriptions lack format constraints. 'paper_id_or_url' accepts arXiv ID, DOI, or URL but does not specify format patterns (e.g., arXiv: \d{4}\.\d{5}, DOI: 10\.\d+/.*). LLMs may pass invalid formats.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | 2026-07-28+ | v2 |
No tool annotations. All three tools are READ_ONLY but lack readOnlyHint annotation. Agents cannot infer safety/idempotency from tool definitions alone.
find_relevant_papers returns lists but lacks pagination guidance. No mention of limit behavior, whether results are capped, or how to retrieve additional papers beyond top_k.