Loading ToolBench
Model Context Protocol server for Semantic Scholar — 200M+ academic papers, 14 tools spanning paper search, citation graph traversal, author profiles, and recommendations.
Semantic Scholar MCP server demonstrates solid definition quality with consistent naming conventions (all tools start with action verbs: semantic_scholar_search, semantic_scholar_get, semantic_scholar_match, etc.), detailed parameter descriptions, and proper input schemas. All 14 tools have explicit descriptions and typed parameters. However, output schemas are NOT documented in the source code, the server returns raw API responses without indicating expected fields, response structure, or how to chain results between tools. Parameter descriptions are present but generic (e.g., 'fields to include in response' doesn't explain valid field names). Tool composition is well-designed with clear separation of concerns (search vs. get vs. bulk vs. recommendations), but error handling guidance is minimal, no explicit recovery paths documented. The codebase shows professional production patterns (TTL caching, rate-limit retry logic, type hints via Pydantic), but the tool definitions themselves lack the LLM-optimization depth required for an A-grade assessment.
Fetch multiple author records in a single call
Fetch multiple paper records in a single call
Search papers with sorting and filtering for large result sets
Export paper citation in BibTeX format
Get full profile details for a specific author
Get full details for a specific paper by ID
Resolve one known paper title to its full record
Output schemas not documented. No indication of what fields are returned, data types, or structure. LLMs cannot plan downstream calls or extract required data (e.g., which fields contain paper IDs for chaining to other tools). All 14 tools lack output documentation.
Parameter 'fields' is vague across all tools. Description states 'Fields to include in response' but does not enumerate valid field names, whether they are comma-separated strings, arrays, or a fixed set. LLMs cannot determine valid values without trial-and-error.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 68 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 14 | - | v1 |
Get paper recommendations based on multiple positive and negative examples
Get authors of a specific paper
Get paper recommendations similar to a seed paper
Search for authors by name
Search for academic papers by keywords with sorting and filtering options
Search inside paper full text
Check connectivity and rate-tier status of the Semantic Scholar API
Parameter 'sort' lacks enumerated options. Tools (semantic_scholar_search_papers, semantic_scholar_bulk_search, semantic_scholar_search_authors) accept a 'sort' parameter but the description only says 'Sort order for results' without listing valid values (e.g., 'relevance', 'date', 'citations'). LLMs will guess.
No error handling guidance. Tool descriptions do not indicate what errors might occur (invalid paper ID, rate limit, API unavailable) or how the LLM should recover. No recovery paths documented (e.g., 'If paper not found, try semantic_scholar_search_papers').
No pagination guidance in descriptions. Tools returning lists (semantic_scholar_search_papers, semantic_scholar_search_authors, semantic_scholar_snippet_search) accept 'limit' and 'offset' but descriptions do not clarify: (a) default limit if omitted, (b) maximum allowed limit, (c) whether results include a total count for LLM planning.
semantic_scholar_status has minimal schema. Input is empty (no parameters), output schema not documented. Unclear what data fields are returned (API version? rate limit tier? remaining quota?).