Loading ToolBench
MCP stdio server for the Semantic Scholar Academic Graph API. Provides paper search, citation and reference traversal, author lookup, and recommendations.
Strong tool naming (all verb-noun, clear actions) and comprehensive parameter schemas with types and descriptions. All 8 tools have descriptions (194 - 250 chars, within baseline 34 - 392). Pagination support (limit/offset) present across search/list tools. However, output schemas are not explicitly documented in the source, responses are mediated through a unified envelope (mediation.py) but the actual field structure is not visible in server.py. Error handling is minimal (no recovery guidance, no actionable error messages visible). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite all tools being read-only. Missing per-parameter constraints (e.g., limit max=100 stated in descriptions but not in schema enums). No confirmation/dry-run patterns for any tool.
Retrieve detailed metadata for a specific author by Semantic Scholar author ID.
Retrieve papers authored by a specific author.
Retrieve detailed metadata for a specific paper by identifier (S2 paper ID, DOI, arXiv ID, PubMed ID, CorpusId, or semanticscholar.org URL).
Retrieve papers that cite a given paper (incoming citations).
Retrieve papers referenced by a given paper (outgoing references).
Get paper recommendations similar to a given paper based on citation and content similarity.
Search the Semantic Scholar Academic Graph for authors by name.
Output schemas not documented in source code. Responses are mediated through mediation.py and response-schema.json (not provided), but the actual field structure, types, and chaining IDs are not visible in server.py. LLMs cannot plan downstream calls without knowing what fields to expect.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite all 8 tools being read-only. Clients cannot distinguish safe tools from destructive ones without explicit hints.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | <=2025-11-25 | v2 |
Search the Semantic Scholar Academic Graph for papers by title, abstract, or keywords. Results are relevance-ranked.
Parameter constraints (limit max=100) stated in descriptions but not enforced in schema. Descriptions say 'max: 100' but no JSON Schema maxItems/maximum constraint visible. LLMs may pass invalid values; validation must happen server-side.
No error handling guidance visible. If API returns 404 (paper not found), 429 (rate limit), or 500, the error response does not guide the LLM on recovery (retry, search instead, etc.). Agents will stall on ambiguous failures.
API key injection via environment variable (SEMANTIC_SCHOLAR_API_KEY) is correct, but no documentation of required setup or error message if key is missing. Users may deploy without the key and see cryptic API errors instead of a clear 'API key not configured' message.