Loading ToolBench
A Model Context Protocol (MCP) server for accessing Semantic Scholar's academic database. Implements the MCP Streamable HTTP transport protocol.
This server provides 12 well-named read-only tools for academic paper search and discovery via Semantic Scholar. All tools follow verb_noun naming convention and have basic descriptions. However, there are significant gaps: (1) Output schemas are completely undocumented, the rubric requires documenting what fields LLMs should expect; (2) parameter descriptions lack important constraints (e.g., num_results max of 100 is stated only in description, not as a maximum constraint; paper_id format is unspecified); (3) no error handling guidance is provided; (4) responses could include unnecessary metadata that wastes token budget. The tools are logically well-composed (each has a single clear purpose), but definition quality for production agent use is below benchmark.
Get details of a specific author on Semantic Scholar.
Get details for multiple authors at once using batch API.
Get citations and references for a specific paper on Semantic Scholar.
Get paper title autocompletion suggestions for a partial query.
Get details of a specific paper on Semantic Scholar.
Find the best matching paper on Semantic Scholar using title-based search.
Get recommended papers for a single positive example paper.
Output schemas are completely undocumented. The rubric requires documenting return types so LLMs know what fields to expect and can chain calls correctly. All 12 tools return either List[Dict[str, Any]] or Dict[str, Any] with zero field-level documentation.
Parameter constraints are incomplete. num_results states 'max 100' only in description text, not as a JSON Schema maximum. paper_id format (Semantic Scholar ID vs DOI) lacks examples or regex patterns. limit and query parameters have minimal constraint documentation.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | <=2025-11-25 | v2 |
| 2026-03-09 | C | 64 | - | v1 |
Get recommended papers based on lists of positive and negative example papers.
Get details for multiple papers at once using batch API.
Search for authors on Semantic Scholar using a query string.
Search for papers on Semantic Scholar using a query string.
Search for text snippets from papers that match the query.
No error handling or recovery guidance. Tools wrap all exceptions with generic 'An error occurred while...' messages. LLMs receive no hint about whether to retry, ask the user, or try an alternative tool. Example: paper_id not found should suggest search_semantic_scholar_papers as a fallback.
Tool descriptions lack dependency hints. Tools like get_semantic_scholar_paper_details do not indicate that users usually need to call search_semantic_scholar_papers first to obtain a paper_id. This forces agents to make discovery lookups instead of guided chains.
Response design does not address token efficiency. No indication that results are paginated (search tools should state if they return partial result sets) or that response fields are trimmed. Semantic Scholar API typically returns verbose metadata, author networks, and citation counts that waste tokens for basic chat use cases.