Loading ToolBench
Google Scholar FastMCP server and Streamlit frontend for scraping academic paper data, author profiles, and citation information
Scholar MCP has significant definition quality gaps. All 4 tools have descriptions (positive), but parameter documentation is sparse and inconsistent. getScholarData, getArxivData, and getScholarProfiles each accept only a 'url' parameter with minimal guidance on format or constraints. getCitedByData has better parameter coverage (5 params with types), but lacks descriptions for most parameters. No tools document output schemas, making it impossible for LLMs to plan downstream operations. Error handling is minimal (try/except with print statements, no recovery guidance). Tool names follow verb_noun convention (positive), but descriptions lack WHEN/WHY context. Overall, the server reads like a web scraper wrapper rather than a production-grade agent tool.
Searches for papers on arXiv using Google Scholar's search page.
Returns the papers that cite a target work, using Google Scholar's ?cites=<cluster_id> endpoint.
Scrapes the organic results from a Google Scholar search for the given query.
Scrapes scholar (author) profiles from a Google Scholar search page.
No output schemas documented. Tools return lists of dicts with fields like 'title', 'title_link', 'snippet', but LLMs cannot plan downstream operations without knowing the structure. This violates pattern:tool and pattern:response-shaper.
Parameter descriptions are minimal or missing. getScholarData, getArxivData, and getScholarProfiles each have only a 'url' param with a brief description but no guidance on format, constraints, or examples of valid URLs. getCitedByData has 5 params but only 'cites_id' has a meaningful description; 'hl', 'as_sdt', 'start', 'max_pages' lack descriptions entirely.
Error handling is non-existent for agent guidance. Tools catch exceptions and print to stdout, but return None or incomplete results on failure. No error messages tell the LLM what went wrong, whether to retry, or what to do next. Violates pattern:recovery-guide.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
No pagination support documented. getCitedByData accepts 'start' and 'max_pages' but tools do not return total_count, next_cursor, or indicate whether more results exist. LLMs cannot reliably paginate through large result sets.
Tool descriptions lack WHEN/WHY context. 'Scrapes the organic results from a Google Scholar search' tells WHAT but not WHEN to use this vs getArxivData or getScholarProfiles, or what the user should expect. Descriptions should be 50-200 chars and answer: what, when, and what it returns.