Loading ToolBench
Quality-focused MCP server for academic paper analysis with hybrid retrieval, Zotero integration, page-based chunking, section-aware retrieval, and verbatim extraction
The server defines 18 tools with mostly adequate naming (verb-based) and descriptions. All tools are READ_ONLY, which is appropriate for an academic paper search/retrieval system. However, there are consistent gaps: (1) Parameter descriptions are sparse or missing for several tools, particularly in 'search_papers' where complex filtering options lack guidance on matching behavior; (2) Output schemas are not documented anywhere in the provided code, this is a critical omission; (3) Error handling patterns are not visible; (4) The tools follow a clear, logical organization and naming convention (verb_noun), but descriptions vary in quality from good (search_papers with examples) to minimal (list_venues, list_domains). The 'search_papers' tool is exemplary with detailed descriptions and example invocations, but most discovery tools (list_*) lack depth. Pagination/limits are present but underscore the absence of documented return structures.
Get results and findings (verbatim quotes with page numbers)
Get limitations and future work (author-stated only)
Get methodology extraction (verbatim + summary)
Get text of a single page
Get text of a page range
Get paper metadata (title, authors, abstract, etc.)
Get verbatim key points extracted from a section
Output schemas not documented. No tool provides structured return type documentation. The LLM cannot infer what fields/objects are returned, forcing guesswork on downstream tool calls or data extraction.
Discovery tools (list_venues, list_domains, list_sections) lack descriptions explaining their purpose or when to call them. At 35-45 characters, these descriptions are below the 50-char minimum for clarity. Example: 'List all venues (conferences/journals) in the database with paper counts. Use this to discover venue names for filtering.' is good; but it's inconsistently applied.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 25 | - | v1 |
Get LLM summary of a specific section
Get full original text of a section
List all research domains in the database with paper counts. Domains are specific and research-actionable (e.g., 'microkernel formal verification using Isabelle').
List all papers imported into the MCP database (with extraction status)
Get document structure (sections with page ranges)
List all venues (conferences/journals) in the database with paper counts. Use this to discover venue names for filtering.
Find paper by citation key (e.g., from \cite{key})
Semantic search within paper content. Returns relevant chunks with page numbers.
Find papers by multiple criteria. All filters are optional and combinable. Examples: - search_papers(query="verification") - text search in title/abstract - search_papers(venue="NDSS") - papers from NDSS conference - search_papers(domain="side-channel") - papers in domains containing "side-channel" - search_papers(author="Heiser") - papers by author - search_papers(year=2024) - papers from 2024 - search_papers(venue="RTSS", year_from=2020) - RTSS papers from 2020+ - search_papers(keywords=["seL4", "microkernel"]) - papers with these keywords
List all Zotero collections with paper counts
List papers in a Zotero collection (from Zotero database)
Parameter descriptions missing or incomplete. 'get_section_summary', 'get_section_text', 'get_section_key_points', 'get_page', 'get_pages' all accept 'section_type' or 'page' but lack guidance on valid values or enumeration. LLMs cannot infer valid section types (introduction, methodology, results, etc.) without an enum or clear description listing options.
Parameter 'section_type' in multiple tools is free-form string with no enum. The description mentions valid types (introduction, methodology, results, etc.) but leaves it to the LLM to guess which string format is accepted. Should be an enum: ['introduction', 'methodology', 'results', 'limitations', ...].
No error handling or recovery guidance documented. If a paper_id is invalid or a section does not exist, the tool behavior is opaque. LLMs need guidance on whether to retry, ask the user, or try an alternative tool.
Inconsistent parameter naming: 'paper_id' is used throughout, but the tool register and descriptions refer to 'Paper citation key', unclear if paper_id is an internal ID or a citation key. Should be uniformly named 'citation_key' or 'paper_id' with clear type guidance in each parameter description.
No documented pagination/limits behavior. 'search_papers' has a default limit of 50, but what if the user needs results beyond that? Is there a cursor, offset, or next_token? How does the LLM request the next page? This forces the LLM to guess.