Loading ToolBench
Remote MCP server that delivers research papers to your e-reader, with Zotero as the source of truth
Paperboy demonstrates above-average definition quality with well-written descriptions, comprehensive parameter documentation, and thoughtful output structuring. All 10 tools have clear, LLM-optimized descriptions (150-400 chars) that explain WHAT, WHEN, and WHY. Parameter schemas are present and typed. However, output schemas are not formally documented in code, tool annotations (readOnlyHint/destructiveHint) are missing, and error handling guidance is incomplete. The tool composition is excellent, each tool has a single clear responsibility, names are action-oriented, and parameter naming is consistent. Security is well-handled with server-side credential injection. This is a B+ implementation: strong fundamentals with room for formalization.
Catalogue a book by ISBN, book DOI, or title as a proper Zotero book item.
File a paper into the Zotero library without sending it.
Ingest a PDF the user already has (grey literature, open-access textbooks) with the PDF attached and metadata supplied, optionally sending it.
List all Zotero collections for organizing papers.
List all papers currently queued for the e-reader.
Stage papers for delivery to the e-reader without sending yet.
Output schemas not formally documented in code. While _summary() shows the return structure for search/recommend, tools like queue_papers, send_papers, and send_queue have implicit outputs (success/failure status, queue contents) that are not declared in tool definitions.
Tool annotations (readOnlyHint, destructiveHint, idempotentHint) absent from tool registration. Risk field is present in this evaluation but not in the actual MCP protocol payload. Tools like send_papers and send_queue are destructive but lack explicit destructiveHint annotation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 69 | 2026-07-28+ | v2 |
Discover papers the user may want to read — old or new. Blends two signals: citation-graph recommendations (Semantic Scholar) seeded from ``seed_refs``, or — by default — the user's Zotero library (what they queue and read IS their interest profile); and keyword discovery (OpenAlex) from ``interests`` — pass 2-4 short phrases distilled from the current conversation. recent_only=True favors newly published work; False searches the all-time pool (computer science only, an upstream limit). Papers already in the user's library are excluded (the queue plus the 100 most recently added items). max_results is capped at 20. Returns {"picks": [...], "problems": [...]}: picks carry refs for send_papers / queue_papers, and each pick has a ``via`` field saying why it appeared — 'interest-keyword' (matched a stated interest), 'related-to-seeds' (citation graph of explicit seeds), or 'related-to-library' (citation graph of the OWNER'S Zotero library; can look off-topic to anyone else). When interests are given they lead the results. problems reports any discovery arm that failed or seed that didn't resolve — ALWAYS relay problems, or a stated interest may silently go uncovered. Present picks and let the user choose; don't send unasked.
Search for papers across the scholarly literature. ``source`` is 'all' (OpenAlex: journals, conferences, and preprint servers including arXiv — broad coverage, but ranking can miss on arXiv-native topics) or 'arxiv' (arXiv's own search — better for recent preprints or when 'all' returns off-topic results); unknown values fall back to 'all'. ``max_results`` is clamped to the 1-25 range. Each result has a ``ref`` (arXiv id, DOI, or exact title) to pass to send_papers / queue_papers. ``open_access_pdf`` means an OA PDF link was found; delivery can still fail if the link is dead (arXiv-hosted papers are the most reliable).
Send selected papers from the queue to the e-reader.
Flush every unsent queued item to the e-reader.
Error recovery guidance incomplete. While search_papers provides rate-limit handling and context-specific recovery hints, most other tools (queue_papers, add_to_library, attach_pdf) do not document failure modes or recovery paths. Example: attach_pdf does not explain what happens if the PDF URL is invalid or if metadata conflicts with Zotero.
Short descriptions for write operations. Tools like send_queue ('Flush every unsent queued item to the e-reader'), list_queue ('List all papers currently queued'), and list_collections ('List all Zotero collections') are 50-60 chars, below the actionable baseline of 80-100 chars. These lack prerequisite context or side effects.
No pagination or result limiting declared for list_queue and list_collections. If a user has hundreds of papers queued or many nested collections, these tools could return unbounded lists that exhaust context. No limit parameter, total_count, or next_cursor field documented.
attach_pdf requires LLM to provide authors array and metadata directly instead of resolving from a URL. If LLM has only a URL, it cannot auto-extract metadata; the tool does not offer a fallback to infer title/authors from the PDF itself or a document metadata service.