Loading ToolBench
Agentic-first review tool for LaTeX papers: live PDF preview + comment queue, designed for Claude Code as author
Scholia exposes 6 tools with clear verb-noun naming (scholia_paper, scholia_compile, scholia_comment, etc.). All tools have descriptions (avg ~180 chars, within baseline 194). Input schemas are present with type definitions for most parameters. However, several parameters lack descriptions (e.g., 'edits' in scholia_comment), and output schemas are not formally documented. Error handling is minimal, tools return JSON but lack recovery guidance. The server is STDIO-only, which caps protocol readiness at 50 regardless of definition quality.
Mutate a comment. Actions: - add: create new comment. Required: text + anchor. - reply: append a thread entry. Required: id + text. - resolve: mark resolved. Required: id + summary. - dismiss: mark dismissed. Required: id + reason. - delete: permanently remove. Required: id. Anchor formats (only for add): {"kind": "paper"} {"kind": "section", "title": "Methods", "label": "sec:methods"} {"kind": "source_range", "file": "intro.tex", "line_start": 42, "line_end": 58} {"kind": "pdf_region", "page": 3, "bbox": [x1, y1, x2, y2]} Optional ``suggestion={"old": "...", "new": "..."}`` (add only)
List comments in the queue, optionally filtered by status and tags. Returns an array of comment objects, each with id, anchor, thread, status, resolved_source, snippet, and suggestion (if present).
Recompile the paper. Returns structured errors and warnings, each with file/line/message and (when possible) source context lines. Use after editing source: read the structured errors instead of re-reading the full latexmk log.
Tell the daemon to scroll the viewer to a target location. Target formats: - "section:Methods" or "sec:methods" (section title or label) - "p5" (page number) - "intro.tex:42" (file and line) - "intro.tex:42-58" (file and line range) Requires a running daemon (HTTP server).
scholia_comment 'edits' parameter lacks description. Parameter descriptions are required for LLM understanding; undescribed params invite misuse.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract fields without knowing return structure.
Error handling returns raw JSON without recovery guidance. Errors like 'add requires text and anchor' lack actionable next steps for the LLM.
scholia_comment uses nullable types (string|null, array|null) for required parameters (id, text, anchor). This invites ambiguous calls; use required/optional distinction instead.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Render a page or region of the PDF as a PNG image. Useful for the agent to see what the paper looks like at a specific location (page, section, or source range). Returns a base64-encoded PNG image.
The "what does my paper look like right now" oracle. Returns: - main_file, watch_dir - sections (with title, label, file, line, line_end) - compile_cache (pdf path + whether it exists) - comments[] (full list of comments at the requested status) Section parsing is deliberately the only structure we hand back; for labels, citations, and \input refs, use ``Grep``.
scholia_goto description mentions 'Requires a running daemon' but does not explain what happens if daemon is absent. No error classification (retryable vs fatal).