Loading ToolBench
GRaDOS has 34 tools with mostly complete schemas and descriptions, but quality is inconsistent. Naming follows verb_noun convention well (search_academic_papers, extract_paper_full_text, save_paper_to_zotero). Descriptions average ~120 chars and explain purpose clearly. However, many parameter descriptions are generic or missing context about dependencies and constraints. Output schemas are not documented in the visible code, only input schemas are defined. Error handling is minimal; tools lack recovery guidance. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. Several tools combine multiple concerns (e.g., extract_paper_full_text handles both extraction AND browser session management). Parameter validation rules are not stated in descriptions (e.g., 'limit' accepts 1-100 but description doesn't say so).
Audit an answer against an evidence pack to verify claims.
Audit a draft answer against saved papers to identify supporting evidence and gaps.
Audit the result of an external consultation.
Build a structured evidence grid comparing papers on key dimensions.
Compare multiple papers on methodology, findings, and conclusions.
Consult ChatGPT Pro with a research question and optional evidence pack.
Extract full text from an academic paper by DOI using publisher-specific routes and browser automation.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. Tools marked WRITE (save_paper_to_zotero, parse_pdf_file, import_local_pdf_library, run_external_consult, save_external_consult_result) and REVERSIBLE (plan_library_pdf_cleanup, manage_failure_cases) lack explicit hints to guide agent behavior.
Output schemas not documented in visible code. Only input schemas are defined; return types and field structures are not declared. LLMs cannot plan downstream tool calls or extract required fields (e.g., what does search_academic_papers return? Does it include DOI, title, abstract?).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
Retrieve the citation graph for a paper showing citing and cited papers.
Inspect or recover a durable long-running GRaDOS operation.
Get full context for multiple papers including sections and cross-references.
Retrieve the structural outline (sections, headings, paragraphs) of a saved paper without loading full text.
Batch import all PDF files from a local directory into the paper library.
Ingest a PDF downloaded via the Codex Chrome extension.
Manage and recover from failed operations.
Resource providing an overview of a specific saved paper.
Resource providing an index of all saved papers in the library.
Parse a local PDF file and save it to the paper library with optional metadata.
Plan and execute cleanup of duplicate or orphaned PDFs in the library.
Prepare an evidence pack from selected papers for structured analysis.
Prepare an external consultation packet from a research topic.
Prepare a consultation packet for external review.
Preview a consultation packet before sending.
Query local research artifacts.
Read the contents of a saved evidence pack.
Read a specific asset (image, table, figure) from a saved paper.
Read the full text of a saved paper by DOI, with optional section filtering.
Run an external consultation with prepared packet.
Save the result of an external consultation.
Save one paper to Zotero via the Web API using the configured library settings. Best used for papers that actually support the final answer.
Persist a reusable research artifact in the local state database.
Search academic papers across multiple sources (Semantic Scholar, CrossRef, etc.) with optional in-depth extraction.
Search papers already saved in the local library using semantic and lexical retrieval.
Suggest missing evidence or papers needed to support claims in an answer.
Verify the integrity and completeness of an evidence pack.
Parameter constraints not stated in descriptions. 'limit' parameter accepts 1-100 (visible in schema) but description does not mention bounds. LLMs cannot infer valid ranges from schema alone and may pass invalid values.
Error handling and recovery guidance missing. Tools lack descriptions of failure modes, retryability, or next steps. E.g., extract_paper_full_text may fail on paywalled papers, no guidance on alternatives.
Tool composition issues: extract_paper_full_text combines extraction, browser session management, and caching. Should split into separate tools (extract_paper, resume_browser_session, clear_cache) so agents can compose as needed.