Loading ToolBench
Long-term research assistant: index papers, articles, and PDFs
22 tools with consistent verb-noun naming (init, ingest, search, list_papers, get_paper, update_*, rate_paper, create_topic, link_paper_to_topic, analyze_gaps, generate_report, fetch_references, search_body_evidence, enrich_keywords, rebuild_index, get_research_state). All have descriptions (10 - 200 chars, baseline 194). Input schemas present with type declarations and parameter descriptions. However, output schemas are NOT documented, responses are opaque serde_json::Value with no field-level guidance. Error handling maps to categories (invalid_params, dependency_missing, upstream_error, internal_error) but lacks recovery guidance. No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Composition is sound: tools chain naturally (ingest → search → link_paper_to_topic → list_papers_by_topic → generate_report). Naming clarity is good (update_paper_status vs update_reading_status are distinct, though semantically similar). Parameter constraints present (e.g., status enum, rating 1 - 5, relevance 0.0 - 1.0) but not all are formally declared as enums in schema.
Analyze knowledge gaps in a research topic. Uses the configured LLM to identify missing areas based on ingested papers.
Clear the rating of a paper.
Create a new research topic. Returns the topic ID.
Generate search keywords for papers to improve query recall. Uses the configured LLM or accepts manual keywords.
Fetch and store the references (citations) of a paper using Semantic Scholar. Optionally labels edges with citation intents.
Generate a research report for one or more topics. Synthesizes ingested papers into a structured markdown report.
Retrieve a single paper by ID. Returns full paper metadata and stored body text if available.
Output schemas not documented. Tools return opaque serde_json::Value with no field-level guidance. LLMs cannot plan downstream calls or extract specific fields without trial-and-error.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Agents cannot distinguish safe reads from destructive writes without parsing descriptions. Increases risk of accidental data loss.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | <=2025-11-25 | v2 |
Retrieve the research state overview for a topic or all topics.
Retrieve a single topic by ID.
Fetch and persist papers from external sources (arXiv, Semantic Scholar, OpenAlex, Europe PMC, Preprints) or local PDFs. Returns newly inserted papers, all papers now in library, and count of skipped files.
Initialize a research workspace: create the SQLite index schema and default config if missing. Returns the db path.
Link a paper to a research topic with an optional relevance score.
List recorded knowledge gaps for a topic or all topics.
List all papers in the library, ordered by insertion date (newest first). Returns papers with metadata.
List papers explicitly linked to a topic, ordered by relevance (highest first).
List all research topics. Returns topics with hierarchical structure.
Rate a paper on a scale of 1–5.
Rebuild the full-text search index. Scans all papers and updates the FTS5 index.
Search papers by title, abstract, and keywords using full-text search. Returns matching papers with relevance scores.
Search the full text of stored paper bodies for a query. Returns matching snippets with section and page context.
Update the processing status of a paper (unread, queued, in_progress, completed, abandoned).
Update the reading status of a paper (unread, queued, in_progress, completed, abandoned).
Status enums (unread, queued, in_progress, completed, abandoned) mentioned in descriptions but not formally declared as enum constraints in input schemas. LLMs may hallucinate invalid status values.
Error handling returns categorized errors (invalid_params, dependency_missing, upstream_error, internal_error) but lacks recovery guidance. E.g., 'upstream_error' does not tell the agent whether to retry, wait, or ask the user.
update_paper_status and update_reading_status are semantically similar and may confuse agents. Descriptions do not clarify the distinction or when to use each.