Loading ToolBench
Academic paper search, ingest, retrieval and analysis backend with MCP server interface.
Server has 6 tools with mostly complete schemas and descriptions. Naming follows verb_noun convention (search_papers, fetch_pdf, ingest_paper, etc.). Descriptions are present and substantive (100-200 chars typical). Input schemas include types, descriptions, and constraints (enums, min/max). However, output schemas are not documented in the visible code, LLMs cannot predict return structure. Error handling exists but lacks recovery guidance. No tool annotations (readOnlyHint/destructiveHint). Parameters are well-constrained but some lack natural-identifier alternatives (e.g., paper_id is opaque).
Run sequential batch ingest for multiple paper URLs and return a batch job identifier.
Download a single PDF to local storage without triggering ingest.
Query the current ingest status for either a job_id or a paper_id.
Run the single-paper ingest pipeline and return a job identifier for progress polling.
Retrieve relevant evidence chunks from ready papers using vector, text, or hybrid search.
Search external academic paper sources and mark whether they already exist in the local library.
Output schemas not documented. LLMs cannot predict return structure for any tool. Responses must include fields needed for downstream tool chaining (e.g., paper_id, job_id, evidence chunks with metadata).
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). fetch_pdf, ingest_paper, and batch_ingest modify state but lack destructiveHint. search_papers and retrieve_evidence are read-only but lack readOnlyHint.
Error handling lacks recovery guidance. build_tool_error() returns error codes and details but does not suggest next steps (e.g., 'Try search_papers() with a different query' for SEARCH_API_ERROR).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | <=2025-11-25 | v2 |
get_ingest_status requires either job_id or paper_id but both are opaque system IDs. No natural-identifier alternative (e.g., paper_title, doi). Agents must retain IDs from prior calls or fail.
batch_ingest 'options' parameter is an untyped object with no schema. LLMs cannot determine what keys/values are valid. Should define explicit optional parameters (skip_if_exists, parser, etc.) or document the object structure.