Loading ToolBench
A production-grade RAG system for academic paper retrieval using MCP, LangGraph, and Qdrant. Provides tools for searching arXiv papers, retrieving paper details, ingesting academic papers, and performing semantic search across indexed papers.
Server has 5 tools with complete input schemas and descriptions, but quality is uneven. search_papers, fetch_paper, and get_recent_papers have well-formed schemas with proper types and descriptions (10-100 chars per param). However, output schemas are entirely undocumented, LLMs cannot plan downstream calls without knowing what fields are returned. search_academic_papers and ingest_arxiv_papers lack parameter descriptions for 'limit' and 'max_results' respectively. Error handling is minimal: rate limiter raises RuntimeError but provides no recovery guidance. No tool annotations (readOnlyHint/destructiveHint) despite clear risk profiles (ingest_arxiv_papers is WRITE). Naming is clear and verb-forward (search_, fetch_, get_, ingest_), following baseline conventions. Parameter naming is consistent (arxiv_id, category, days_back). Missing: output schema documentation, error recovery hints, tool annotations, and per-parameter validation rules in descriptions.
Retrieve details of a single paper using its unique arXiv ID.
Fetch recent papers for a specific category within a number of days back.
Search arXiv, download, and index academic papers into the vector store.
Search for semantic contents and chunk sections from indexed academic papers.
Search for research papers on arXiv with optional category and date filtering.
Output schemas completely undocumented. LLMs cannot infer what fields search_papers, fetch_paper, get_recent_papers, search_academic_papers, and ingest_arxiv_papers return. This breaks tool chaining, if fetch_paper returns 'paper_id' but downstream tools expect 'arxiv_id', the agent wastes calls on field mapping.
No tool annotations (readOnlyHint/destructiveHint/idempotentHint). ingest_arxiv_papers is marked WRITE but has no destructiveHint in the tool definition. search_papers, fetch_paper, get_recent_papers, and search_academic_papers are READ_ONLY but lack readOnlyHint. Agents cannot distinguish safe from risky tools without explicit annotations.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | <=2025-11-25 | v2 |
Error handling provides no recovery guidance. TokenBucketRateLimiter raises RuntimeError('Rate limit exceeded. Please wait before making more requests.') but does not indicate retry delay, backoff strategy, or which tool to call instead. Per pattern:recovery-guide, errors must tell the LLM what to do next.
Parameter descriptions incomplete or missing. search_academic_papers 'limit' param has description but no min/max bounds. ingest_arxiv_papers 'max_results' has description but no validation rules.
No pagination or result-limiting guidance in tool descriptions. search_papers defaults to max_results=5, but description does not state this is a hard cap or explain what happens if more results exist. Per pattern:paginated-result, tools returning lists should document pagination behavior and limits.