Loading ToolBench
MCP server that loads and queries research papers from arXiv using LangChain's ArxivLoader
Single tool 'query' has critical definition gaps. Description is present but minimal (56 chars, below 10-char baseline for context). Input schema exists but lacks parameter descriptions, the 'input' parameter has no explanation of expected format, constraints, or examples. No output schema documented. No error handling guidance. Tool does not follow verb_noun naming (should be 'search_arxiv' or 'query_arxiv_papers'). No pagination, result limits, or structured output documented. Fails 6 of 7 definition quality dimensions.
Query arXiv for research papers and return summaries
Input parameter 'input' lacks description. LLMs cannot infer what search query format is expected (free text? arXiv ID? author name?). Parameter descriptions are mandatory per pattern:tool-description.
No output schema documented. Tool returns dict with 'page_content' and 'metadata' but LLMs have no visibility into structure, field types, or what metadata contains. Breaks pattern:tool and pattern:response-shaper.
Tool name 'query' is generic and ambiguous. Does not start with action verb. Should be 'search_arxiv_papers' or 'query_arxiv' to clarify intent. Current name conflicts with common generic patterns.
No pagination, result limits, or error handling. Tool truncates results to first 300 chars of first doc without documenting this limit. No guidance if no papers found (returns generic 'No documents found'). Violates pattern:paginated-result and pattern:recovery-guide.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 32 | 2026-07-28+ | v2 |
Description 'Query arXiv for research papers and return summaries' (56 chars) lacks context for LLM selection. Does not explain WHEN to use, what format input should be, or what the response contains. Below baseline of 194 chars for tool descriptions.