Loading ToolBench
MCP server that provides arXiv research article search and retrieval capabilities
Single tool with adequate but minimal description. Input schema is complete with proper types and descriptions. Tool name follows verb_noun convention (search_arxiv). However, output schema is not documented, the tool returns a plain string of concatenated Article objects rather than a structured JSON response. Description is functional but generic (95 chars) and lacks guidance on when to use the tool or what downstream processing might expect. Parameter descriptions are present but brief. No error handling documentation or recovery guidance. Baseline: avg tool description 194 chars (this is 95); avg params per tool 4 (this has 2); output must document schema (missing).
Get `max_results` articles for a given search query on Arxiv.
Output schema not documented. Tool returns a plain string of concatenated Article objects. LLMs cannot infer the structure of returned data, cannot parse individual articles programmatically, and cannot chain to downstream tools. Per pattern:tool, 'Document the output schema. LLMs need to know what fields to expect so they can plan downstream tool calls and extract the right data.'
Tool description is generic and lacks actionable guidance (95 chars vs baseline avg 194). Does not explain WHEN to use this tool (e.g., vs other research sources), WHAT the user should expect in the response, or HOW to interpret the returned string. Missing dependency hints or context for tool selection.
No error handling documentation or recovery guidance. If the arxiv search fails, times out, or returns no results, the LLM receives no actionable error message guiding what to try next (e.g., 'Try a simpler query' or 'Broaden the search term'). Per pattern:recovery-guide, 'Error responses must tell the LLM what to do next.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 7 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 47 | - | v1 |
No result limit stated or enforced in tool description. The tool accepts arbitrary max_results values, risking context window exhaustion if an LLM requests 1000+ articles. Per mxe:enforce-result-limits, 'Even if the API allows returning thousands of items, cap results at a reasonable limit (e.g. 20-50) and offer pagination.'
Parameter descriptions are minimal and do not specify constraints. 'max_results' lacks guidance on valid range (min/max bounds). 'query' has no guidance on format, language, or example syntax. Per pattern:tool-description, 'Describe the expected format, range, and allowed values directly in the parameter description.'