Loading ToolBench
An MCP server for searching and managing academic papers from arXiv, with tools to search papers by topic, extract paper information, and access papers organized by research topics.
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation, error handling, and output structure. search_papers has a clear action verb and acceptable description (95 chars), but lacks guidance on what constitutes valid topics or error recovery. extract_info has a shorter description (89 chars) and no documented output schema. Neither tool includes error classification, recovery guidance, or validation rules. Parameters lack constraint documentation (e.g., max_results range, paper_id format). No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Resources are present but not evaluated as tools. Overall structure is functional but falls short of production-grade quality.
Search for information about a specific paper across all topic directories. Args: paper_id: The ID of the paper to look for Returns: JSON string with paper information if found, error message if not found
Search for papers on arXiv based on a topic and store their information. Args: topic: The topic to search for max_results: Maximum number of results to retrieve (default: 5) Returns: List of paper IDs found in the search
No error handling or recovery guidance. extract_info returns a bare string error message ('There's no saved information...') instead of structured error with actionable next steps. search_papers silently fails on invalid topics or network errors.
Parameter constraints undocumented. max_results lacks min/max bounds (baseline: 1 - 100). paper_id format not specified (arXiv ID format: YYMM.NNNNN or YYMM.NNNNNvN). topic parameter accepts free-form strings with no validation or guidance on arXiv query syntax.
Output schemas not documented. search_papers returns List[str] (paper IDs) but LLM cannot infer structure or downstream usage. extract_info returns JSON string (not structured object), forcing LLM to parse unstructured text. No pagination support despite potential for large result sets.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
No tool annotations. search_papers is destructive (writes to disk) but lacks destructiveHint. extract_info is read-only but lacks readOnlyHint. Agents cannot infer safety properties or retry logic.
Composition gap: search_papers stores results to disk but does not return structured metadata (file path, storage location, topic directory). Agents cannot verify storage or chain to extract_info without additional discovery.