Loading ToolBench
An MCP server for searching academic papers on arXiv, extracting paper information, and managing research topics with prompt-based paper discovery and synthesis
The server defines 2 tools with partial quality. Both tools have descriptions and visible input schemas, but descriptions are generic and lack LLM-optimized guidance. Parameter descriptions are minimal. No output schemas are documented. Error handling is present but not recovery-oriented. The tools follow basic naming conventions but lack the specificity required for confident LLM selection. Neither tool has error categorization (retryable vs fatal) or actionable recovery guidance.
Search for information about a specific paper across all topic directories.
Search for papers on arXiv based on a topic and store their information.
Missing output schema documentation. Both tools return untyped results (List[str] and str) without documenting the structure or fields. LLMs cannot plan downstream operations or extract specific data.
No parameter descriptions. 'topic' parameter in search_papers lacks guidance on expected format, length, or domain (e.g., 'AI research', 'quantum computing'). 'paper_id' in extract_info lacks examples or format hints (e.g., '2301.12345').
No error recovery guidance. extract_info returns a plain string 'There\'s no saved information related to paper {paper_id}.' without suggesting next steps (e.g., 'Try searching for papers first using search_papers'). search_papers has no documented failure modes.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 45 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 39 | - | v1 |
Generic tool descriptions lacking context. 'Search for papers on arXiv based on a topic' does not explain WHEN to use search_papers vs extract_info, what the max_results default (5) means for typical queries, or what the returned paper IDs enable downstream.
No idempotency guarantee. search_papers writes to the filesystem each call. Repeated calls with the same topic may overwrite or merge JSON files unpredictably. LLMs may retry on transient failures, causing data loss or duplication.
No chaining IDs in responses. search_papers returns a List[str] of paper IDs only. To use extract_info, the LLM already has the IDs, but no context about which topic they came from. A complete response should include {'topic': str, 'paper_ids': [str]} to maintain context.
No parameter constraints or validation guidance. max_results defaults to 5 but has no documented bounds (min/max). LLMs could pass 0, negative, or excessive values. No enum or range constraint in the schema.
Naming ambiguity between tools. extract_info says 'Search for information' which mirrors search_papers' verb. A clearer name like 'get_paper_details' or 'retrieve_saved_paper' would disambiguate intent for LLM selection.