Loading ToolBench
MCP server providing research paper search and weather information services
This server has 4 tools split across two domains (research papers and weather). The tool definitions are present with schemas and descriptions, but fall short of production quality in several critical areas. Naming is generally verb-noun compliant (search_papers, extract_info, get_amap_weather_*), but descriptions are inconsistent: research tools have decent English descriptions (78-94 chars), while weather tools are entirely in Chinese without English fallback, creating LLM usability issues. Parameter descriptions are present but sometimes minimal (e.g., paper_id: 'The ID of the paper to look for' lacks guidance on format). Output schemas are not explicitly documented, only return types are visible in docstrings. The server lacks error recovery guidance, validation constraints, and proper handling of missing data (extract_info returns a plain string on failure rather than structured error). No pagination support despite tool operations that could return large result sets. Security: no API key exposure in tool params (good), but AMAP_KEY is stored in .env without vault-level injection pattern.
Search for information about a specific paper across all topic directories.
获取未来几天（含今天）的天气预报情况。
获取当前城市的天气情况。
Search for papers on arXiv based on a topic and store their information.
Weather tool descriptions are in Chinese only ('获取当前城市的天气情况。'). LLM agents trained primarily on English will have difficulty understanding when and why to invoke these tools. This violates the tool-description pattern which requires descriptions to be machine-parseable for LLM selection.
Output schemas are not formally documented in tool definitions. search_papers returns 'List[str]' (paper IDs only) per docstring, but the actual tool writes to JSON files with rich metadata (title, authors, summary, pdf_url, published). LLMs cannot plan downstream calls without knowing what fields are available. extract_info returns 'str' which could be JSON or plain text, ambiguous. Weather tools have no documented response structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 57 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 44 | - | v1 |
extract_info returns a plain error string ('There\'s no saved information related to paper {paper_id}.') instead of structured error with recovery guidance. The pattern error-classification requires errors to tell the LLM what to do next (e.g., 'Paper not found. Try search_papers() to find it first.').
Weather tool parameters accept optional city_name (default=null) with no guidance on fallback behavior. Parameter description says 'use IP address for geolocation if None' but this is a hidden dependency. If IP geolocation fails, what happens? No recovery path documented.
search_papers tool has no pagination support. If an LLM searches for a popular topic and gets 5 results, there's no way to fetch the next 5. The tool description sets max_results=5 but doesn't document how to explore beyond that. For a research tool, this is a significant limitation.
extract_info parameter description 'The ID of the paper to look for' is too generic. It doesn't specify the format (short ID vs full arxiv ID), where to get it (from search_papers output), or what happens if the ID is malformed. Invites LLM confusion.
No tool-chain documentation. search_papers returns paper IDs, but how does an LLM know that extract_info expects those exact IDs? The two tools should cross-reference each other in descriptions.
AMAP_KEY is loaded from .env without validation or fallback. If the key is missing or invalid, the weather tools will fail silently or with unhelpful HTTP errors. No guidance to users on obtaining or configuring the API key.