Loading ToolBench
The server provides three tools with basic functionality but significant quality gaps. All tools have descriptions and input schemas are present, but descriptions are inconsistent in quality and lack the depth needed for LLM decision-making. Parameter descriptions are minimal. No output schema documentation. Error handling is present but generic. The tools follow a reasonable naming convention (verb_noun) but lack rich contextual guidance.
Get detailed information about a specific paper. Args: paper_id: Paper identifier (DOI for Crossref, paper ID for Semantic Scholar) source: Source database ("semantic_scholar" or "crossref")
Search for papers by topic with optional date range. Note: Query length is limited to 300 characters. Longer queries will be automatically truncated. Args: topic (str): Search query (max 300 chars) year_start (int, optional): Start year for date range year_end (int, optional): End year for date range limit (int, optional): Maximum number of results to return (default 10) Returns: str: Formatted search results or error message
Search for papers across multiple sources. args: query: the search query limit: the maximum number of results to return (default 10)
No output schema documentation. Tools return unstructured strings with 'formatted' paper data, forcing LLMs to parse free-text responses instead of working with structured data. This increases token waste and hallucination risk.
Parameter descriptions lack actionable constraints. 'limit' is described as 'maximum number of results to return (default 10)' but no bounds (min/max) are specified. 'topic' mentions '300 chars max' but other params don't declare constraints. LLMs cannot validate inputs without explicit ranges.
Tool descriptions do not explain WHEN to use each tool vs. alternatives. 'search_papers' and 'search_by_topic' both search, what's the functional difference? When should an agent call one vs. the other? This ambiguity forces the LLM to guess and may cause wrong tool selection.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 30 | - | v1 |
No error recovery guidance. Error messages are generic (e.g., 'Error searching papers.', 'Unable to fetch paper details from {source}.') and tell the LLM nothing about what went wrong or how to recover. No suggestions for alternative actions.
Parameter 'source' in fetch_paper_details accepts an enum ('semantic_scholar', 'crossref') but the enum constraint is not explicitly declared in the schema, it's only mentioned in the description. LLMs cannot reliably parse text constraints; JSON Schema enum fields are required.
The code contains dead logic and incomplete implementation. The Dockerfile and pyproject.toml reference API keys (SEMANTIC_SCHOLAR_API_KEY, CROSSREF_API_KEY) but these are never actually retrieved from the environment or validated. The header construction in search_by_topic is incomplete (line 'Accept': 'appl...' is truncated).
No idempotence or retry guidance. If a network call times out or returns a 5xx error, the agent has no way to know if the action was partially executed (e.g., was the paper indexed?). Tools should declare idempotence or include unique request IDs.