Loading ToolBench
Multi-agent report generator with research paper and web search capabilities
Two tools present with basic FastMCP registration. Both have descriptions and input schemas visible in code. However, significant quality gaps limit practical usability: (1) Parameter descriptions are minimal and lack format/constraint guidance, (2) Output schemas are not formally documented, return types are inferred from code (lists of dicts, formatted strings) but not declared in tool metadata, (3) No error categorization or recovery guidance, exceptions are caught but error responses are generic, (4) Missing enum constraints on free-form string parameters (search_depth), (5) No pagination support despite potentially returning unbounded results. Tool naming is adequate (verb_noun pattern: get_*, web_search) but compositions and parameter relationships lack depth. Average per-tool score: 48.
Fetches paper details including title, authors, abstract, PDF URL from arXiv API.
Conduct web search using Tavily API. Returns formatted results
Output schemas not formally documented. Both tools return complex structures (list of dicts, formatted strings) but tool metadata does not declare output type/structure. LLM cannot plan downstream calls or extract fields reliably.
web_search accepts 'search_depth' as free-form string but only 'basic' and 'advanced' are valid. Should be declared as enum to prevent LLM hallucination.
web_search returns unstructured formatted string instead of JSON-structured results. Violates pattern:response-shaper, LLM must parse markdown, wastes tokens, increases errors.
Parameter descriptions lack format constraints and bounds. max_results has no stated min/max (should be 1-50 or similar). No guidance on valid query formats.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | <=2025-11-25 | v2 |
Error responses are generic exception strings without recovery guidance. Exceptions caught but not categorized (retryable vs fatal). Errors like 'Search failed: ...' offer no actionable next step for LLM.
No pagination support. get_research_paper_details caps at max_results=5 (acceptable), but web_search default is also 5 and is silently capped by Tavily API. If results grow, context window risks without offset/limit pattern.