Loading ToolBench
An MCP server built with Smithery that provides tools to search and fetch arXiv papers via RSS feeds
Three tools with basic descriptions and schemas, but significant gaps in parameter documentation and output structure. The 'hello' tool is trivial (greeting only). The two arXiv tools have functional descriptions but lack parameter constraints (e.g., valid category values), output schema documentation, and error handling guidance. No tool annotations (readOnlyHint, etc.). Parameter descriptions are minimal, 'category' and 'keyword' lack format/constraint details. Output is returned as raw Python lists with no documented schema, forcing LLMs to infer structure. Average tool score: 52.
Returns all of today's brand-new arXiv submissions in a category via RSS.
Say hello to someone.
Search the daily arXiv RSS feed for a category, filtering by keyword in title or summary.
Output schemas not documented. Both arXiv tools return lists of dicts with fields (title, authors, summary, url, published) but no formal schema is declared. LLMs cannot reliably extract or chain results.
Parameter 'category' lacks constraints. No enum of valid arXiv categories (cs.AI, cs.LG, etc.), no format description, no validation. LLMs will guess or hallucinate invalid values.
No error handling or recovery guidance. If RSS feed is unreachable, malformed, or returns no results, tools fail silently or return empty lists. No actionable error messages for LLM.
No pagination or result limits. Tools return all matching papers in a single list. Large result sets (100+ papers) will exhaust context window and degrade LLM reasoning.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
Tool 'hello' is trivial and adds no value. Greeting tools are rarely useful in production agents. Consider removing or replacing with a meaningful arXiv-specific tool.