Loading ToolBench
A Model Context Protocol server for searching and downloading academic papers from ArXiv. Provides unified access to ArXiv's search API and PDF download capabilities.
Server has 3 tools with reasonable descriptions and parameter documentation. Naming follows verb_noun conventions (call_llm, list_models, echo_test). All tools have documented input schemas with type information. However, output schemas are not explicitly documented in the source code provided, parameter descriptions lack detail about constraints and expected formats, and error handling guidance is minimal. The call_llm tool is complex with multi-modal support but documentation doesn't clearly specify constraints or recovery paths. No pagination documented for list_models, which could return arbitrarily large results.
Download the PDF for an ArXiv paper to local storage.
Simple echo test to verify the server is working correctly.
Get detailed metadata for a specific ArXiv paper.
Get a list of available ArXiv subject categories for searching.
Search ArXiv for academic papers matching the given query.
Output schemas not documented. The call_llm tool returns an LLMResponse object, but the fields, types, and structure are not visible in the provided source. LLMs cannot plan downstream operations without knowing what fields to expect.
list_models lacks pagination and result-limiting documentation. No mention of how many models are returned, whether results are paginated, or if there's a limit. Large provider model lists could blow the context window.
Parameter constraints not formally specified. The 'temperature' parameter description says '0.0 to 1.0' but provides no enum or min/max constraints in schema. The 'provider' parameter description enumerates valid values in text, but no enum type constraint is visible in the schema definition.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 8 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 52 | - | v1 |
Error handling lacks recovery guidance. The _get_provider function raises ValueError for missing API keys and unsupported providers, but no mechanism is shown for returning actionable error messages that tell the LLM what to do next (e.g., 'Set OPENAI_API_KEY environment variable').
Multi-modal content handling is underdocumented. The call_llm description mentions 'OpenAI provider currently has a serialization bug with multi-modal content', but no structured guidance on which providers support which formats, expected file size limits, or supported MIME types beyond the inline enum.
No idempotency documentation. The call_llm tool makes external API calls with side effects (consuming tokens, generating responses). No statement about whether the tool is idempotent or if agents should expect different results on retry.