Loading ToolBench
A minimal research agent API that accepts research questions and returns concise answers with sources. Integrates with multiple data sources including arXiv, Papers with Code, data.gouv.fr, and web search.
This server has 21 tools with significant quality gaps. Most tool descriptions are present but lack depth and LLM-optimization guidance. Input schemas are visible for all tools but many lack proper type constraints and validation hints. Parameter descriptions are minimal (often single phrases). No output schemas are documented. Error handling is absent, tools return empty lists or bare exceptions rather than actionable recovery guidance. Tool naming is inconsistent: 'post_query' and 'post_query_stream' conflate HTTP verbs with action semantics; 'scrape_user_sources' is vague about what 'scraping' means in context. No tool declares permissions, idempotency, or destructiveness hints. Security concerns: register_user and login endpoints expose password parameters; no mention of secret injection or rate limiting.
Create or update user preferences.
Create a new preferred link for the current user.
Delete a preferred link for the current user.
Returns previous research queries and answers.
Login with username and password. Returns success message and sets HttpOnly cookie with access token.
Logout the current user by deleting the access token cookie.
Passwords exposed as tool parameters in register_user and login_for_access_token. Credentials must never appear in tool parameters, use server-side secret injection via environment variables or vault. Agent traces log every parameter, leaking secrets into logs and prompt history.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields without knowing response structure. Tools like search_arxiv, get_history, and read_users_me lack documented return types.
HTTP verb names (post_query, post_query_stream) conflate REST semantics with action semantics. Names should reflect what the tool does (query_research, stream_research_steps), not the HTTP method. LLMs infer intent from tool names before reading descriptions.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | <=2025-11-25 | v2 |
Accepts JSON with `question` and optional metadata and returns an answer with sources. Provide a question (and optional context) and receive a short answer with source items.
Streams Server-Sent Events (SSE) representing the execution steps of the agent.
Get user preferences by user ID.
Get all preferred links for the current user.
Get current logged in user. Useful for the frontend to check if the user is authenticated and retrieve their details since the token is stored in an HttpOnly cookie.
Register a new user with username and password. Returns success message and sets HttpOnly cookie with access token.
Scrapes a URL and returns the text content extracted via BeautifulSoup, truncated to 5000 characters.
Scrape toutes les sources (preferred links) de l'utilisateur, effectue un chunking de 1000 caractères et renvoie les résultats.
Stub search that performs a single HTTP GET if query looks like a URL, otherwise returns a fake result list.
Query arXiv using the `arxiv` package and return a list of results with title, summary and url. Note: `arxiv` is synchronous; run it in a thread to avoid blocking the event loop.
Search Hacker News via the public Algolia API and return top hits.
If `query_or_url` looks like a YouTube URL, return oEmbed metadata. Otherwise perform a lightweight search by scraping the YouTube search results page and returning titles + watch urls. Note: for robust search results prefer the official YouTube Data API.
Very small summarizer: take first result's text and truncate to 400 characters.
Update existing user preferences.
Update an existing preferred link for the current user.
No error handling guidance. Tools return empty lists or bare exceptions (e.g., search_arxiv, search_hackernews catch Exception and return []). LLMs cannot distinguish retryable failures from invalid input. Errors must tell the agent what to do next.
Destructive tool (delete_preferred_link) lacks confirmation or dry-run support. Agents make mistakes, irreversible operations should support a confirmation step before execution.