Loading ToolBench
General-purpose Claude agent with Exa web tools for research, code analysis, data processing, and fact-checking
Server provides 11 tools with mostly complete schemas and descriptions. Tool names follow verb_noun convention (search, fetch, read, download, calculate, describe). Descriptions are present and reasonably detailed (avg ~120 chars), explaining WHAT and WHEN. However, several tools lack visible input schemas in the provided code (transcribe_audio, read_pdf, read_spreadsheet, read_document, describe_image, firecrawl_fetch show only descriptions, no schema). Parameter descriptions are generally good but some tools (e.g., transcribe_audio) have minimal documentation. Error handling and output schemas are not explicitly documented in the source. No evidence of permission gates, audit trails, or secret injection patterns despite tools accessing external APIs (Exa, Firecrawl, Whisper).
Evaluate a mathematical expression precisely using sympy. Supports arithmetic, algebra, calculus (integrate, diff), trig functions, factorials, solving equations, and more. Use this instead of mental math for any non-trivial computation.
Analyze an image file using Claude's vision capabilities. Returns the image for visual analysis. Supports PNG, JPEG, GIF, WEBP. Use this for any task involving images: charts, diagrams, photos, screenshots, scanned documents, etc.
Download a file from a URL and save it locally. Returns the local file path. Use this to fetch PDFs, images, spreadsheets, or other files before processing them with read_pdf, read_spreadsheet, read_document, or describe_image.
Get a direct answer to a question using Exa's answer API, which searches the web and synthesizes an answer with citations. Best for factual questions where you want a sourced answer rather than raw search results.
Extract clean text content from one or more URLs using Exa. Returns structured content with titles, dates, and text. Use this instead of WebFetch for content extraction.
Search the web using Exa's neural search engine. Returns relevant pages with titles, URLs, dates, and optionally extracted text content. Supports category filtering (news, research paper, tweet, company, github, etc.) and domain filtering. Use this instead of WebSearch.
Six tools (transcribe_audio, read_pdf, read_spreadsheet, read_document, firecrawl_fetch) have no visible input schemas in source code.
Five tools have minimal descriptions (<50 chars): transcribe_audio ('Transcribe audio files (mp3, wav, m4a) to text'), read_pdf, read_spreadsheet, read_document, firecrawl_fetch.
No evidence of error handling patterns in source. Tools calling external APIs (Exa, Firecrawl, Whisper) should document recovery guidance, retryability, and actionable error messages. Current code shows no try-catch with LLM-friendly error responses.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | 2026-07-28+ | v2 |
Fallback for JS-heavy pages that exa_fetch can't handle
Extract text from .docx and .pptx files
Extract text and tables from PDF files
Read Excel/CSV files as formatted tables
Transcribe audio files (mp3, wav, m4a) to text
No visible output schema documentation. Tools return results but LLMs cannot infer response structure. E.g., exa_search returns 'relevant pages with titles, URLs, dates, and optionally extracted text' but no formal schema showing field names, types, or pagination.
API keys and secrets (Exa API key, Firecrawl API key, OpenAI key for Whisper) are not visible in tool definitions, suggesting server-side injection. However, no audit trail, permission gates, or scope declarations are documented. Tools accessing external services should declare required permissions (e.g., 'read:web', 'write:local-files').