Loading ToolBench
MCP server for the Spider web crawling and scraping API — crawl, scrape, search, and extract web data for AI agents, RAG pipelines, and LLMs.
Spider Cloud MCP exhibits solid parameter schema definitions with extensive option coverage, but suffers from generic descriptions, missing output schema documentation, and poor error handling guidance. Tools are well-structured for a web scraping API but lack LLM-optimization patterns. All 3 tools have proper input schemas with Zod validation and enum constraints for categorical parameters (request type, return_format, proxy type). However, descriptions are minimal ('Crawl a website...' is 30 chars, below the 50-200 char optimal range) and do not answer WHEN to use the tool vs alternatives or what the LLM should expect in responses. No output schema is documented for any tool. Parameters like 'css_extraction_map' and 'wait_for' accept arbitrary objects with complex nested structures but lack examples or clear shape documentation. Error handling is absent, the apiRequest() function returns generic 'Spider API error $status' with no recovery guidance. Naming is clear and action-oriented (crawl, scrape, search) but descriptions fail to distinguish tool selection (when use crawl vs scrape?). The server does accept API key via environment variable (pattern:secret-injection compliance) but lacks permission gates and audit trails (pattern:permission-gate, pattern:audit-trail missing).
Crawl a website or multiple URLs and extract content
Scrape one or more URLs and extract structured content
Search for information and extract results
Tool descriptions are below optimal length (20-35 chars vs recommended 50-200). 'Crawl a website or multiple URLs and extract content' lacks WHEN to use this tool, differentiator from scrape/search, and expected output structure. LLMs cannot infer tool selection without explicit guidance.
No output schema documented for any tool. Users and LLMs cannot predict response structure. Crawl/scrape/search should document that they return array of objects with {url, content, title, metadata?, css_extracted?, links?, ...}. This forces LLMs to guess what fields exist for chaining.
Complex nested parameters (css_extraction_map, wait_for, automation, event_tracker) accept arbitrary objects but lack shape documentation. 'Shape: { "<url-path>": [{ name, selectors: [...] }] }' is too vague. Provide complete examples and TypeScript interfaces or JSON Schema for nested structures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | 2026-07-28+ | v2 |
No error handling or recovery guidance. apiRequest() returns 'Spider API error $status: $text' with no actionable next steps. Missing: error categorization (retryable vs user-fixable vs fatal), suggestions for invalid inputs, and rate-limit handling. LLM receives raw 500s with no path forward.
Tool descriptions do not differentiate crawl vs scrape vs search. When should LLM choose crawl (recursive site exploration, multiple pages) over scrape (single/multiple URLs, no crawl) or search (search engine queries)? Current descriptions do not answer this.
Parameter 'return_format' accepts union of enum and array (z.union([z.enum([...]), z.array(z.string())])). Documentation says 'Output format. Default: raw' but does not explain when to pass array vs string, what array semantics mean, or default behavior when omitted. This ambiguity will cause LLM confusion.
No parameter defaults are enforced by the tool definition, all optional params default to undefined. For critical params like 'limit' (0 for all pages vs explicit number) and 'depth' (25 default), LLM behavior is undefined. Clarify defaults in descriptions AND enforce them in code.
No pagination support documented. If crawl/scrape return large arrays (100+ results), context will explode. Should accept page/offset/limit and return total_count or next_cursor. Current schema has 'limit' for crawl but no offset/cursor mechanism.
Missing permission gates and audit trails. Tool descriptions do not state what permissions are required (e.g., does calling crawl with proxy_enabled require special API tier?). No audit logging of who called which tool with what params. Compliance/security tooling is absent.