Exposes Anansi's adaptive web scraping capabilities as MCP tools so any LLM can fetch pages, extract structured data, run full crawls, and manage pause/resume — all through a clean tool interface.
Anansi demonstrates solid tool design with comprehensive parameter schemas, clear descriptions, and thoughtful security validation. All 9 tools have explicit descriptions (avg 95 chars) and detailed input schemas with type definitions. Naming follows verb_noun convention (fetch_url, extract_data, crawl_site, etc.). However, output schemas are not documented, LLMs cannot predict response structure for planning. Error handling lacks recovery guidance (no 'try X instead' patterns). Tool descriptions are functional but brief; they state WHAT but not WHEN or WHY to use each tool. Security validation is exemplary (URL/proxy/regex/selector validation, sandbox confinement, action allowlisting), but descriptions don't explain these constraints to the LLM. Missing: per-tool risk classification in descriptions, output schema documentation, and error recovery hints.
Launch a full site crawl with concurrency, proxy rotation, pause/resume, and automatic selector learning.
Delete a crawl and all its associated data from the database.
Export crawl results to a file (CSV, JSON, or JSONL) in a sandboxed directory.
Extract structured data from a page using CSS selectors with self-healing capability.
Fetch a single page via HTTP or headless browser with optional actions, chunking, and format conversion.
Retrieve extracted items from a crawl as JSON, paginated via offset.
List all active, paused, and completed crawls with their current state and item counts.
Output schemas not documented. LLMs cannot predict response structure (fields, types, pagination) for planning downstream calls or extracting data. fetch_url, extract_data, crawl_site, get_crawl_items, list_crawls all lack documented return types.
Error handling lacks recovery guidance. Validation errors (UnsafeURLError, InvalidImpersonateError, PathOutsideSandboxError) are raised but descriptions don't tell LLM what to do next. E.g., 'Invalid proxy URL' should suggest 'Use http://host:port or socks5://host:port format.'
Destructive operations (delete_crawl) lack confirmation/dry-run pattern. Tool description does not warn that deletion is irreversible or suggest a list_crawls preview step first.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | <=2025-11-25 | v2 |
Pause an active crawl. Can be resumed later with resume_crawl.
Resume a paused crawl from where it left off.
Tool descriptions are functional but lack WHEN/WHY context. E.g., 'Fetch a single page via HTTP or headless browser' doesn't explain when to use fetch_url vs extract_data, or when browser mode is necessary. LLMs must infer intent.
Parameter constraints (e.g., max_pages, max_depth, concurrency limits) are enforced in code (_MAX_PAGES=100_000, _MAX_DEPTH=100, _MAX_CONCURRENCY=32) but not documented in parameter descriptions. LLMs cannot know valid ranges.