Loading ToolBench
Web scraping and focused crawling MCP server. Exposes six tools for clean Markdown conversion, structured extraction with CSS selectors, outbound link discovery, focused crawling with strategy comparison, and hidden JSON API access.
ByteCrawl presents 6 well-named, focused tools with clear purposes. Tool names follow verb_noun convention (fetch_markdown, extract, list_links, focused_crawl, compare_strategies, fetch_json_api). Descriptions are detailed and actionable (average ~180 chars), explaining WHAT the tool does and WHEN to use it. Input schemas are present with type definitions for all parameters. However, several schema issues emerge: (1) extract accepts multiple optional parameter combinations (item+fields vs select alone) but this conditional dependency is mentioned only in the description, not expressed in the schema structure; (2) most parameters lack explicit constraints (no enums for strategy values, no bounds for max_pages, no pattern for URLs); (3) output schemas are completely absent from the visible code, the tool documentation does not specify what each tool returns; (4) error handling is translated into ToolError but recovery guidance is minimal. The code shows thoughtful error categorization (RateLimitError, BlockedError, UnreachableError) but the MCP descriptions do not expose these distinctions to guide LLM retry logic.
Run all three frontier strategies (Shark-Search, OPIC, BFS) over the same site on the same page budget and return them side by side, so you can see which one actually finds pages about 'query'. Costs three crawls; when you just want the relevant pages, use focused_crawl. Returns per-strategy stats, relevant-page counts and rankings, plus the winner.
Extract data from a page with CSS selectors. Three shapes. Call it with ONLY the url to discover what is extractable: it returns the page's repeated blocks, each with a ready-to-use item + fields and a sample record — use this whenever you have not seen the markup. Then call it with 'item' (the selector delimiting one record, e.g. 'article.product') and 'fields' (names to relative selectors supporting ::text and ::attr(name), e.g. {"title": "h3 a::attr(title)", "price": "p.price::text"}; a name ending in [] collects a list). Or pass 'select' alone for a flat list of one selector's values across the page, e.g. 'h3 a::attr(title)'.
Fetch a JSON API endpoint (the 'hidden API' scraping technique).
Fetch a web page and return it as clean Markdown, ready for LLM consumption. Reports tokens_estimate against tokens_html so you can state the saving rather than assume it (typically 5-10x).
Crawl a site starting from 'url', visiting the pages most relevant to 'query' first (focused crawling). strategy: 'shark' (topical best-first, default), 'opic' (structural importance) or 'bfs' (level by level). Returns pages ranked by relevance plus crawl stats.
No output schemas documented. All 6 tools return structured data but the MCP server code visible here does not specify return types, field names, or response shapes. LLMs cannot plan downstream operations (e.g., passing a link from list_links to fetch_markdown) without knowing the exact fields returned.
Parameter constraints missing. strategy in focused_crawl accepts a free-form string instead of enum:[shark, opic, bfs]. LLMs can hallucinate invalid values like 'random' or 'greedy' without enumerated constraints. max_pages appears unbounded, no min/max limits visible.
Conditional parameter dependencies not formally expressed. extract accepts (item + fields) OR (select alone), but this is stated only in the description. The JSON schema does not use oneOf or enforce mutual exclusivity. The LLM may pass conflicting combinations and receive an ambiguous error.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | 2026-07-28+ | v2 |
Every outbound link on a page as an absolute, deduplicated URL, with #fragments, mailto: and asset files dropped — the set of pages you could actually visit next. Pass raw=true for the untouched href attribute values instead.
Error recovery guidance incomplete. The code handles RateLimitError with retry hints and BlockedError/UnreachableError as terminal. However, the tool descriptions do not expose these error classes to guide the LLM, it sees only generic ToolError. LLMs cannot distinguish 'retry after 30s' from 'this URL is permanently blocked' and may waste retries.
fetch_json_api description is undersized (~60 chars) and vague. 'Hidden API scraping technique' assumes the LLM knows what that means. Missing: when to call this vs fetch_markdown, how to discover the API endpoint, what structure the params object should have.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present. All six tools are read-only and idempotent, but the server does not declare this via MCP tool annotations. This makes it harder for MCP clients to apply safety heuristics or optimize retry logic.
URL parameter validation not visible. No pattern constraint, no guidance on relative vs absolute URLs, no error message for malformed URLs. LLMs may pass invalid or relative URLs without local context.