MCP server for Octivas web scraping, crawling, and search API
Three web-scraping tools with solid, action-verb naming and comprehensive parameter schemas. Descriptions are adequate (150-200 chars typical), enums are properly declared, and most parameters have type constraints and descriptions. However, output schemas are not formally documented in the code, responses are formatted as markdown strings without a declared structure. Error handling is present but minimal (catch-all with generic error messages). All tools are read-only, reducing security complexity. Tool chaining is weak: tools do not coordinate well (scrape returns markdown, crawl returns markdown, no structured output format documented for agent consumption).
Crawl a website starting from a URL and extract content from multiple pages. Useful for indexing documentation sites, sitemaps, or gathering content across a domain.
Scrape a web page and extract its content. Returns markdown by default. Can also extract structured JSON data using a schema or natural language prompt.
Search the web and extract content from the results. Returns search results with their full page content in markdown by default.
Output schemas are not formally documented. All three tools return formatted text (markdown strings) via formatResponse() helpers, but the MCP protocol definition does not declare the structure of these outputs. LLMs cannot plan multi-step workflows or reliably extract data from unstructured markdown responses.
Error handling is generic. All three tools catch errors and return a text-formatted error message ('Failed to scrape/search/crawl "X": <message>'), but do not classify errors as retryable, user-fixable, or fatal. LLMs cannot determine whether to retry, ask the user, or fail gracefully.
No pagination support in search or crawl. The search tool accepts a 'limit' parameter (max 20) but does not return a next_cursor or offset to retrieve subsequent results. For a search tool, this severely constrains multi-page result retrieval and forces agents to re-execute queries with different parameters rather than iterating through pages.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 13 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 51 | - | v1 |
Parameter descriptions lack explicit constraints and valid examples. For example, the 'tbs' parameter in search accepts specific time-filter codes ('qdr:h', 'qdr:d', 'qdr:w', 'qdr:m', 'qdr:y') but these are only listed in the description text, not as an enum. This forces LLMs to parse the description rather than relying on schema constraints.
No tool composition hints. The scrape tool accepts both 'schema' (JSON Schema) and 'prompt' (natural language) parameters for extraction, but does not document whether these are mutually exclusive or how they interact. This invites ambiguous calls from LLMs.
formatResponse() generates markdown strings without clear field delimiters or structures. For crawl, responses include page titles, links, summaries, and HTML snippets mixed into markdown. LLMs must parse this free-text output to extract specific fields (e.g., page URLs, summaries), which is error-prone and wastes tokens.