A URL fetching and web scraping MCP server that extracts text content, links, and performs Google searches with support for robots.txt compliance and domain policies
Thomisidae provides three web-scraping tools with explicit schemas and generally clear naming. However, several definition quality gaps prevent a higher score. All three tools have descriptions (though some are generic), and all have JSON Schema input definitions. Parameter descriptions are present but inconsistent in depth, some parameters like 'mode' in fetch_url_text lack actionable guidance on when to use each enum value. Output schemas are undocumented (not visible in the code provided), which violates pattern:tool-output-schema. Error handling is basic: the code shows validation errors (e.g., 'Invalid url', 'Blocked by robots.txt') but lacks recovery guidance or actionable next steps for agents. The tool names are clear and verb-first (fetch_*, google_search), meeting naming baseline standards. However, parameter naming has minor inconsistencies: 'raw' could be clearer as 'return_raw_content', and the 'respect_robots' override parameter is not self-documenting.
Fetches links from a page
Fetches the text content of a URL (optionally chunked and as raw HTML)
Search the web using Google Programmable Search (Custom Search API).
Output schemas are not documented. LLMs cannot predict what fields are returned, forcing trial-and-error or requiring external documentation.
Parameter descriptions lack actionable guidance. 'mode' enum in fetch_url_text lists 'auto', 'best_blocks', 'readability', 'raw' but does not explain when to use each or what 'best_blocks' extracts vs 'readability'.
Error handling is generic and non-actionable. 'Blocked by robots.txt' gives no recovery path. Should suggest: 'Blocked by robots.txt. You may retry with respect_robots=false if appropriate for the use case.'
fetch_page_links description is generic: 'Fetches links from a page'. Does not clarify whether it returns all links, unique links, absolute vs relative URLs, or what 'format' options produce.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 66 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 44 | 2025-06-18+ | v1 |
Parameter 'respect_robots' uses nullable boolean (type: ['boolean', 'null']), which is non-standard and confusing. Should use boolean with default=true. Null-checking in LLM context is error-prone.