MCP server for web content scanning and analysis
This server has 6 well-defined READ_ONLY tools with consistent naming patterns (verb-noun: check-links, crawl-site, extract-links, fetch-page, find-patterns, generate-site-map). All tools have descriptions (10 - 392 chars, within baseline 34 - 392) and explicit input schemas using Zod with type declarations and per-parameter descriptions. However, OUTPUT schemas are not documented, the tools return JSON but LLMs cannot determine what fields to expect in responses. Error handling maps service errors to McpError with descriptive messages, but recovery guidance is minimal. No tool annotations (readOnlyHint, etc.) are present despite all tools being READ_ONLY. Parameter descriptions are present and mostly complete, but lack concrete constraints like regex patterns or min/max bounds for numeric limits (e.g., maxDepth accepts 0 - 5 but no enum constraint visible in schema). Missing: pagination helpers for large result sets (extract-links and crawl-site could return thousands of URLs with no offset/limit mechanism), documented output field schemas, and tool annotations.
Fetches a given URL, extracts all anchor ('a') links, and checks each linked URL for validity (reachability via HEAD request). Returns a list of checked links with their status ('valid', 'broken', or 'invalid_url' if the href attribute couldn't be resolved to a full URL).
Recursively crawls a website starting from a given URL up to a specified maximum depth. It follows links within the same origin and returns a list of all unique URLs found during the crawl.
Extract and analyze all hyperlinks from a web page, organizing them into a structured format with URLs, anchor text, and contextual information. Performance-optimized with stream processing and worker threads for efficient handling of large pages. Works with either a direct URL or raw HTML content. Handles relative and absolute URLs properly by supporting an optional base URL parameter. Results can be limited to prevent overwhelming output for link-dense pages. Returns a comprehensive link inventory that includes destination URLs, link text, titles (if available), and whether links are internal or external to the source domain. Useful for site mapping, content analysis, broken link checking, SEO analysis, and as a preparatory step for targeted crawling operations.
Fetches the HTML content of a given URL. Optionally, it can select a specific part of the HTML using a CSS selector and convert only that part (or the whole body if no selector is provided) to Markdown format. Returns the resulting Markdown text.
Output schemas are not documented. LLMs cannot determine what fields responses contain, risking failed downstream parsing or chaining. Tools return JSON.stringify(results) but no schema is visible in source.
No tool annotations present. All 6 tools are READ_ONLY, which should be declared via readOnlyHint for agent planning and safety verification.
Numeric constraints (maxDepth 0 - 5, limit 1 - 5000) are documented in descriptions but not enforced as enum/min/max in Zod schemas. Description text alone is insufficient, LLMs cannot parse prose constraints reliably.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | C | 60 | <=2025-11-25 | v2 |
| 2026-03-09 | C | 63 | - | v1 |
Fetches a web page, extracts all anchor ('a') links, resolves their absolute URLs, and returns a list of links whose URLs match a given JavaScript-compatible regular expression pattern. Includes the URL and anchor text for each match.
Crawls a website starting from a given URL up to a specified depth and generates an XML sitemap containing the discovered URLs (up to a specified limit).
No pagination support visible in crawl-site, extract-links, or generate-site-map. Tools can return unlimited results, risking context window exhaustion. Spec recommends limit + offset/cursor + total_count pattern.
Error messages are logged with combined context, but no recovery guidance is returned to the LLM. McpError is thrown with generic messages like 'An unexpected error occurred' rather than actionable next steps (e.g., 'URL unreachable, verify it's publicly accessible').