An MCP server that attaches an agent to a live Scrapy crawl and runs code inside it
Four tools with clear action verbs and reasonable descriptions (avg 180 chars). All have input schemas with type definitions. However, parameter descriptions are sparse or missing entirely, most params lack guidance on format, constraints, or valid values. The 'execute' tool accepts arbitrary Python code with minimal safety guardrails documented. Output schemas are not formally documented. Error handling exists but lacks recovery guidance. Naming is solid (list_jobs, status, execute, inspection_reference), but parameter-level documentation falls short of production baseline (100% of A+ tools have param descriptions; this server has ~40%).
Send code to execute in a live Scrapy crawl, return formatted result. Inspect and control Scrapy crawls running as processes on this machine. Whenever the user asks about a crawl that is currently running (whether it is alive, its progress, stats, effective settings, in-flight requests, enabled components, why it is slow or stuck), call list_jobs to find it and execute to read its live state, which is the source of truth: source files and logs only show what was configured. Scrapy Cloud jobs are out of scope; these tools only see local processes.
Reference for inspecting a live crawl: how it's wired and what's safe to read.
List the Scrapy crawls running as local processes on this machine that this server can attach to, and check each one's health.
Check that a crawl is alive and responsive, and report information about it.
Parameter descriptions missing or minimal. 'job_id' appears in 3 tools but only status/execute provide a description. 'code' parameter in execute lacks format guidance, constraints, or safety notes. 'timeout_sec' has no range or default documented.
Output schemas not formally documented. Tools return structured data (status dict, job list, reference text) but LLMs cannot infer field names, types, or required fields. No pagination guidance for list_jobs despite potentially returning many jobs.
execute tool accepts arbitrary Python code with no documented safety constraints, sandboxing, or validation. Description warns about 'live state' but does not explain what code is safe to run, what is forbidden, or how errors are handled.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
Error handling exists (RequestError, JobError) but recovery guidance is minimal. Code comments mention 'isError' mapping but tool descriptions do not tell LLMs what to do on failure (retry? ask user? abort?).
inspection_reference tool has empty input schema ({}). No description of what reference data it returns or when to call it. LLMs cannot determine its purpose or expected output structure.