Loading ToolBench
MCP server for scraping Readly magazines and generating PDFs
Readly MCP has 10 tools with basic naming and descriptions, but significant quality gaps. Tool names follow verb_noun convention (open_, smart_, get_, stop_, search_, list_, read_, extract_). Input schemas are present but sparse, most tools have minimal parameters with no type constraints or enums. No output schemas documented. Error handling is absent; tools do not guide recovery or categorize errors. No tool annotations (readOnlyHint/destructiveHint). Security: browser automation with credentials stored in env vars is reasonable, but no audit logging visible. Composition: tools are single-purpose but lack chaining IDs in responses (e.g., list_articles does not return article URLs for extract_article_text to use directly).
Extract the full text of an article from the current magazine issue by index. Call list_articles first to get available articles and their indices.
Returns the current status of the scraping job.
Parse the current Readly magazine page and extract article titles + URLs. Requires the browser to be on a magazine issue page.
Scrape the Readly newsstand for all available magazines/issues in your library.
Search Readly and open the latest issue for a magazine by name.
Opens the browser and navigates to Readly. Auto-logs in if READLY_AUTH_TOKEN env var is set. First run without the token will need manual login (persisted via user_data/ cookies).
Descriptions too short and lack context. 'Returns the current status of the scraping job' (9 words) omits WHEN to call, what fields are returned, or how to interpret status values. Rubric baseline: 194 chars avg, 10 - 1024 range. Most descriptions here are 20 - 60 chars.
No output schemas documented. Tools return data but LLMs cannot plan downstream calls without knowing response structure. E.g., list_articles says 'extract article titles + URLs' but no schema shows field names, types, or whether URLs are absolute/relative.
No error handling guidance. Tools do not indicate what errors are retryable, user-fixable, or fatal. E.g., smart_scrape may fail if browser is not open, but no error message guides the LLM to call open_readly_browser first.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | <=2025-11-25 | v2 |
Batch-extract full text for articles on the current issue page.
Search Readly for magazines matching a keyword query.
Starts the scraping process in the background.
Stops the current scraping job gracefully.
Missing tool annotations. No readOnlyHint or destructiveHint declared. smart_scrape and stop_scrape modify state (WRITE risk) but are not annotated. LLMs cannot distinguish safe reads from destructive writes without explicit hints.
Weak parameter descriptions. 'interval_seconds' says 'How long to stay on each page (default 120s/2min)' but omits valid range, units, or constraints. 'max_pages' has no guidance on typical values or performance impact. Rubric requires format, range, and allowed values in descriptions.