Loading ToolBench
Scrape a URL into structured JSON from a natural-language field spec
reapfield has 4 tools with complete input schemas and descriptions. Naming is clear and verb-driven (scrape, list_, refresh_, prepare_). Descriptions are detailed and contextual, explaining cost implications and partial results. However, output schemas lack explicit documentation in the code, ScrapeResult is Pydantic-derived but not formally documented in tool descriptions. Error handling is minimal; no recovery guidance or actionable error messages. Parameters are well-constrained (enums, defaults, ranges) but lack per-parameter descriptions in the schema itself. Security is strong (URL validation, trust boundary comments), but no tool annotations (readOnlyHint, destructiveHint) are present.
Show the CSS selectors already learned for a domain. Costs nothing.
Prepare a bug report about reapfield itself. Does NOT submit it. Call this only when you hit a real bug in reapfield during real work and have verified a fix. It searches existing issues first; if `duplicate` is true, stop. Otherwise show the user `body` and ask for authorization -- they open `submit_url` themselves. Never include credentials, private URLs, or personal data.
Forget the cached selectors for these fields so the next scrape re-derives them. Returns the entries that were dropped. Re-derivation is lazy by design: it needs a page to look at, and it happens on the next scrape of one. Config-pinned selectors live in a separate store and are never touched.
Extract structured fields from a web page. `fields` is comma-separated with optional inline types, e.g. "title, price:float, in_stock:bool". Field names are arbitrary. A field that could not be extracted comes back as null with an entry in `misses` explaining why -- that is a normal result, not a failure. Cost control: `no_llm=True` uses only structured data and already-cached selectors, so the call is free; `max_llm_calls` caps what an uncached page may spend. `scroll` and `paginate` cost extra fetches, not extra tokens. There is no `strict`: it exists on the CLI only to pick an exit code, and `misses` already tells you what was not found.
Output schemas not documented in tool descriptions. ScrapeResult is Pydantic-derived but LLMs cannot infer field meanings from class names alone. Missing: what fields are in 'records', what 'misses' contains, what 'llm_calls' represents.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). refresh_selectors modifies state (WRITE risk) but is not marked as destructive. scrape is read-only but unmarked. Agents cannot infer safety properties.
Error handling lacks recovery guidance. No tool returns actionable error messages (e.g., 'URL blocked: private IP. Try a public URL instead'). check_url() validates but exceptions are not caught or translated to user-friendly guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
prepare_issue_report description incomplete. Docstring cuts off mid-word ('IssueRepor'). Output schema unknown. LLM cannot determine what fields are returned or how to use the result.
Parameter descriptions missing from JSON Schema. 'domain' and 'fields' parameters have no descriptions in the schema object, only in docstrings. LLMs relying on schema alone cannot understand what these parameters do.