Loading ToolBench
Local-first Python SDK for AI agents: DOM compression, Action Map, SchemaSnap, self-healing selectors, MCP, and shadow-web exec agent loop
Shadow Web MCP provides 15 tools with explicit schemas and descriptions in source code (src/shadow_web/agent/tools.py, src/shadow_web/mcp/server.py). Strengths: clear verb-noun naming (navigate, click, fill, search), well-documented Action Map and content extraction workflows, good per-parameter descriptions. Weaknesses: inconsistent schema completeness (some tools missing required fields in snapshot/shadow_query definitions), output schemas largely undocumented (tools return objects but structure not formally declared in schema), minimal error guidance (no recovery hints), no destructive operation warnings despite write-risk tools (navigate, click, fill, search). Average tool description ~140 chars (within baseline 194-char median but leaning terse). Parameters well-typed but some lack constraint documentation (e.g., format enum values not fully explained in snapshot detail param).
Click an element by data-sid from the Action Map. Returns a page diff.
Compress raw HTML into clean markup, flat action map, and semantic groups.
Fetch full text for selected content_outline block ids (comma-separated, max 5 ids, e.g. 'p1,p3,p5'). Call at most once per page, then done(finalAnswer).
Build a terse Rendered Text Index outline (block ids p0, p1, …) from the current page HTML. Best for SPA/catalogs where products are not in HTML tables. Then ONE content_blocks(ids) or done.
Finish the bounded task with a final answer. Call as soon as you have enough for THIS task (not the whole site). Set complete=false + next_hint when the answer is a slice of a larger dataset.
Parse search engine results from the current page into a list of titled links.
Output schemas not documented in tool definitions. Tools like navigate, click, fill return structured observations (Action Map, diffs) but the response schema is not explicitly declared in the function schema object. LLMs cannot reliably plan downstream tool chains without knowing what fields to extract.
No error handling guidance in tool descriptions. Tools that interact with pages (navigate, click, fill, search) lack recovery hints. For example, navigate does not document what happens if the URL is unreachable or returns an error; shadow_query lacks guidance on malformed queries. Agents have no recovery path.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | 2026-07-28+ | v2 |
Fill an input field by data-sid. Returns a page diff.
Return the Content Block Index for the current page (cached, updated on navigate).
Return content coverage and signal-retention diagnostics for the current page.
Open a URL and return a bounded Action Map observation. Dense pages auto-downgrade to minimal (groups + top actions).
Extract tables, forms, and lists from the current page as JSON. Good for static catalogs. On SPA product grids with empty schema, use content_outline instead.
Execute a search query on a search engine (brave or yahoo) and extract result links as Action Map elements.
Filter actions in arbitrary HTML before navigating.
shadow_grep — filter current page actions before sending to LLM. Query examples: type:button intent:login group:Login Form label~/checkout/i type:button intent:login id:1,3,5 format: json | terse | xml
Refresh the page observation. detail=terse|minimal for Action Map; detail=diff for post-action deltas.
Missing destructive operation warnings. Tools marked WRITE (navigate, click, fill, search) modify page state but lack descriptions saying 'This modifies the page and may trigger side effects.' No confirmation or dry-run hint provided. Agents cannot reason about idempotency or rollback.
Enum descriptions incomplete. snapshot's 'detail' parameter declares enum values [terse, minimal, full, xml, diff] but does not explain what each returns (e.g., 'terse = Action Map compact view', 'diff = page delta since last action'). LLMs must guess which format to use.
shadow_query parameter 'format' lacks complete constraint documentation. Description says 'json | terse | xml' but does not explain the semantic difference (e.g., 'terse = readable action list', 'json = full object structure'). Agents will guess the wrong format.
Inconsistent parameter naming for IDs. Tools use 'sid' (shadow ID) for element references (click, fill) but the Action Map structure itself is not formally defined in tool schemas. An agent reading the schema cannot tell what 'sid' values to pass, it must infer from examples in the description.
search tool returns raw search results without pagination hints. If search returns 100+ results, there is no limit parameter or next_cursor documented. LLMs have no guidance on stopping or resuming pagination.
content_blocks max_ids constraint (5) is documented in code comment (MAX_CONTENT_BLOCK_IDS = 5) but NOT in the parameter description. An LLM cannot see the code and will not know to limit the 'ids' parameter, causing rejections on oversized requests.