Loading ToolBench
Witness — the quality gate for agent-built web apps. Independent browser acceptance testing: CLI, MCP, AI-drafted checklists, visual regression, multi-browser. Agents self-test and pass; real browsers tell the truth.
dsh-verify defines 5 tools with complete input schemas and descriptions. Naming is clear and action-oriented (verify_*, generate_and_verify, health). Descriptions are detailed (150 - 300 chars) and explain WHAT, WHEN, and RETURN. However, output schemas are NOT documented, LLMs cannot predict response structure. Parameters lack type constraints (e.g., 'checks' is array of object with no item schema). Error handling is minimal; no recovery guidance. Tool composition is sound (each does one thing), but missing pagination/limits guidance for large result sets.
Witness — independent browser acceptance testing for agent-built web apps. Runs the dsh-verify CLI (real browser; PASS/FAIL with evidence receipts). Use when the agent built or changed a web app and must prove it works in a real browser.
AI-drafts an acceptance checklist for a URL (LLM writes it, a real browser executes it), then immediately runs it in real Chromium. Pass requirements as a plain-language description of what a human QA should verify (e.g. "dark-mode toggle changes the background color"). Requires DEEPSEEK_API_KEY or OPENAI_API_KEY in the server environment. Returns the drafted spec path plus the PASS/FAIL verdict, per-step results and report path.
Check that the dsh-verify MCP server and its Chromium browser are ready to run verifications.
Run a dsh-verify acceptance spec (JSON file, or glob) against a real headless Chromium. The spec may serve a local static dir (spec.serve) or target any URL (spec.base / step.url). Returns PASS/FAIL with per-step results and a self-contained HTML report path. Exit-code semantics: ok=true means every step passed. This is deterministic — no LLM judges the outcome.
Verify a live URL against a list of human-style checks, driven by a real headless Chromium. Checks are dsh-verify steps: goto/click/fill/wait/expect_text/expect_class/capture_style/expect_style_changed/expect_url_contains/expect_navigation/expect_console_errors/expect_network_errors/screenshot. If no goto step is given, a goto to the url is prepended. Returns PASS/FAIL with per-step results and a self-contained HTML report path. Deterministic — no LLM judges the outcome.
Output schemas not documented. LLMs cannot predict response structure (ok, passed, total, failed, report fields). Agents must guess what fields exist and risk extraction errors.
Parameter 'checks' in verify_url is array of object with no item schema. LLMs cannot infer valid check structure (action, selector, prop, var fields). Should define items schema with required/optional fields and descriptions.
No error handling guidance. Tool descriptions do not explain failure modes (e.g., 'if Chromium not installed', 'if spec JSON invalid', 'if URL unreachable'). Agents cannot plan recovery.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
generate_and_verify requires DEEPSEEK_API_KEY or OPENAI_API_KEY but does not document which takes precedence, fallback behavior, or error if both missing. Ambiguous dependency.
No pagination or result-limit guidance. If a glob spec matches 1000 files, verify_spec may return massive output. Should document max results and offer pagination.