Loading ToolBench
Autonomous web QA testing powered by Claude AI + Playwright. Launch a visible browser, explore websites, run accessibility audits, measure performance, and generate test reports.
WebMobAI demonstrates solid tool design with 26 well-named, action-verb-prefixed tools (webmobai_*). All tools have descriptions (avg ~150 chars, within baseline 34-392 range). Input schemas are present and properly typed for all tools. However, several tools lack parameter descriptions, and output schemas are not documented. Error handling is minimal, most tools return raw text without recovery guidance. Tool composition is strong (single responsibility, clear chains), but parameter validation rules are underdocumented.
Run a comprehensive accessibility (a11y) audit on the current page. Checks for missing alt text, form labels, color contrast, ARIA attributes, landmarks, and more. Returns issues grouped by severity.
Assert the number of elements matching a selector. Useful for verifying list lengths, table row counts, etc.
Assert that an element matching the selector is NOT visible on the page (either absent or display:none). Fails if the element becomes visible within the timeout.
Assert that an element matching the selector contains the expected text (substring match, case-sensitive). Useful for verifying labels, headings, error messages.
Assert that the current page URL matches an expected substring or pattern. Use this after navigation, form submit, or redirect to verify the user landed where you expected. You must supply `contains`, `pattern`, or both — an assertion with neither verifies nothing and is rejected.
Output schemas not documented. Tools return text responses but LLMs cannot plan downstream calls without knowing the structure of returned data (e.g., what fields does webmobai_get_run_history return? Is it JSON, markdown, or plain text?).
Error handling lacks recovery guidance. Tools return raw errors without actionable next steps. E.g., if webmobai_launch_browser fails, the LLM has no guidance on whether to retry, check prerequisites, or try a different browser.
Timeout parameters lack validation constraints. webmobai_assert_visible, webmobai_assert_hidden, webmobai_assert_text, webmobai_assert_url, webmobai_assert_count all accept timeout_ms but do not document min/max bounds. LLMs could pass 0 or 999999999.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | <=2025-11-25 | v2 |
Assert that an element matching the selector is visible on the page within a timeout (default 5s). Fails the test if the element is missing or hidden after the timeout.
Compare a current run's metrics against the median of recent runs for the same URL and flag deviations. Useful right after a test run to answer 'did this deploy regress LCP / introduce new console errors / etc.?'
Click an element on the page using a CSS selector. Supports standard CSS selectors and Playwright-specific selectors like text=, role=, etc.
Close the browser and end the session. Saves any recorded video.
Inspect what a selector currently matches on the page. Returns the match count and a per-match summary (tag, accessible name, role, position, key attributes). Use this when an assertion or click is failing and you're not sure whether the selector is wrong, ambiguous, or just too early.
AI-powered narration of a visual regression diff. Pass the paths to the baseline + actual PNGs (and optionally the diff mask PNG produced by webmobai_visual_snapshot) and Claude returns a plain-English bullet list of what visibly changed, plus a severity tag (cosmetic | content | structural). Requires WEBMOBAI_ANTHROPIC_API_KEY.
Generate a WebMobAI Scenario JSON from a natural-language description. Reads the current page (title, headings, interactive elements with selectors) and asks Claude to compose a deterministic, validated scenario you can run with webmobai-scenario. Browser must be launched on the target page. Requires WEBMOBAI_ANTHROPIC_API_KEY.
Get the full accessibility tree of the current page as seen by screen readers. Shows the hierarchy of roles and accessible names.
Return previous WebMobAI run summaries persisted to ~/.webmobai/history.json. Filter by URL to see only runs against the same target. Useful for trend analysis or surfacing recent regressions.
Launch an isolated browser in visible (headed) mode. Must be called before any other browser operation. Creates a fresh context with no cookies, cache, or user profiles. Supports Chromium, Firefox, and WebKit, and Playwright device presets (iPhone 13, Pixel 5, etc.) for mobile emulation including touch events and devicePixelRatio.
Run a full Google Lighthouse audit and return the official Performance, Accessibility, Best Practices, and SEO scores plus the lowest-scoring audits. Complements (doesn't replace) WebMobAI's per-axis tools. Lighthouse and chrome-launcher are optional dependencies — if missing, the tool returns install instructions. No browser is required: Lighthouse spawns its own headless Chrome.
Navigate to a URL. Waits for the page to load (DOM content loaded + network idle). Returns the page title and final URL.
Run N performance measurements against a URL and aggregate. Web Vitals have ±15-20% single-run variance; median over multiple runs is what you actually want to track. Returns median + p95 + min + max per metric.
Save the current browser session (cookies + localStorage) to a Playwright storageState JSON file. Use after logging in so a later launch can reuse the session via storage_state_path — the 'log in once, replay authenticated' flow. Requires a launched browser. WARNING: the written file contains session tokens; treat it as a credential and add it to .gitignore.
Take a screenshot of the current viewport. Returns the screenshot file path.
Scroll the page up or down by a specified number of pixels.
Slow down JavaScript execution by a multiplier. 4 mirrors Lighthouse's mobile profile (1/4 CPU); 1 (or null) clears. Chromium only.
Apply Chrome DevTools-style network throttling to the page (Chromium only). Use before measurement to approximate mobile users. Pass null to reset.
Change the browser viewport size. Useful for responsive testing.
AI-powered executive summary of the current audit. Collects accumulated test results + console errors from this session, plus a fresh accessibility audit and performance snapshot, then asks Claude to produce a prioritized markdown summary with top fixes and what's working. Browser must be launched. Requires WEBMOBAI_ANTHROPIC_API_KEY.
Type text into an input field. Clears any existing value first.
Selector parameter descriptions lack format guidance. Tools like webmobai_click, webmobai_type, webmobai_describe_selector accept 'CSS or Playwright selector' but do not explain the difference or provide examples of valid formats.
Dependency on external API keys (WEBMOBAI_ANTHROPIC_API_KEY) exposed in tool descriptions. Tools webmobai_explain_visual_diff, webmobai_summarize_audit, webmobai_generate_scenario_from_prompt mention 'Requires WEBMOBAI_ANTHROPIC_API_KEY' but do not explain how the LLM should handle missing credentials.