Loading ToolBench
Autonomous QA scanner: point it at your running app and it walks the real user flows in a headless browser and reports what's actually broken: broken pages/links, console and network errors, empty/placeholder data, broken forms, state-loss, bad loading/error states, responsive and accessibility issues, each with reproduction proof, severity, confidence, and a fix. CLI + MCP server. No API key. Uses Playwright and axe-core.
sniff-qa defines 6 tools with reasonable descriptions and schemas. The primary tool 'sniff' has a comprehensive description (280+ chars) and detailed parameter schema covering 14 optional parameters with enums and constraints. However, several tools (sniff_scan, sniff_run, sniff_discover, sniff_report) are narrow variants that duplicate functionality, violating the single-responsibility principle. Parameter descriptions are present but inconsistent in depth. Output schemas are not documented, LLMs cannot predict return structure. Error handling guidance is absent. sniff_install is a WRITE operation but lacks confirmation/dry-run safeguards.
Find real, reproducible QA bugs in a web app. Pass { mode, rootDir, baseUrl? }. Use mode:"walk" (RECOMMENDED) to drive a real browser, walk the app's user flows, and report broken pages/links, console & network errors, empty/placeholder data, broken forms, state-loss, dead-ends, bad loading/error states, responsive issues and accessibility, each with reproduction proof, severity, confidence, and a fix. mode:"scan" = source-only (no browser). mode:"report" = load last results. (mode:"run"/"discover" are legacy.) Auto-detects a running dev server if baseUrl is omitted. Prefer this over the narrow tools (sniff_scan / sniff_run / sniff_discover / sniff_report).
Run autonomous end-to-end discovery: extract the app domain from source, classify the app type, generate scenarios with edge-case variants, drive them through Playwright, and return a compact summary. Requires a running dev server (baseUrl optional, auto-detected if omitted). Writes HTML/JSON reports to sniff-reports/discovery/.
Install Playwright browsers and other dependencies needed for sniff to run. Idempotent; safe to call multiple times. Returns a summary of what was installed.
Load and display the last sniff crawl report (HTML or JSON). Pass the project root; sniff finds sniff-reports/sniff-report.html or .json and returns it.
Output schemas not documented. LLMs cannot predict return structure (fields, types, pagination). Breaks downstream tool chaining and forces agents to guess at response format.
Tool duplication: sniff_scan, sniff_run, sniff_discover, sniff_report are narrow variants of the primary 'sniff' tool. Violates single-responsibility principle and forces LLMs to reason about which variant to call.
sniff_install is a WRITE operation (installs Playwright browsers) but lacks dry-run, confirmation, or idempotency hints. Agents cannot safely preview impact before executing.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Run a full quality audit: source scan + browser checks (accessibility, visual regression, performance). The URL is optional -- sniff auto-detects running dev servers. If no server is found, runs source scan only. No API key needed.
Scan project source code for bugs. Finds debug statements, placeholder text, dead links, broken imports, hardcoded URLs, and API endpoint issues (missing validation, auth, error handling, secrets). Works offline, no browser or API key needed. Just pass the project path.
No error recovery guidance. Tools lack descriptions of failure modes, retryability, or next steps. E.g., if baseUrl auto-detection fails, what should the agent do?
Parameter descriptions inconsistent in depth. 'headless' and 'seed' lack descriptions entirely in sniff_run and sniff_discover. 'format' enum values not explained (what does 'summary' vs 'json' contain?).