Loading ToolBench
Custom MCP server that exposes framework-specific context to Playwright agents. Sits alongside @playwright/mcp (browser control) and playwright run-test-mcp-server (test execution) to complete the three-server architecture.
Server exposes 7 well-intentioned tools for QA automation with clear domain focus (requirements normalization, framework conventions, test health tracking, validation). Tool names follow verb_noun pattern and are action-oriented. However, critical gaps exist: most tools lack visible input schemas in the provided source code, parameter descriptions are present but inconsistent in depth, and output schemas are not documented. Error handling is minimal, tools return success/failure without recovery guidance. The normalize_requirements tool has a complex union type for 'tags' that is poorly specified. Overall, definitions are domain-appropriate but fall short of production-grade clarity needed for reliable LLM invocation.
Get this project's QA framework conventions before writing any code. Returns POM rules, tagging strategy, locator preferences, fixture/mock-data usage, and file naming. All generated tests must comply with these conventions.
Reads test-results/results.json (written by the JSON reporter) and returns structured failure details for every failing test: file path, line, error message, the step that failed, and screenshot path. Use this after test_run to give the healer agent clean, targeted input instead of raw terminal output.
Check the heal history of a test file before attempting to fix it. If healCount >= 3, regenerate from the source plan instead of healing again.
List all tracked test files with their heal counts. Use this to identify tests that need regeneration across the whole suite.
Normalizes requirements from Jira/file/pasted text into one canonical JSON contract for downstream planning.
Input schemas not visible in source code. Tools declare parameters in descriptions but lack explicit JSON Schema definitions with type constraints, enums, and validation rules. Cannot verify schema completeness or parameter constraints.
Output schemas not documented. Tools return structured data (test failures, health metrics, validation results) but no schema documentation visible. LLMs cannot plan downstream tool calls or extract fields reliably.
normalize_requirements 'tags' parameter uses union type (string|array) without clear guidance. LLMs will struggle to decide whether to pass 'tag1,tag2' or ['tag1','tag2']. Should be enum or explicit array with item type.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
Record a heal event after successfully fixing a test. Call this after every healing session so health data stays current and prevents over-healing in future sessions. Use healType='assertion' only after explicit human approval and include an approval note. 'evidence' must describe what you observed (DOM snapshot, screenshot path, or description) that proves the fix targets a real selector mismatch and not a product bug.
Validate a generated test file against framework conventions. Returns FAIL violations (must fix — blocked by pre-commit) and WARN suggestions. Call this after generator_write_test and fix all FAILs before reporting done.
Error handling lacks recovery guidance. Tools return success/failure but do not explain what to do on error (retry, ask user, check logs). No error categorization (retryable vs fatal).
record_heal_event requires 'humanApprovedAssertionChange' boolean but lacks confirmation/dry-run pattern. Tool accepts destructive assertion changes without explicit pre-execution confirmation step.