Loading ToolBench
Claude Code plugin: pairwise / t-wise combinatorial coverage for AI-written tests, backed by the coverwise WASM engine
Four well-defined tools with complete JSON schemas, strong parameter documentation, and clear use-case guidance. Tool names follow verb_noun convention (pairwise_generate, pairwise_analyze, pairwise_extend, pairwise_estimate). Descriptions are detailed (150-300 chars) and explain WHEN to use each tool. All parameters have types and descriptions. However, no output schemas are documented, LLMs cannot predict return structure. No error handling guidance. No security/permission declarations. Missing tool annotations (readOnlyHint, destructiveHint). Composition is sound: each tool has one responsibility, and they chain logically (generate → analyze → extend).
USE BEFORE calling a test suite finished, whenever the code under test takes 3 or more independent parameters. Reports which parameter interactions an EXISTING (AI- or hand-written) suite never exercises: `uncovered` diagnostics (capped at 1000 entries), the authoritative `uncoveredCount`, `omittedUncovered`, and `invalidTests` for rows that do not match the model. Reading the test file cannot answer this question; this tool can. Pass `constraints` when the model has them — tuples that cannot extend to a valid complete test are removed from the coverage universe, matching the generator's semantics.
Estimates the size of a covering array without generating it. Returns the estimated row count and the number of t-tuples that would be covered. Useful for deciding whether to increase strength, add constraints, or adjust the model before committing to a full generation.
USE when tests already exist and must be kept as they are. Appends the minimum extra rows needed to reach full t-wise coverage, preserving every existing row verbatim at the head of the suite. Returns `newRows` (just the additions, ready to append to the file), the full `rows` (existing + new), and the same coverage diagnostics as pairwise_analyze.
USE BEFORE hand-picking test cases whenever the code under test takes 3 or more independent parameters — CLI flags, config options, enum arguments, query params, feature flags, form fields, state-machine inputs. This applies even when the request is simply "write tests" and says nothing about coverage, combinations, or pairwise testing; choosing combinations by eye is exactly the failure this prevents. Builds the minimum t-wise (default pairwise) covering array from a parameter model plus optional constraints, and returns `columns` + `rows` that map 1:1 onto a table-driven test (test.each / t.Run table / pytest.mark.parametrize). Values come back in the JSON type they were declared with. Constraint-unreachable tuples are excluded after checking whether each partial tuple can extend to a valid complete test.
No output schemas documented. LLMs cannot predict return structure (columns, rows, coverage diagnostics, uncoveredCount, etc.). Forces agents to guess field names and types.
No error handling guidance. Tool descriptions do not explain what to do if constraints are unsatisfiable, model is invalid, or test cases don't match the model. No recovery paths.
No tool annotations. All four tools are read-only (no side effects), but readOnlyHint is not declared. Agents cannot distinguish safe tools from destructive ones.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
Parameter 'constraints' lacks examples or grammar reference. Description says 'See the coverwise skill for grammar' but no inline guidance. LLMs may struggle to write valid constraint DSL.
No pagination or result limits documented. pairwise_analyze caps uncovered diagnostics at 1000 but does not explain what happens if there are more. No guidance on handling large test suites.