Loading ToolBench
Read-only MCP server and release gate over Verdict QA state: verdicts, findings, flaky quarantine, run history.
Verdict QA MCP demonstrates solid definition quality with consistent naming patterns, clear descriptions, and proper schema definitions. All 7 tools follow verb_noun naming (list_, get_) and include substantive descriptions (avg 156 chars). Input schemas are present with type definitions and parameter descriptions. However, output schemas are not explicitly documented in the visible code, and error handling guidance is minimal. The server is read-only (all tools annotated with readOnlyHint), which simplifies composition concerns but limits practical utility. Tool descriptions are domain-specific and actionable, though some could benefit from clearer guidance on when to use each tool vs. alternatives.
Findings for a project. status: 'open' (default), 'all', 'accepted' (the maintainer's accepted risks), or one of NEW / STILL_OPEN / RESOLVED / REGRESSED / ACCEPTED. REGRESSED sort first, then by severity, then by age (oldest first shown last run's pressure).
Run history, oldest first. Machine-native from runs.jsonl where it exists; the INDEX.md markdown parse remains as the legacy fallback for history that predates the file.
The flaky-test quarantine ledger, with an expired flag per entry.
The questions the tester parked for a person — parked ones with their age, and answers no run has read yet. Answer with `verdict-answer`.
Content of a QA report. Default: the last run's report; pass `report` (a path relative to the QA root, as returned by get_verdict/get_history) for an earlier one. Reports over 512KB are truncated with a flag.
The last recorded verdict, release blockers, and report path for a project.
Output schemas not documented in visible code. While input schemas are present with types and descriptions, return value structures are not formally specified. LLMs cannot plan downstream tool calls or extract fields without knowing what fields to expect.
Error handling lacks recovery guidance. Tools return error dicts with 'error' field but do not guide the LLM on next steps (e.g., 'Try list_projects() to see available projects'). Agents cannot self-correct from failures.
get_findings 'status' parameter accepts free-form strings with implicit enum values (open, all, accepted, NEW, STILL_OPEN, RESOLVED, REGRESSED, ACCEPTED). Should be declared as an enum constraint in schema to prevent hallucinated values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
List every project with a Verdict baseline in the solo state root.
get_report 'report' parameter description mentions truncation flag but does not specify the truncation threshold (512KB stated in code comment, not in description). Parameter constraints should be explicit in descriptions.
No pagination support documented. Tools like list_projects and get_findings could return large result sets. No limit, offset, or cursor parameters visible, risking context window exhaustion.