MCP server that finds tests which pass no matter what the code does
Three well-named, verb-led tools with clear, detailed descriptions (avg 180+ chars). All parameters have types and descriptions. Output schemas are documented in docstrings. Error handling is present but minimal, no recovery guidance or actionable error messages. Tool composition is sound: each tool has one responsibility. Naming follows verb_noun convention (scan_tests, list_rules, explain_rule). Parameter descriptions include constraints and defaults. Main gaps: no input validation examples, no error classification guidance, no per-item success/failure patterns for batch operations.
Explain one rule: what it detects and how the finding is usually resolved.
List every rule the scanner applies, with a one-paragraph description of each.
Scan a file or directory for tests that cannot fail. Reports tests that pass regardless of the behaviour of the code they claim to cover: assertions that are true by construction, tests with no assertion at all, empty bodies, and tests that are skipped while still appearing in the suite. Python is analysed with the `ast` module and is exact. Rust and JavaScript/TypeScript use a brace-matching text scanner, which is a heuristic tuned to miss cases rather than invent them. Every finding carries a file and line so it can be checked directly.
Error responses lack recovery guidance. scan_tests returns {'error': '...'} with no actionable next step. LLMs cannot infer whether to retry, ask user, or call a different tool.
No input validation or constraint enforcement visible in code. scan_tests accepts any path string; no check for path traversal, symlink attacks, or malicious input. explain_rule returns generic error for unknown rule but does not validate rule format.
scan_tests output schema not fully documented. Docstring describes 'findings' as a list but does not specify the structure of each Finding object (fields, types). LLMs must infer from code.
list_rules returns a flat dict with no pagination or limit. If RULES grows large, response could bloat context. No max_results parameter or truncation guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
explain_rule does not document the structure of the returned dict (keys: 'rule', 'detects', 'fix'). LLMs must reverse-engineer from code.