MCP Server for Counterfactual Execution Test Generator Skill. Generates counterfactual edge-case inputs for tool schema parameters to systematically reproduce agent tool execution failures.
Single tool 'synthesize_test_suite' has a basic schema but critical gaps in parameter descriptions and output documentation. The tool name is clear (verb_noun pattern), but parameter descriptions are minimal or absent. The 'param_schema' parameter lacks guidance on expected format and structure. No output schema is documented, LLMs cannot predict what fields the response contains. Error handling is absent. The tool definition is explicitly visible in mcp_server.py, but quality falls well below production baseline (194 chars avg description, 72 chars avg param annotation).
Generate counterfactual edge-case inputs for tool schema parameters
Parameter 'param_schema' lacks description. LLMs cannot infer that it expects a dict mapping parameter names to type hint strings like {"amount": "float", "user_id": "str"}. This forces guessing and likely misuse.
No output schema documented. The tool returns a list of test case dicts, but LLMs have no visibility into the structure. They cannot plan downstream processing or extract specific fields.
Tool description is 97 characters, below the 194-char baseline and lacks context on WHEN to use it or WHAT it returns. Does not explain the counterfactual/edge-case testing purpose clearly enough for LLM selection.
No error handling or recovery guidance. If param_schema is malformed or max_tests is invalid, the tool will fail silently or return a generic error. LLMs have no guidance on what to do next.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
Parameter 'max_tests' has a default (10) but no description of valid range, constraints, or why this default was chosen. LLMs cannot reason about whether to override it.