Loading ToolBench
Computes statistical significance, Z-score, and p-value for conversion A/B tests.
This MCP server has critical definition quality issues. The single tool 'execute_skill_action' is present with a clear input schema and basic description, but the server itself lacks proper MCP protocol implementation. The mcp_server.py file does not implement the MCP protocol specification, it merely prints a JSON status message and exits. There is no request/response handling loop, no support for tool invocation, and no error reporting mechanism. The tool definition exists in the schema but is never wired into an actual MCP server implementation. The naming is generic ('execute_skill_action') rather than action-verb specific ('calculate_ab_test_significance' or 'run_ab_test_analysis'). The description, while present, is terse (7 words) and lacks context on when/why to use the tool or what it returns. Input parameters have types and basic descriptions (average ~30 chars per param), meeting minimum bars but lacking actionable format constraints or examples of valid inputs.
Execute A/B test calculator statistical analysis
Tool name is generic and does not start with a domain-specific action verb. 'execute_skill_action' is vague; recommend 'calculate_ab_test_significance', 'analyze_ab_test_conversion', or 'compute_statistical_test'. Generic names force LLMs to read the description to understand intent.
Tool description is 7 words: 'Execute A/B test calculator statistical analysis'. This is below the minimum of 20 characters for actionable LLM guidance. Required: state WHAT it does, WHEN to use it (e.g., 'after collecting conversion data from A/B tests'), and WHAT it returns (statistical metrics). Example: 'Calculates z-score, p-value, and relative uplift for A/B test conversion rates. Use this after collecting visitor and conversion counts for both variants. Returns confidence level, significance status, and uplift percentage.'
Input schema lacks validation constraints and format documentation. Parameters (visitors_a, conversions_a, visitors_b, conversions_b) have types and basic descriptions, but no minimum/maximum bounds, no constraint that conversions ≤ visitors, and no guidance on data types (integers only, no floats). Add: min=0, examples, and constraints like 'conversions must be ≤ visitors' to prevent invalid calls.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 30 | 2026-07-28+ | v2 |
Output schema is not documented anywhere. LLMs cannot plan downstream operations or extract expected fields. The client.py shows the schema returns: conversion_rate_a_pct, conversion_rate_b_pct, relative_uplift_pct, z_score, p_value, is_significant, confidence_level. This must be declared in the MCP tool definition so agents know what to expect.
No error handling or recovery guidance. If invalid data is passed (e.g., negative visitors, conversions > visitors), the server will silently compute incorrect results or crash. Implement validation with actionable error messages: 'Invalid input: conversions_a (150) exceeds visitors_a (100). Ensure conversions ≤ visitors for both variants.'