Loading ToolBench
MCP server for SplitLaunch A/B testing and experimentation platform API integration
SplitLaunch MCP server has 7 tools with basic schemas and descriptions, but significant gaps prevent production use. All tools have input schemas with typed parameters and descriptions, meeting minimum baseline. However, descriptions are generic (avg ~60 chars, well below 194-char baseline), lack WHEN/WHY context, and omit error guidance. No output schemas documented. Missing tool annotations (readOnlyHint, destructiveHint). experiments.set_status has NO visible input schema. Tools lack idempotency guarantees, recovery hints, and actionable error messages. Naming is acceptable (verb_noun pattern) but descriptions fail to distinguish similar tools (reports.experiment.details vs reports.experiment.channels both pull reports with identical params). No pagination, rate-limit, or batch operation guidance.
Add a domain to a SplitLaunch project for tracking
Create a new A/B experiment in a SplitLaunch project
Set the status of an experiment (e.g., launch, pause, stop)
Create a conversion goal in a SplitLaunch project
Create a new SplitLaunch project
Pull channel report for an experiment
Pull detailed experiment report for a specific goal
experiments.set_status has no visible input schema. Description only says 'Set the status of an experiment (e.g., launch, pause, stop)' with no parameters documented. Cannot determine what fields are required or their types.
All tool descriptions are under 60 characters and lack WHEN/WHY context. Baseline is 194 chars. Descriptions like 'Create a new SplitLaunch project' do not explain when to use this vs similar tools, what it returns, or prerequisites. LLMs cannot distinguish between reports.experiment.details and reports.experiment.channels, both pull reports with identical input params.
No output schemas documented. LLMs cannot plan downstream tool calls or extract required fields (e.g., does projects.create return a projectId?). Baseline: 100% of A+ tools have documented return types.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | 2026-07-28+ | v2 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Agents cannot distinguish safe reads from destructive writes. experiments.create and experiments.set_status are marked WRITE but lack destructiveHint annotation. reports.* are marked READ_ONLY but lack readOnlyHint.
No error handling guidance. Tools do not explain what to do on failure (retry? ask user? fatal?). No recovery hints like 'If projectId not found, call projects.create first.' No actionable error messages documented.