Loading ToolBench
MCP server for running Cypress e2e tests and retrieving test results, screenshots, and videos
Server has 6 well-named tools with clear action verbs (list_, run_, get_). All tools have descriptions (avg 120 chars, within baseline 34-392). Input schemas are present with type definitions and descriptions. However, output schemas are not documented in the source code, LLMs cannot predict return structure. Error handling exists but lacks recovery guidance. Tool descriptions are functional but generic; they do not explain WHEN to use each tool vs alternatives (e.g., run_spec vs run_all). Parameter descriptions are adequate but lack format constraints (e.g., spec path format, browser enum values). No pagination support despite tools returning lists (screenshots, videos). Security: projectPath is environment-injected (good), but no rate limiting or permission gates visible.
Get results from the most recent Cypress test run.
Find Cypress failure screenshots. Returns absolute file paths that can be viewed with the Read tool.
Find Cypress test video recordings. Returns absolute file paths.
List all Cypress e2e spec files in the project.
Run all Cypress specs (or a filtered subset). The dev server must be running at baseUrl (default localhost:8080) before calling this tool.
Run a single Cypress spec file and return results. The dev server must be running at baseUrl (default localhost:8080) before calling this tool.
Output schemas not documented. LLMs cannot predict return structure (fields, types, nesting). RunSummary, TestResult, SpecResult types exist in code but are not exposed in tool definitions.
Browser parameter lacks enum constraint. Description says 'Browser to run tests in' but does not list valid values (chrome, firefox, edge, electron). LLMs may hallucinate invalid browser names.
Error handling lacks recovery guidance. DevServerNotRunningError and CypressRunError are thrown but error responses do not tell LLM what to do next (e.g., 'Start dev server at baseUrl before retrying').
No pagination for list tools. get_screenshot and get_video return unbounded lists. If a project has 500+ screenshots, response bloats context window. No limit, offset, or cursor parameters.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | <=2025-11-25 | v2 |
Tool descriptions do not explain WHEN to use each tool. run_spec vs run_all distinction is unclear from descriptions alone. No guidance on prerequisites (dev server must be running) until you read the full description.