Loading ToolBench
MCP server for Zephyr Scale on self-hosted Jira Server / Data Center: test cases, cycles, executions, plans, folders, attachments and automation via REST API v1
Strong schema definitions with Zod validation and comprehensive parameter documentation. Tool names follow verb_noun convention (delete_, update_, add_, remove_, reorder_). Descriptions are exceptionally detailed (500-1500 chars), explaining API internals, constraints, and error behavior, well above the 194-char baseline. All 6 tools have explicit input schemas with type definitions and descriptions. However, descriptions are overly verbose for LLM consumption (pattern:tool-description recommends 10-1024 chars; these exceed 1000), and output schemas are not formally documented in the code. Error handling is sophisticated (ToolInputError, ZephyrApiError, NetworkError) with recovery guidance in descriptions, but no per-tool error classification or confirmation patterns for destructive operations. Security is strong: secrets scrubbed from output, no credentials in parameters, read-only mode guard implemented. Composition is clean: each tool has one responsibility, though some (recreate_test_run_with_items) combine multiple steps.
Add test cases to a run in-place (the run is not recreated). Appends to the end of the item list in the order given. Partial update: only the fields passed are changed. Ids come from get_test_run_results values[].id or a create_test_result response; they are global, so one call may span runs. Verified live against a legacy Zephyr Scale Server build.
UNOFFICIAL — internal API: this tool calls /rest/tests/1.0, the same undocumented API the Jira UI uses. The vendor does NOT support it: endpoints may differ or be absent on another Zephyr Scale version. Registered only because ZEPHYR_ALLOW_INTERNAL_API=true. Permanently delete individual test executions (results) by their numeric ids (POST /testresult/bulk/delete with a bare JSON array of ids) — the public API v1 cannot delete results at all, they normally die only together with their run. CANNOT BE UNDONE. CONSTRAINT: the LAST execution of a run item cannot be deleted — the API rejects it with 'Test execution is the lastTestResult and cannot be deleted', so only older history entries are deletable; to drop an item's whole history remove the item itself with remove_test_cases_from_run. That rejection is the expected business rule, not a version problem, and it discards the whole batch: a mixed list is all-or-nothing with respect to REFUSED ids (nothing is deleted), while ids that simply do not exist are ignored and the rest of the batch is deleted. Ids come from get_test_run_results values[].id or a create_test_result response; they are global, so one call may span runs. The request shape was captured live from the Jira UI, not guessed. Verified live against a legacy Zephyr Scale Server build. Returns { requestedIds, deletionConfirmed: false, note } plus apiResponse when the endpoint answers with a body — it does NOT claim a deletion, because the endpoint cannot confirm one per id (see note); read get_test_run_results back to check.
Descriptions exceed 1000 characters, wasting LLM tokens. Pattern recommends 10-1024 chars; these average ~1200. Trim to 150-200 chars with key constraints only.
Output schemas not formally documented. Code shows tools return structured objects (e.g., { requestedIds, deletionConfirmed, note }), but no JSON Schema or TypeScript interface is visible in the source. LLMs cannot plan downstream calls without knowing return fields.
No confirmation or dry-run pattern for destructive operations (delete_test_results, remove_test_cases_from_run). Agents can permanently delete data without a safety gate. Implement a confirm_before_execute pattern or dry-run parameter.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | <=2025-11-25 | v2 |
Escape hatch from run immutability: create a new run with the same items and optionally updated header fields, then delete the source run. The new run gets a fresh key and starts with no execution history (or optionally carries over the latest execution of each item). Partial update: only the fields passed are changed; everything else falls back to the source run's value. The source run is deleted AFTER the new one is created successfully, so a failure leaves both runs in place — the new one is always reported in the response. Ids come from get_test_run_results values[].id or a create_test_result response; they are global, so one call may span runs. Verified live against a legacy Zephyr Scale Server build.
Remove test cases from a run in-place (the run is not recreated). Removes the items matching the given test case keys, along with all their execution history. Ids come from get_test_run_results values[].id or a create_test_result response; they are global, so one call may span runs. Verified live against a legacy Zephyr Scale Server build.
Reorder the items of a run in-place (the run is not recreated). Assigns new positions to the items matching the given test case keys, in the order given. Items not mentioned keep their current positions. Ids come from get_test_run_results values[].id or a create_test_result response; they are global, so one call may span runs. Verified live against a legacy Zephyr Scale Server build.
UNOFFICIAL — internal API: this tool calls /rest/tests/1.0, the same undocumented API the Jira UI uses. The vendor does NOT support it: endpoints may differ or be absent on another Zephyr Scale version. Registered only because ZEPHYR_ALLOW_INTERNAL_API=true. Edit ANY test execution (result) by its numeric id, including OLDER history entries (PUT /testresult with a single-element array) — update_last_test_result reaches only the newest execution of an item. Partial update: only the fields passed are changed, and at least one is required. `status` is the case-sensitive status NAME and is resolved to the internal testResultStatusId via GET /rest/api/2/project/{projectKey} + GET /project/{id}/testresultstatus, so projectKey (or ZEPHYR_DEFAULT_PROJECT_KEY) must be available whenever status is passed. Status/priority values are case-sensitive internal (non-localized) names; check how they are configured for this project. Ids come from get_test_run_results values[].id or a create_test_result response. Verified live against a legacy Zephyr Scale Server build. Everything not passed is preserved on that execution, including its environment and per-step scriptResults. Editing an OLDER execution does not change which execution the run item reports as its last one; editing the item's NEWEST execution does update the item's derived status. An id that does not exist (or was already deleted) is answered by this build with HTTP 500 and an EMPTY body, not 404, so a bare 500 here most often means the id is gone rather than that the input was too long or the endpoint is missing — the error hint for that response says the same. Returns { testResultId, applied, statusName?, note }. applied is an object of the INTERNAL field names actually SENT mapped to the values sent (executedBy → userKey, actualEndDate → executionDate, status → testResultStatusId as its resolved numeric id; comment, executionTime and actualStartDate keep their names) — the exact patch that went on the wire, with statusName echoing the status NAME behind that numeric id whenever status was passed.
recreate_test_run_with_items combines multiple concerns: create new run, optionally copy results, delete old run. This violates single-responsibility principle. If deletion fails, both runs exist; if creation fails, the old run is orphaned. Split into separate tools or add explicit rollback guidance.
Error messages reference internal API details (e.g., 'HTTP 500 with EMPTY body means id is gone'). While informative for developers, LLMs cannot act on this guidance. Wrap errors with actionable recovery steps: 'Test result not found. Verify the ID with get_test_run_results() and retry.'