Loading ToolBench
This JIRA MCP server demonstrates significant gaps across all definition quality dimensions. All 11 tools are explicitly registered with Zod schemas in src/index.ts, so inference is not an issue. However, critical problems emerge: (1) Naming is generally acceptable but lacks verb clarity and specificity, 'get-jira-boards-using-board-name' and 'get-jira-boards-using-project-id-or-project-key' are awkwardly specific compound names rather than clean verb_noun patterns. (2) Descriptions are extremely terse (5-6 words each: 'fetch jira issue details by key'), all fall below the minimum recommended 20 characters for meaningful LLM context. 94% of A+ tools have substantive descriptions (avg 194 chars); these are ~50% of that minimum. (3) Parameter schemas are present with Zod type declarations, but descriptions are universally minimal and non-instructive (e.g., 'The JIRA issue key' without format guidance). (4) Output handling is problematic: all tools return raw JSON.stringify() of API responses without documented schemas, field stripping, pagination support, or result limiting. The get-jira-issues-by-jql tool accepts pagination params but doesn't document the expected output structure, field names, or total count. (5) Error handling is generic and unhelpful, all catch blocks return identical 'Error fetching <resource>' text, providing no recovery guidance. No tool documents expected output fields or chains IDs for downstream calls. (6) No tool declares its output schema; LLMs cannot plan multi-step operations. The server reads data from JIRA but provides minimal guidance on what fields the agent should expect or how to chain tool calls. Per-tool analysis follows.
fetch jira boards in project
fetch jira boards in project
fetch jira components in project
fetch jira issue details by key
fetch jira issue types in project
fetch jira issues by jql
All tool descriptions are under 20 characters and provide almost no context for LLM tool selection. Examples: 'fetch jira issue details by key' (31 chars but generic), 'fetch jira users in project' (27 chars). These fail to answer WHAT the tool does, WHEN to use it, or what it returns.
Input parameter descriptions are minimal and fail to guide LLM input. Example: 'The JIRA issue key' tells the LLM nothing about expected format (e.g., 'PROJ-123'), valid length, case sensitivity, or when to use it vs other lookups.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 39 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 29 | - | v1 |
fetch jira priorities
fetch jira projects
fetch jira sprints in board
fetch jira status in project
fetch jira users in project
No tool documents its output schema. All tools return JSON.stringify(apiResponse) in a generic text field, leaving the LLM unable to predict what fields will be present, how to chain calls, or what IDs/references to extract. Per pattern:tool and pattern:response-shaper, tools must document return field names, types, and required chaining IDs (e.g., get-jira-issue-details should return issueKey, projectKey, assigneeId for downstream tools).
Error handling is generic and provides no recovery guidance. All catch blocks return identical 'Error fetching <resource>' text without indicating whether the error is retryable, what the LLM should try next, or what constraint was violated. Per pattern:recovery-guide and pattern:error-classification, errors must tell the agent: can I retry? Should I ask the user? What went wrong?
Naming is inconsistent and overly specific. 'get-jira-boards-using-project-id-or-project-key' and 'get-jira-boards-using-board-name' are two separate tools doing the same thing (fetch boards) but with different parameter names. Per pattern:tool and naming conventions, one tool should accept both project_id_or_key AND board_name (as separate optional params), or the tool names should be crisply differentiated (e.g., 'search_boards_by_project' vs 'search_boards_by_name'). Current naming invites LLM confusion about which to call.
Pagination parameters in get-jira-issues-by-jql (startAt, maxResults) lack validation and guidance. No maximum limit is enforced, description in the code has no guidance on default page size, and no total count or next_cursor is returned. Per pattern:paginated-result and mxe:enforce-result-limits, pagination must include a documented cap (e.g., max 50 results per call), return total count, and offer next_cursor or offset for safe iteration.
Two tools (get-jira-status-in-project and get-jira-issue-types-in-project) have a parameter named 'project' but expect a project ID, not a project name. The parameter description says 'The project ID' but the parameter name alone suggests it accepts a human-friendly project name. Per mxe:natural-identifiers, tool interfaces should accept human-friendly names (project names, usernames) and resolve them server-side, OR use suffixed parameter names (project_id, project_name) to disambiguate for the LLM.
No tool documents or enforces API rate limits, timeout policies, or partial failure recovery. If a JIRA API call hangs or rate-limits, the agent will block indefinitely. Per pattern:error-classification and production baselines, services should declare timeout values and return clear guidance if a call exceeds them.