Loading ToolBench
MCP stdio server exposing Jira Data Center REST v2 tools to AI agents
Server has 9 well-named tools with clear verb-noun patterns (search_issues, get_issue, create_issue, etc.). Descriptions are present and moderately detailed (avg ~120 chars), explaining what each tool does and key constraints (e.g., 'max_results capped at 100'). Input schemas are visible with type definitions and parameter descriptions. However, output schemas are not formally documented, responses are inferred from code inspection. Error handling is present but minimal; most errors are Jira API pass-throughs without recovery guidance. Read-only mode guard is well-implemented. Missing: enum constraints for issue_type/transition, detailed field documentation for custom fields, and structured error messages with actionable next steps.
Add a comment to an issue. Returns the created comment id and date.
Create a Jira issue in a project. Custom fields may be passed by display name or raw customfield_XXXXX id.
List comments on an issue (id, author, created, body).
Get a single issue by key, including all fields and the workflow transitions available to the authenticated user.
List all fields with id, name, custom flag, type, and allowed values.
List projects with their keys, names, and issue types.
Output schemas not formally documented. Responses are inferred from code (e.g., search_issues returns {issues: [{key, summary, status, assignee, priority, issue_type}]}), but LLMs cannot see this structure in tool definitions. Agents must guess what fields to extract.
Custom field resolution (fields parameter in create_issue/update_issue) lacks documentation. LLMs cannot know whether to pass 'display name' or 'customfield_XXXXX' id, or how to discover valid custom fields. Requires calling list_fields first, but this dependency is undocumented.
Error handling is minimal. JiraError exceptions are raised but responses lack recovery guidance. E.g., 'TRANSITION_INVALID' error lists available transitions, but generic Jira API errors (404, 403, 500) are passed through without actionable context for the LLM.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | <=2025-11-25 | v2 |
Search Jira issues by JQL. Returns key, summary, status, assignee, priority, and issue type. max_results defaults to 50 and is capped at 100.
Move an issue through its workflow by transition name or id.
Update fields of an existing issue (custom fields by display name or raw id). Fails if a field is not editable.
No enum constraints for issue_type (create_issue) or transition (transition_issue). LLMs may hallucinate invalid values. transition_issue does validate at runtime and returns available options, but this is reactive, not preventive.
list_projects and list_fields have empty input schemas ({}). Descriptions are generic ('List projects...', 'List all fields...'). No pagination support documented, though Jira API likely supports it. Large result sets could exhaust context.