Loading ToolBench
The Model Context Protocol (MCP) is an open-source implementation that bridges Jenkins with AI language models following Anthropic's MCP specification. This project enables secure, contextual AI interactions with Jenkins tools while maintaining data privacy and security.
The server provides 20 tools covering Jenkins operations (builds, jobs, nodes, queues). Most tools have basic descriptions and parameter schemas, but quality is inconsistent. Naming is generally verb_noun compliant (get_*, build_*, stop_*, etc.), but descriptions are often terse (typically 30-60 chars, below the 50-200 char baseline for LLM optimization). Parameter descriptions are sparse, many parameters lack clarity on expected format, range, or constraints. Output schemas are not documented in the MCP tool definitions (only parameter schemas visible). Error handling guidance is absent. Security considerations are undocumented. Average per-tool score across all 20 tools: 62.
Build a job in Jenkins
Cancel a specific item in Jenkins queue
Get all jobs from Jenkins
Get all nodes from Jenkins
Get all items in Jenkins queue
Get specific build info from Jenkins
Get logs from a specific build in Jenkins
Output schemas are not documented. Tool descriptions state what is returned (e.g. 'Get all running builds') but do not formally specify the structure (which fields, types, whether arrays are paginated). This forces LLMs to reason about response structure without explicit guidance, increasing hallucination risk and causing agents to request data that may not be present in the response.
Descriptions are too brief (30 - 60 characters average, well below the 50 - 200 char baseline). They state WHAT the tool does ('Get all running builds') but not WHEN to use it, what it returns, or how it differs from related tools. Example: 'get_build_info' could clarify 'Retrieves full build metadata including status, duration, and artifacts' instead of just 'Get specific build info from Jenkins'. This brevity reduces LLM confidence in tool selection and composition.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 57 | - | v1 |
Get the pipeline source code of a specific build in Jenkins
Get stage information from a Jenkins pipeline build
Get specific job config from Jenkins
Get specific job info from Jenkins
Get all branches for a specific multibranch pipeline job
Get all multibranch pipeline jobs from Jenkins, optionally filtered by patterns
Get node config from Jenkins
Get a specific item in Jenkins queue
Get all running builds from Jenkins
Get test results from a specific build in Jenkins
Trigger a scan of a multibranch pipeline to discover new branches
Search job by specific field
Stop a specific build in Jenkins
Parameter descriptions lack detail. Parameters like 'fullname' (appears in 8+ tools) and 'build_number' lack guidance on format, allowed values, or what 'fullname' means in Jenkins context (job path, full name with folders?). The pattern description says parameter descriptions should explain format, range, and constraints. Current param descriptions are 1 - 2 words; they should be 20 - 50 chars with actionable detail.
Error handling guidance is absent. No tool documents what happens on failure (e.g. 'job not found' → what should the LLM do next? Try search_jobs?). The pattern 'recovery-guide' requires error responses to tell the agent what to do next. Current implementation likely returns raw Jenkins errors without recovery hints.
build_job and scan_multibranch_pipeline are write operations, but descriptions do not explicitly state that they modify Jenkins state. Per the 'command-tool' pattern, tools that create, update, or trigger actions must clearly say so. This is critical for agents to understand which calls are idempotent and which have side effects.
No pagination parameters visible. Tools returning lists (get_running_builds, get_all_jobs, get_all_nodes, get_all_queue_items) have no limit, offset, or cursor parameters. Without pagination, LLMs requesting 'all' items risk exhausting context or hitting Jenkins API timeouts. The 'paginated-result' pattern requires pagination support and result limits.
No security declarations. No tool documents required permissions (e.g. 'requires Jenkins read:jobs, read:builds'), credential injection, or scope boundaries. The 'scope-declaration' and 'secret-injection' patterns require tools to declare permissions and ensure secrets are never exposed in parameters.
Search tools lack result limits. search_jobs accepts multiple regex patterns but does not state a maximum number of results returned. If a pattern matches 10,000 jobs, the response will be massive and likely exceed token budgets. The 'enforce-result-limits' pattern requires caps at 20 - 50 items and offers pagination.