Loading ToolBench
MCP Server for extracting, parsing, and transforming Jenkins build logs into XML format
Server has 15 tools with basic schemas and descriptions, but significant quality gaps. Most tool descriptions are present but generic (avg ~80 chars, below 194-char baseline). Parameter descriptions exist but lack constraint details (ranges, enums, formats). No output schemas documented. Error handling is minimal, no recovery guidance or actionable error messages. Tool naming follows verb_noun convention (good), but composition issues exist: extract_job_logs and extract_last_build_log are near-duplicates; transform_logs_to_xml and process_job overlap responsibilities. No tool annotations (readOnlyHint/destructiveHint). Security: no evidence of secret injection or permission gates. This is a typical community server with functional but under-optimized definitions.
Extract and parse logs for a specific job.
Extract and parse log for the last build of a job.
Get information about a specific build.
Get console log for a specific build.
Get console logs for multiple builds of a job.
Get information about a specific job.
Get list of Jenkins jobs.
No output schemas documented. Tools return data but LLMs cannot predict field names, types, or structure. Downstream tool chaining is error-prone.
Duplicate/overlapping tools: extract_job_logs (limit=1 default) vs extract_last_build_log; get_build_log vs get_last_build_log; transform_logs_to_xml vs process_job. LLM wastes reasoning cycles choosing between near-identical tools.
Parameter descriptions lack constraint details. 'limit' has no min/max (baseline: 1 - 100). 'job_name' has no format guidance. 'output_path' is optional but no default behavior documented. LLMs pass invalid values.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | <=2025-11-25 | v2 |
Get console log for the last build of a job.
Parse Jenkins build log text into structured data.
Process a Jenkins job: extract logs, parse, and transform to XML.
Save XML to file with pretty formatting.
Transform parsed logs to XML and save to file.
Transform parsed log data to XML Element.
Validate server configuration.
Validate XML structure.
No error handling guidance. Tools raise ValueError/exceptions but do not return actionable recovery steps. LLM cannot self-correct or retry intelligently.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). LLM cannot determine which tools are safe to retry or have side effects. Agents may retry destructive operations.