Loading ToolBench
A server to interact with Woodpecker CI.
This server has significant definition quality gaps. Tool naming is partially verb-based but lacks clarity; descriptions are minimal and fail to guide LLM tool selection; input schemas are present but poorly documented; output schemas are entirely missing. The two tools are READ_ONLY operations on a CI/CD system, which is appropriate, but the definitions do not meet production baselines for LLM-driven agents.
Analyzes CI pipeline failures using git context from IDE. Can auto-resolve repository and pipeline details from git information like repo name, PR number, or branch name.
This tool is used to extract all the different machine logs for the given pipeline
Missing output schemas. Neither tool documents what fields the LLM should expect in the response. Tool responses contain 'content' arrays with 'type', 'details', and 'moreDetails' fields, but this structure is not explicitly declared in a schema. LLMs cannot plan downstream tool calls or extract structured data without knowing the response shape.
Incomplete input schema definitions. The pipelineDetail.ts file is truncated mid-definition ('z.string().length(1)'), and gitPipelineDetailSchema uses inline Zod types not JSON Schema. Input parameters lack formal JSON Schema constraint declarations (min/max length, enum, pattern regex as JSON Schema constraints rather than prose).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 37 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 29 | - | v1 |
Tool descriptions are too brief and lack actionable guidance. 'This tool is used to extract all the different machine logs for the given pipeline' (73 chars) does not explain WHEN to use it, what conditions trigger it (successful vs failed pipelines), or what the agent should expect. Production baseline is 194 chars. Neither tool description states prerequisites, return value shape, or error recovery paths.
No error handling guidance in tool responses. Both tools can fail (e.g., 'Failed to fetch', running pipelines, git context resolution failures) but do not return actionable error messages. A response 'Failed to fetch' tells the LLM nothing about whether to retry, what to check, or how to recover.
Parameter names lack type suffixes for clarity. 'repoName' could be a human-readable name or system ID; 'pullRequestNumber' could be a numeric ID or string. Production tools use suffixes like 'repoName' vs 'repoId' and clarify in descriptions. The current schema does not distinguish whether inputs must be exact matches or can be resolved.
No pagination or result limits documented. The 'moreDetails' array in responses could grow unbounded, especially for pipelines with many failed steps. No pagination parameters, result limits, or documentation of maximum response size.