Self-healing CI/CD agent system using Microsoft Agent Framework
PipelineHealer defines 6 GitHub workflow-related tools with acceptable naming consistency (all start with action verbs: get_, list_) and reasonably complete schemas. However, descriptions are generic and lack LLM-optimized context about WHEN to use each tool vs. alternatives. All 6 tools have input schemas with proper types and parameter descriptions, which is a strong foundation. Parameter descriptions are present but brief (mostly 20-50 chars). Error handling and output schemas are not visible in the provided code. The tools follow a read-only pattern (no state-modifying operations), which reduces some concerns, but descriptions do not articulate what makes each tool distinct or when the LLM should prefer one over another. Naming convention is solid (verb_noun pattern), but composition and schema documentation could be stronger.
Get annotations for a specific check run.
Get logs for a specific job.
Get recent commits for repository context correlation.
Get jobs for a workflow run.
Get details of a workflow run.
List workflows configured for a repository.
Tool descriptions are generic and lack LLM-optimized context. They do not explain WHEN to use each tool, how they differ from similar tools, or what downstream tools can be called with their outputs.
Output schemas are not visible or documented in the provided code. LLMs cannot plan downstream tool calls without knowing what fields are returned (e.g., does get_workflow_run return run_id, status, conclusion, logs_url?). This violates the pattern of documenting return types.
Descriptions do not clarify the distinction between tools that operate on similar resources. For example, get_workflow_run, get_workflow_jobs, get_job_logs, and get_check_run_annotations all retrieve job/run information but with different scopes. Descriptions should say when to call each one.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 62 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
No pagination guidance visible in tools that return lists (list_repo_workflows accepts per_page but no offset/cursor). Tools returning lists should document total count, pagination mechanism, and result limits. The default per_page=100 may produce large results that blow context windows.
Error handling patterns are not visible in the source code. No evidence of recovery guidance, error classification (retryable vs. user-fixable vs. fatal), or actionable error messages that guide LLM remediation.
Parameter descriptions for owner and repo are minimal ('Repository owner', 'Repository name'). Descriptions should clarify format constraints and give context, e.g., 'Repository owner: GitHub username or organization name (case-sensitive, no spaces or special chars except hyphen).'