Loading ToolBench
skill-loop has 9 tools with schemas and descriptions, but significant quality gaps limit production readiness. Tool naming is inconsistent (skill_loop vs skill_loop_apply_fixes, mixing underscores and prefixes). Descriptions are present but often generic or lack actionable guidance. Parameters lack type constraints (enums, ranges). Output schemas are not documented. Error handling is absent, no recovery guidance, no categorization of retryable vs fatal errors. The unified skill_loop tool accepts free-form action strings instead of enums, inviting hallucinated values. No tool declares permissions or security boundaries. Composition is weak: skill_loop_apply_fixes, skill_loop_evaluate, and skill_loop_rollback operate on amendment IDs without clear chaining or idempotency guarantees.
Universal entry point for skill-loop. Accepts natural language actions like "scan", "status", "review", "fix", "update", or "history". Routes to the appropriate operation internally. Use this when the user asks about skill health in conversational terms.
Apply proposed amendments to flagged skills. Accepts an array of proposal IDs from a prior fix/amend action. Creates git branches and commits changes for evaluation.
Dry-run the detection pipeline against a hypothetical tool call. Returns detected skill, confidence score, and signal breakdown.
Audit data integrity. Checks registry consistency, validates run references, verifies index synchronization, and reports file sizes and pending contexts.
Score and evaluate a proposed amendment. Runs the amended skill against recent runs to determine if the fix improves outcomes. Returns pass/fail verdict and reasoning.
Prune old skill runs based on retention policy. Removes runs older than maxRunAgeDays and rebuilds the index.
skill_loop accepts free-form action strings ('scan', 'status', 'review', etc.) instead of an enum. LLMs will hallucinate invalid actions like 'analyze', 'debug', 'optimize'. This violates the constrained-input pattern and invites silent failures.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., amendmentId, proposalId). This violates the tool and response-shaper patterns.
skill_loop_gc and skill_loop_sync accept empty input objects with no parameters. Descriptions do not explain what they do, when to call them, or what they return. skill_loop_gc is marked DESTRUCTIVE but has no dry-run option or confirmation step.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
Manually log a skill run with outcome. Used to record skill executions outside of automatic detection.
Revert an accepted amendment. Marks the amendment as rolled_back and provides instructions for reverting git changes.
Flush skill telemetry to configured sync plugins. Retries failed events and expires stale entries.
No error handling guidance. Tools do not indicate which errors are retryable, which require user intervention, or which are fatal. No recovery hints (e.g., 'If amendment not found, call skill_loop to list recent amendments'). Violates error-classification and recovery-guide patterns.
Tool naming is inconsistent and vague. 'skill_loop_doctor' does not convey 'audit data integrity'. 'skill_loop_detect' is ambiguous, detect what? Detect skills? Detect issues? Names should start with action verbs (audit_, check_, validate_) and be self-documenting.