Loading ToolBench
AI co-pilot for VFX artists using ComfyUI: 134 MCP tools across 6 swappable LLM providers (incl. any OpenAI-compatible endpoint), zero-LLM artist recipes, a default-deny safety gate, reversible LIVRPS edits, EXR-aware vision, workflow.lock provenance, and durable cross-session learning.
Comfy-Cozy provides 16 tools with clear action verbs and descriptive names, but quality is inconsistent across dimensions. All tools have descriptions (184-char average, within the 10-1024 baseline), and most parameter schemas are present with type definitions. However, critical gaps emerge: (1) No tool provides output schema documentation, the rubric requires 'Document the output schema' so LLMs know what fields to expect, yet none of the 16 tools declare their return types. (2) Many tools accept complex nested objects (proposed_change, initial_state, outcome) without documenting their internal structure or required/optional fields, forcing LLMs to guess. (3) Error handling is implicit, no tool description mentions what happens on failure, recovery paths, or classification (retryable vs fatal). (4) No tool implements validation guidance in parameter descriptions (e.g., 'must be 1-100', 'ISO 8601 format'). (5) Tool descriptions focus on WHAT but rarely explain WHEN to use a tool vs a similar one. These are systematic issues across the suite, not isolated tool problems. The server demonstrates competent naming and basic schema structure but lacks the precision and completeness expected of production-grade agentic tools.
Classify a change category into its evolution tier. Tier 1: auto-evolve. Tier 2: ratchet-validated. Tier 3: human-gate (constitutional anchor).
Build a USD scene from ComfyUI generation outputs (image, depth, normals, segmentation). Creates camera, mesh, material, and segmentation prims. All paths are optional — compose what you have.
Export the composed USD scene to disk. Supports usda (text), usdc (binary), and usdz (packaged) formats.
Extract ComfyUI workflow conditioning from a composed USD scene. Returns ControlNet depth path, FOV, DOF parameters, and prompt additions derived from camera and material analysis.
Get aggregate statistics over the meta-agent's calibration history. Shows total events, outcomes (improved/degraded/neutral), and average score delta.
Get aggregate statistics over stored experiences. Returns total count, average outcome per axis, average prediction accuracy, and unique signature count. Optionally filter by signature hash.
No output schema documentation across all 16 tools. LLMs cannot determine what fields to expect from responses, blocking downstream tool composition and forcing exploratory calls.
Complex nested object parameters (proposed_change, initial_state, outcome, decisions) lack internal field documentation. LLMs cannot construct these objects without examples or schema clarity.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
List improvement proposals and their current status. Optionally filter by tier or status.
Get prediction accuracy statistics from the current ratchet session. Shows how well the CWM predicted actual outcomes.
List pending and validated counterfactual simulations. Counterfactuals are 'what if' alternatives generated at session close. Pending ones can be validated; validated ones have been promoted based on observed accuracy.
Predict the outcome of a proposed workflow change using the Cognitive World Model (CWM). Returns predicted axis scores, confidence, learning phase, and the arbiter's surfacing mode.
Propose a system improvement to the meta-agent. The change is auto-classified into Tier 1 (auto-evolve), Tier 2 (ratchet-validated), or Tier 3 (human-gate) based on its category. Returns the proposal with tier and status.
Download a registered model to the ComfyUI models directory. Automatically resumes partial downloads and verifies SHA256 on completion. The model must be registered in the registry with a source_url. Blocks until the download finishes or fails.
Return current download progress and materialization state for a model. Shows registry status, bytes downloaded so far, total size, and percentage complete. Safe to call while a download is in progress.
SHA256-verify an already-downloaded model file against the hash stored in the registry. Returns pass/fail. On mismatch, updates the registry status to 'failed' so re-download can be triggered.
Manually record a workflow experience with outcome scores. Captures initial state, decisions made, and 6-axis outcome scores (aesthetic, depth, normals, camera, segmentation, lighting).
Validate a composed USD scene for quality. Returns multi-axis scores (depth_consistency, normal_agreement, segmentation_quality, camera_fidelity) plus an overall score and list of issues.
No error handling or recovery guidance in tool descriptions. Tools do not document what failures look like, whether they are retryable, or what the LLM should do next.
Parameter descriptions lack validation rules. Tools like provision_download and validate_scene do not specify ranges, formats, or constraints (e.g., 'prim_path must match USD naming convention /path/to/name').
Descriptions prioritize WHAT over WHEN. No tool explains when to call it vs a similar tool (e.g., when to use predict_experiment vs get_experience_stats). LLMs must infer selection criteria.
No documented composition or chaining hints. Unclear which tools produce outputs that feed into others (e.g., does compose_scene's output match extract_conditioning's input requirements?).
Parameters with optional enum values (format in export_scene, status in list_counterfactuals) document defaults but not consequences. No guidance on what happens if an invalid enum is passed.