Loading ToolBench
Chorus has 5 tools with clear action verbs (ask, ask_all, parallel_ask, managed_task, run_workflow), but critical gaps undermine quality. All tools have descriptions (50-150 chars, within baseline 194 avg), but parameter descriptions are sparse or missing. Input schemas are visible in code but lack formal JSON Schema type declarations, parameters are documented in docstrings, not in structured schema objects. No output schemas documented. Error handling is minimal (generic 'Unknown provider' message). The server is STDIO-only, which is a hard transport cap. Tool composition is reasonable (each does one thing), but schema completeness and error guidance are weak.
Ask a specific AI model a question, either by provider name or by role. By role (recommended): role="researcher", role="coder", role="reasoner", role="reviewer" By provider: provider="gemini", provider="copilot", provider="codex", provider="claude" Roles map to the best provider+model for that task (configured in ~/.chorus/config.yaml). agent_id creates an isolated session for this logical agent on the selected provider.
Ask ALL available models the same question in parallel and return all responses. Use this for comparisons, research, or getting multiple perspectives. All models run simultaneously — total time equals the slowest model.
Let an owner split a task, run workers in parallel, then summarize. The owner (you) provides a goal. Chorus asks a planner to break it into subtasks, spins up parallel workers to execute them, then asks a summarizer to synthesize results. Each worker gets an isolated session and can iterate independently.
Run multiple ask calls in parallel. Each task specifies a provider/role and prompt. Use this when you need to call the same or different providers multiple times simultaneously. All tasks run at the same time — total time equals the slowest task.
Execute a predefined multi-step workflow from ~/chorus/workflows/. Workflows are markdown files with numbered steps, each specifying a role and prompt. Use run_workflow(name='list') to see available workflows.
Input schemas lack formal JSON Schema type declarations. Parameters are documented in docstrings but not in structured schema objects with 'type', 'enum', or 'pattern' fields. LLMs cannot parse constraints from docstrings alone.
No output schemas documented. Tools return strings (e.g., ask returns str) but LLMs cannot infer response structure, field names, or downstream chaining requirements. Baseline: 100% of A+ tools document return types.
Error handling is minimal. 'Unknown provider' is the only explicit error message. No recovery guidance (e.g., 'Try search_users() first'), no error classification (retryable vs fatal), no actionable constraints on invalid input.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
Parameter descriptions are incomplete. 'cwd' appears in all tools but is never explained (what happens if it doesn't exist? is it relative or absolute?). 'agent_id' in ask is described but not in parallel_ask or managed_task.
STDIO transport only. Not remotely accessible; cannot be used by hosted MCP clients.