Loading ToolBench
Local supervisor for bounded collaboration across configured AI agents.
Strong foundation with 11 well-named tools using clear verb_noun patterns (start, list, status, read, wait, post, stop, prune). All tools have descriptions (avg ~180 chars, within 10-1024 baseline). Input schemas present for all tools with proper JSON Schema structure. However, parameter descriptions are MISSING entirely, the schema shows property names and types but no 'description' field for any parameter. This is a critical gap: LLMs cannot infer what 'cursor', 'limit', 'view', 'tool_output' mean without descriptions. Output schemas are undocumented, no response structure defined. Error handling guidance is absent. Tool annotations (readOnlyHint, destructiveHint, idempotentHint) are not present in the schema definitions, though risk levels are noted in the evaluation metadata.
Run the versioned pre-start discovery protocol for one absolute workdir, including the backend catalog, effective workflow selections, probe evidence, policy, remediation, and accepted options. model_refresh controls the per-backend model catalog: 'none' and 'cached' (default) never contact a provider; 'fresh' runs live model-listing commands that can require auth and incur cost — confirm with the user before requesting it.
Return Markdown guidance for using agent-collab MCP tools safely and effectively.
List daemon-owned agent-collab sessions.
Post a follow-up message to a session awaiting input. The session must be settled and awaiting_input=true (check via wait_result or status). message is required; optional text carries the user's follow-up, and optional tool_results carries tool-call outcomes from the user's side (e.g. file edits, command outputs). Returns the updated session state immediately; use wait_events or wait_result to follow the session's next turn.
Prune daemon-owned sessions by age and status. apply=false (default) returns a dry-run report; apply=true executes the prune. retention_days (default from daemon config) sets the age threshold; sessions older than that are candidates. status_filter (optional) restricts to one status (e.g. 'done', 'failed'); omit it to prune all statuses. Returns a report with session_ids pruned, errors, and summary counts.
Parameter descriptions completely missing from all 11 tools. Schema defines property names and types but no 'description' field for any parameter (e.g., 'cursor', 'limit', 'view', 'tool_output', 'timeout_ms'). LLMs cannot infer parameter semantics without descriptions.
Output schemas not documented. No response structure defined for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., session_id from start, event structure from read_events).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 67 | 2026-07-28+ | v2 |
Read daemon session events after a numeric cursor offset. Tool payloads default to one-line summaries carrying absolute event ids; re-fetch one with cursor=<id>, limit=1, tool_output='full' (no types filter on a re-fetch: limit counts scanned events, not returned ones). view='digest' drops raw, caps text, and stamps event_id — the cheap view for watching.
Start a supervised collaboration session over the configured agent backends (Claude, Codex, Antigravity, xAI, ...) and return a session id. workdir is required because it selects project config and subprocess cwd. Call agent_collab_describe_options first before passing non-default backend_options or members (per-workflow slot names and eligible agents are advertised under workflows[].member_selection). The response settings default to a compact view; pass detail='full' for command_preview and backend_summary.
Return status, settings, and log paths for a daemon-owned session. settings default to a compact view (per-agent type, backend, capabilities, options); pass detail='full' to add command_preview and backend_summary.
Stop a running session. The session must not be terminal (check via status or wait_result). Returns the updated session state; use wait_events or wait_result to follow any final events.
Long-poll session events after a numeric cursor. Preferred way to follow a session: bounded polls keep you steerable, because your user's message reaches you only when a tool call returns — timeout_ms 20000-30000, never above 45000 (clients kill tool calls near 60 s). Cheap watch loop: view='digest' (drops raw, caps text, stamps event_id) plus types=['message','error']; re-fetch any event whole via read_events(cursor=event_id, limit=1, tool_output='full'). types filters the returned batch only — the cursor still advances over every scanned event and the wait still wakes on any event or status change, so an empty batch can mean filtered-out rather than idle. Always advance to the returned cursor and inspect status/terminal/error/failure/turn_outcomes on every response, including empty ones. Pacing: a full-duration block already paced the loop; wait ~20 s only after an early return carrying just routine progress or tool events.
Block until a session settles, then return its result. Settled = terminal, or awaiting_input while accepting input (none pending) — then you may post_message a follow-up. Result carries status, terminal, settled, answers (each agent's latest completed-turn answer plus event_id to re-fetch full via read_events), failure, and cursor; a settled non-done terminal result also carries events_tail — the last ~20 events as digest lines, usually enough to debug a failure; use read_events when an earlier event or full fidelity is needed (each line carries its event_id). timeout_ms bounds one server-side block (default 45000, max 600000; above ~45000 clients kill the call near 60 s); on expiry it returns a heartbeat (settled=false, no answers) — re-poll immediately, the 20s pacing rule does not apply. timeout_ms=0 never blocks: an instant peek of the current state, the cheap sweep across several delegated sessions. On an already-terminal session it returns immediately, so this is also the cheap harvest after a wait_events watch loop: never rebuild a result by re-reading message events. You are unresponsive for the whole block, so as a collector use it only when you want nothing but the outcome; otherwise watch with wait_events, then call this.
Tool annotations missing from schema definitions. Risk levels (READ_ONLY, WRITE, REVERSIBLE, DESTRUCTIVE) are noted in metadata but not exposed as readOnlyHint/destructiveHint/idempotentHint in inputSchema. LLMs cannot determine safety/reversibility from the schema alone.
Error handling guidance absent. No recovery hints in descriptions (e.g., 'If session not found, call list_sessions first'). No categorization of errors as retryable vs user-fixable vs fatal.
Enum constraints present but lack descriptions. E.g., agent_collab_start.sandbox=['read-only','none'] and agent_collab_read_events.view=['events','digest'] are not explained in parameter descriptions. LLMs must guess the semantic difference.