Loading ToolBench
MCP server exposing the pi coding agent as a delegable, steerable worker
Strong tool design with clear naming (verb_noun pattern), comprehensive descriptions (avg 180 chars), and well-structured schemas. All 12 tools have explicit input schemas with typed parameters and descriptions. Risk annotations (READ_ONLY, WRITE, REVERSIBLE, IRREVERSIBLE) are well-documented. However, output schemas are not formally documented in the source code, and error handling guidance is minimal. Tool composition is excellent, each tool has a single responsibility and chains naturally (spawn → status → steer/answer/follow_up). The init tool's gating mechanism is a thoughtful security pattern. Missing: explicit error recovery guidance, pagination for sessions list, and structured output schema documentation.
Stop a running pi session. Partial output stays readable via `status`.
Answer a question raised by a pi agent. Get `requestId` from `status`. Only pi extensions can ask, so questions appear only for delegates spawned with `extensions: true`; the MCP adapter's tool-approval and elicitation prompts are the usual source. A delegate waiting on one is blocked until you answer it.
Send another prompt to a delegate that has already finished, keeping everything it read and said. Use this instead of spawning a fresh delegate and re-explaining the task: the session still holds its own context, which yours never had to absorb. Returns immediately; poll with `status` as usual. For a delegate that is still working, use `steer` instead.
Drop a finished session from the review history, freeing its id for reuse.
READ THIS FIRST. Reports what this server can reach and how to drive it: permitted tools, the default model, models available per provider, and the recipes for delegating. Every other tool refuses until this has been called once.
Output schemas not formally documented. Tools return complex objects (sessionId, state, toolCalls, questions, text) but no JSON Schema definitions visible in source. LLMs cannot reliably parse responses or plan downstream calls.
Error handling lacks recovery guidance. No visible error messages that tell LLMs what to do next (e.g., 'Session not found. Call sessions() to list active sessions'). Errors are likely raw exceptions without actionable context.
answer tool's 'value' parameter uses union type without enum constraint. Description says 'Chosen option, text, or boolean' but no formal schema constraint. LLMs may pass invalid types.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
List models this delegate may use: pi's own scoped set plus any custom provider. Use to pick a `model` value.
Start a pi delegate and block until it finishes. Returns the final snapshot. Use for quick tasks that finish in under a minute; prefer `spawn` for longer work.
List pi sessions held by this server, running and finished. Finished ones stay readable for review until evicted (keeps the newest 50).
Start a pi delegate on a task. Returns immediately with a sessionId; poll with `status`. The delegate reads files and reasons on its own budget, then returns a result. Its context never enters yours.
Start several pi delegates at once. Each gets its own sessionId; poll with `status`. Useful for parallel work, but keep the fan-out reasonable (default ceiling is 10).
Check a background pi session. Returns state, turn count, tools used, latest text, and any pending questions the agent is waiting on. A non-empty `questions` array means it is blocked until you call `answer`. `toolCalls` traces every tool the delegate ran, in order.
Redirect a running pi agent mid-task. The message lands after its current tool call finishes, before the next model call. Use this instead of aborting when the agent is going the wrong way.
sessions tool lacks pagination. No limit, offset, or cursor parameters visible. Returning all sessions (kept up to 50) could still bloat responses. Should support limit/offset for scalability.
spawn_batch lacks per-item error reporting. If 3 of 10 requests fail, unclear whether tool returns partial success or complete failure. Should return array of {success, sessionId|error} per request.