Loading ToolBench
A coding-agent SDK built on pydantic-ai, supporting 600+ models across 22 providers
Single tool 'Agent' has a well-structured Pydantic input model (AgentInput) with three parameters, each with descriptions. The tool description is comprehensive (280+ chars) and explains the subagent delegation pattern clearly. However, the output schema is not documented in the visible code, only the input is formally defined. The tool performs a complex operation (spawning subagents with isolation, permission inheritance, depth limiting) but lacks explicit error recovery guidance in the description. Parameter descriptions are adequate but could be more prescriptive about format/constraints. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are present.
Launch a subagent to handle a multi-step task. The subagent runs with its own context and returns only its final report, which keeps verbose intermediate work out of this conversation. Give it a complete, self-contained prompt naming exactly what it should report back: it cannot ask you follow-up questions and does not see this conversation. Its report reaches you and not the user, so summarize what matters in your own reply.
Output schema not documented. The tool returns ToolOutput but no schema is visible in the code showing what fields the LLM should expect from the subagent report.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). The Agent tool spawns subagents which may have side effects, but this is not signaled to the LLM via annotations.
Error recovery guidance missing from description. The tool validates input (unknown subagent_type, depth limit) but the description does not tell the LLM what to do if validation fails.
Parameter 'subagent_type' lacks constraint documentation. Description says 'Which agent definition to use' but does not explain valid values or how to discover them. The code shows validation against _definitions dict, but the LLM has no guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
No idempotency guarantee documented. Subagent spawning with the same prompt may produce different results (LLM non-determinism), but the tool description does not clarify retry semantics.