Loading ToolBench
Educational repository containing tutorial implementations of AI coding agents with tool-based execution patterns. Includes minimal agent templates, bash-only agents, todo-based planning agents, and subagent patterns for context isolation.
This is a tutorial/reference implementation, not a production MCP server. Tool definitions are present with schemas and descriptions, but quality is inconsistent. bash tool lacks safety guardrails in description; TodoWrite and Task tools have vague, tutorial-style descriptions that don't guide LLM selection. No output schemas documented. Error handling is minimal. The code shows good security practices (path validation, command blocking) but these aren't reflected in tool descriptions. Average description length ~120 chars (below baseline 194), and several parameters lack actionable constraints.
Spawn a subagent for a focused subtask. Subagents run in ISOLATED context - they don't see parent's history. Use this to keep the main conversation clean. Agent types: - explore: Read-only agent for exploring code, finding files, searching - code: Full agent for implementing features and fixing bugs - plan: Planning agent for designing implementation strategies Example uses: - Task(explore): "Find all files using the auth module" - Task(plan): "Design a migration strategy for the database" - Task(code): "Implement the user registration form"
Update the task list. Use to plan and track progress.
Run a shell command. Use for: ls, find, grep, git, npm, python, etc.
Replace exact text in a file. Use for surgical edits.
Read file contents. Returns UTF-8 text.
Write content to a file. Creates parent directories if needed.
bash tool description omits safety constraints. Description says 'Run a shell command' but implementation blocks dangerous commands (rm -rf, sudo, shutdown). LLM cannot infer these guardrails from description alone, risking misuse attempts.
TodoWrite and Task tool descriptions are vague and tutorial-focused ('Use to plan and track progress', 'Spawn a subagent'). They lack WHEN to use, WHAT it returns, and HOW it differs from alternatives. LLMs cannot reliably select these tools.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract return values. E.g., bash returns a string, but what structure? read_file returns text, but does it include line numbers? Undocumented outputs force LLMs to guess.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Task tool uses generic enum values ('explore', 'code', 'plan') without explaining what each agent type does or when to pick one. Description mentions 'isolated context' but doesn't explain implications for tool availability or state sharing.
read_file 'limit' parameter lacks actionable constraint. Description says 'Max lines to read (default: all)' but doesn't specify valid range (e.g., 1 - 10000). LLMs may pass absurd values.