Loading ToolBench
A hand-written terminal coding agent with agent loop, tools, context management, safety gate, and self-built benchmark. No agent frameworks.
MiniClaudeCode defines 10 tools with explicit schemas and descriptions. Most tools have clear verb-noun naming (read_file, write_file, edit_file, list_dir, grep_search, run_command). Descriptions are present and reasonably detailed (avg ~80 chars), explaining WHAT each tool does. However, several critical gaps reduce quality: (1) Parameter descriptions are sparse or missing for optional params (offset, limit in read_file; timeout in run_command; replace_all in edit_file). (2) No output schemas documented, LLMs cannot predict return structure. (3) Error handling is minimal; no recovery guidance. (4) No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels. (5) write_file and edit_file lack confirmation/dry-run patterns for destructive ops. (6) run_command accepts arbitrary shell commands with no input validation or injection guards documented.
Replace exact text in a file. old_string must match exactly and be unique.
Find files by glob pattern, e.g. 'src/**/*.py'.
Regex search across files; returns path:line: text matches.
List directory entries (dirs first, with file sizes).
Read a text file with line numbers. Use offset/limit for large files.
Run a shell command in the workspace root (tests, builds, git, etc.).
Convert text to a lowercase URL slug; non-alphanumerics become dashes.
No output schemas documented. LLMs cannot predict return types (e.g., does read_file return {content: string} or {lines: [string]}? Does list_dir return {entries: [...]} or flat array?). This forces agents to guess and wastes tokens on clarification.
run_command accepts arbitrary shell commands with no documented input validation, injection guards, or sandboxing. Agents can be tricked into executing malicious commands. Missing timeout enforcement documentation and error recovery guidance.
Destructive tools (write_file, edit_file, run_command) lack confirmation/dry-run patterns. No tool annotations (destructiveHint) to signal irreversibility. Agents can overwrite files or execute commands without safeguards.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | 2026-07-28+ | v2 |
Write your full todo list. Use for multi-step tasks; mark exactly one item in_progress.
Count words (whitespace-separated) in the given text.
Create or overwrite a file with full content. Prefer edit_file for existing files.
Optional parameters (offset, limit, timeout, replace_all, include) lack descriptions. LLMs cannot infer their purpose, ranges, or defaults. E.g., does limit=500 mean 500 lines or 500 bytes? What is the valid range?
No error handling guidance. Tools do not document what errors are retryable, user-fixable, or fatal. E.g., if read_file fails with 'permission denied', should the agent retry, ask the user, or abort? No recovery paths documented.