Loading ToolBench
Forge, run, and orchestrate sandboxed LLM agents with native tools (bash, file operations, grep, glob)
Agent Forge defines 6 tools with explicit schemas and descriptions. All tools have input schemas with typed parameters and descriptions. However, descriptions are terse (10-50 chars), falling below the 50-200 char baseline for LLM-optimized docs. Output schemas are not documented. Error handling is minimal, no recovery guidance or actionable error messages. Tool names are clear (bash, write, read, edit, grep, glob) but lack verb-noun convention (e.g., 'execute_bash' would be clearer). No tool annotations (readOnlyHint, destructiveHint) despite clear risk profiles. Parameter descriptions exist but lack format constraints, ranges, and examples. The 'timeoutMs' parameter on bash lacks min/max bounds.
Execute a shell command via bash -lc inside /workspace
Patch a file by exact substring replacement
List files matching a glob pattern
Regex search across files matching a glob pattern
Read a file with optional line-based offset and limit
Create or overwrite a file at a path relative to /workspace
Descriptions are too terse (10-50 chars vs. 50-200 char baseline). LLMs cannot determine when to select tools or how they differ. E.g., 'Execute a shell command via bash -lc inside /workspace' lacks context on when to use bash vs. other tools, what it returns, and error recovery.
No output schemas documented. LLMs cannot plan downstream tool calls or extract required fields. E.g., bash result structure (stdout, stderr, exit code, duration) is not declared.
No tool annotations despite clear risk profiles. bash and write are destructive/irreversible; read, grep, glob are read-only. Missing readOnlyHint and destructiveHint annotations prevent agents from reasoning about safety and retry semantics.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
Parameter constraints missing. 'timeoutMs' on bash lacks min/max bounds (code shows default 30000, cap 120000, but description does not state this). 'limit' on read defaults to 200, max 2000, but not documented in schema. LLMs cannot validate inputs.
No error handling guidance. Tools do not document what errors are retryable, user-fixable, or fatal. No recovery suggestions (e.g., 'If file not found, use glob() to list available files'). Agents cannot self-correct.