Loading ToolBench
12 tools with complete input schemas and descriptions, but quality is uneven. Naming follows verb_noun convention (git_*, lint_*, repo_*, shell_*, worktree_*). Descriptions are present but brief (avg ~60 chars, below 194-char baseline). Parameters have types and descriptions, but lack constraints (enums, ranges, patterns). No documented output schemas. Error handling is minimal, no recovery guidance or categorization. Security concerns: shell_run accepts arbitrary commands with allowlist/denylist validation, but no clear permission model. Worktree tools are destructive but lack confirmation patterns. Overall: functional but below production baseline.
Get git diff output for changed files
Get list of changed files with optional path filtering
Get git status with porcelain format showing file changes
Detect available linters in the repository based on project files and installed binaries
Run a linter with optional command override and allowlist/denylist validation
Search files in repository using regex pattern with optional glob and path filtering
List files in repository with optional glob pattern and path filtering
Execute shell commands with allowlist/denylist validation and path scope checking
No output schemas documented. LLMs cannot predict response structure, forcing them to guess field names for downstream tool chaining. E.g., git_status returns porcelain format but no schema specifies field names or types.
Destructive tools (worktree_remove, shell_run) lack confirmation/dry-run patterns. Agents can delete branches or execute arbitrary commands without a safety gate. No recovery guidance in error responses.
shell_run accepts arbitrary commands with allowlist/denylist validation, but no permission model or audit trail. Agents can execute any command not explicitly blocked. DEFAULT_BLOCKED_COMMANDS is hardcoded and insufficient.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Create an isolated git worktree and branch for a task with optional setup command
List all git worktrees in the repository with their branches and status
Plan and optionally create worktrees for all Workers defined in ownership.json
Remove a git worktree and optionally delete its associated branch
Descriptions are too brief (avg ~60 chars vs 194-char baseline). E.g., 'Get git status with porcelain format showing file changes' lacks context on when to use it vs git_diff or git_name_only. No dependency hints or prerequisites stated.
Parameters lack constraints. E.g., shell_run timeout is integer with no min/max (default 120s). repo_index_grep pattern is regex with no length limit or validation. LLMs can pass absurd values (timeout=999999, pattern='.*' on huge repos).