Loading ToolBench
8 tools evaluated. All have names starting with action verbs (git_*) and descriptions present. However, descriptions are terse (10-60 chars, well below the 50-200 char LLM-optimized baseline). Input schemas visible for all tools with proper Zod validation, but parameter descriptions are minimal and lack constraint details (e.g., no guidance on branch name format, commit hash format, or valid ranges). No output schemas documented. Error handling not visible in provided code samples. Tool composition is sound (each does one thing), but descriptions do not explain WHEN to use each tool vs. similar ones (e.g., git_merge vs git_rebase distinction unclear from descriptions alone). No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels assigned (READ_ONLY, WRITE, DESTRUCTIVE).
Switch to a different branch.
Apply commit(s) from another branch to current branch.
Create a new branch, optionally from a specific commit or ref.
Delete a local branch. Use force to delete unmerged branches.
List git branches. Can show local, remote, or all branches.
Merge a branch into the current branch.
Move uncommitted changes to a new branch. Stashes changes, creates branch, and applies them.
Descriptions are terse (10-60 chars) and lack LLM-optimized guidance. 'Switch to a different branch' does not explain when to use git_checkout vs git_create_branch, or what happens if the branch doesn't exist. Baseline for A-grade tools is 50-200 chars with WHEN/WHY context.
No output schemas documented. LLMs cannot plan downstream calls or extract required fields (e.g., does git_list_branches return branch names, commit hashes, or both?). All tools lack return type documentation.
Tool annotations missing despite clear risk levels assigned (READ_ONLY, WRITE, DESTRUCTIVE). git_delete_branch is destructive but has no destructiveHint annotation. Agents cannot reason about safety without explicit hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
Rebase current branch onto another branch. Can also abort or continue a rebase.
Parameter descriptions lack constraint details. 'branch' param has no guidance on valid format (alphanumeric, hyphens allowed?). 'commit' param does not specify hash length or format. LLMs cannot validate inputs without explicit constraints.
Error handling not visible in provided code. No recovery guidance documented (e.g., 'If branch not found, call git_list_branches to see available branches'). Agents cannot self-correct on failures.