Loading ToolBench
The server defines 9 Git tools with reasonable naming conventions (verb_noun structure: repo/refs, repo/clone, etc.). All tools have descriptions and input schemas are visible in the code. However, several definition quality gaps prevent a higher score: (1) Input schemas lack depth, parameters have type and description but some lack constraints (enums, min/max); (2) Output schemas are undocumented, the code shows ToolDefinition with input_schema but no explicit output schema documentation in responses; (3) Parameter descriptions are terse (many under 100 chars); (4) Error handling lacks recovery guidance (no 'try this next' hints in error messages); (5) Some tool names are compound without clear justification (repo/clone/start + repo/clone/chunk + repo/clone/status + repo/clone/cancel is a 4-part stateful operation that could be better surfaced); (6) No tool annotations (readOnlyHint, destructiveHint) visible despite clear risk levels being assigned. The server sits in the 'Fair' band due to present but incomplete schemas and generic error handling.
Clone a Git repository (non-streaming, full repository tar.gz archive)
Cancel an ongoing clone operation
Retrieve a chunk of data from an ongoing clone operation
Initiate a Git repository clone operation
Get the status of an ongoing clone operation
Show differences between revisions in a Git repository
Pull changes from a remote Git repository
Push commits to a remote Git repository
Output schemas undocumented. Tool definitions show input_schema in ToolDefinition struct but no explicit documented output schema. Clients cannot plan downstream calls or extract structured data without knowing what fields are returned.
Parameter descriptions are terse and lack actionable constraints. Examples: 'URL of the Git repository' for repo_url appears 6 times but never specifies format (http://, ssh://, https://), supported protocols, or what to do if invalid. Parameter 'chunk_index' in repo/clone/chunk lacks guidance on valid range or how chunks are numbered.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 56 | - | v1 |
List references (branches, tags) in a Git repository
No tool annotations despite clear risk assignments. Tools marked DESTRUCTIVE (repo/push) and WRITE (repo/clone, repo/pull) lack explicit tool annotations (destructiveHint, readOnlyHint). Clients cannot determine safety/reversibility from definitions alone.
Stateful multi-step operation (repo/clone/start + chunk + status + cancel) lacks clear lifecycle documentation. Tool descriptions do not explain: What state is maintained? How long is a session valid? Can a chunk be retrieved twice? What happens if cancel is called mid-transfer? This forces clients to guess the semantics.
repo/clone accepts optional 'depth' parameter but description does not explain: What is the default? Is depth strictly positive? What does depth mean for non-git protocols? Does it interact with submodule handling?
repo/push allows 'force' parameter but description says 'subject to security policy' without explaining what that policy is or when force is rejected. LLM cannot determine if force is safe to use.
No recovery guidance in error cases. Tool descriptions do not indicate: What should LLM do if clone fails due to auth? If pull fails due to merge conflict, what next? If push is rejected, what are the options? Agents cannot self-correct without recovery hints.