Loading ToolBench
A Go-based MCP server that provides agent tools for workspace file operations, git integration, and command execution within a supervised environment
StudioForge provides 13 tools with complete JSON schemas and descriptions, but quality is uneven. All tools have descriptions (10-100 chars, mostly adequate) and input schemas with type definitions. However, descriptions are often terse (e.g., 'Show short git status with branch info' for git_status), lacking context on WHEN to use each tool or what downstream tools expect. Parameter descriptions are present but minimal. Output schemas are not documented, LLMs cannot predict what fields to expect from responses. Error handling is basic; no recovery guidance. Tool composition is reasonable (file ops, git ops, command execution), but no pagination support for list_dir or grep despite potentially large results. No tool annotations (readOnlyHint, destructiveHint) despite clear risk stratification (READ_ONLY vs WRITE vs IRREVERSIBLE). Overall: functional but below production baseline.
Apply a set of exact-text replacements across one or more files, all or nothing.
Create a directory (and any missing parents) within the project.
Create or overwrite a file with the given content, creating parent directories as needed.
Show unstaged changes as a unified diff, falling back to the diff against HEAD.
Show short git status with branch info for the project.
Search text files for lines matching a regular expression.
List the entries of a directory within the project (name, kind, size).
No output schemas documented. LLMs cannot predict response structure (fields, types, pagination). Forces agents to guess what data is available after each call.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Tools are marked READ_ONLY/WRITE/IRREVERSIBLE in metadata but not in schema. LLMs cannot infer safety properties from tool definitions alone.
Descriptions lack context on WHEN to use each tool. E.g., 'Show short git status' does not explain when to call git_status vs git_diff vs report_changed_files. LLMs waste reasoning cycles choosing between similar tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Read a text file's contents, capped at a maximum size.
Read an inclusive 1-based line range from a text file.
Replace the unique occurrence of old_text with new_text in a file.
List files changed in the working tree via git status.
Run a supervised command inside the project workspace.
Find files whose name or relative path matches a glob pattern.
No pagination support for potentially large results. list_dir and grep can return hundreds of items but lack limit/offset/cursor parameters. Large results blow context window.
run_command lacks error recovery guidance. No distinction between retryable (timeout) vs fatal (permission denied) vs user-fixable (invalid syntax) errors. LLMs cannot self-correct.