Loading ToolBench
An AI-powered code assistant that uses MCP (Model Context Protocol) to execute tools for reading files, writing files, running commands, searching code, and other development tasks. Supports both local execution and delegation to editor capabilities via ACP (Agent Communication Protocol).
GPTCode provides 10 tools with complete input schemas and descriptions, but quality is uneven. Tool names follow verb_noun convention (read_file, write_file, search_code). Descriptions are present but often generic (e.g., 'Read the contents of a file' lacks context on when to use vs. list_files). Parameters have types and descriptions, but lack constraints (enums, ranges, patterns). Output schemas are not documented, LLMs cannot predict return structure. Error handling is minimal; no recovery guidance. Security is weak: run_command accepts arbitrary shell input with no sanitization or permission checks, creating command injection risk. Composition is reasonable (single responsibility per tool), but tools like apply_patch lack idempotency guarantees.
Replace a block of text in a file
Find files most relevant to a task by searching for keywords. Use this FIRST before browsing directories. Returns ranked list of files containing task-related keywords.
List files in a directory of the current repository
Get a tree-like view of the project structure
Read the contents of a file in the current repository
Read detailed coding guidelines from ~/.gptcode/guidelines/ directory. Use when you need language-specific guidance, naming conventions, or TDD workflow details.
Execute a shell command in the repository directory (use for tests, linting, etc.)
run_command accepts arbitrary shell input with no sanitization, validation, or permission checks. LLMs can be tricked into executing destructive commands (rm -rf /, git push --force). No dry-run or confirmation step for irreversible operations.
Output schemas are not documented for any tool. LLMs cannot predict return structure, forcing them to guess field names and types. This breaks tool chaining and increases hallucination risk.
Descriptions lack context on WHEN to use each tool. 'Read the contents of a file' does not explain when to call read_file vs. list_files vs. find_relevant_files. LLMs waste reasoning cycles deciding between similar tools.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | <=2025-11-25 | v2 |
Search for a pattern in code files using grep
Search the web for information, documentation, or answers. Use when you need to look up error messages, API docs, or general knowledge not in the codebase.
Write content to a file (creates or overwrites). Use this to save edited files.
apply_patch uses exact text matching for search/replace, which is fragile and non-idempotent. If whitespace or formatting changes, the patch fails silently. No guidance on recovery.
Parameters lack constraints. run_command has no max length, timeout, or allowed command whitelist. search_code pattern parameter accepts any regex with no validation. LLMs can pass invalid or expensive queries.