Loading ToolBench
Lean AEE governance for requirements, linked tests, epistemic context, evidence, and AI-assisted changes.
Specsmith provides 13 tools with complete input schemas and descriptions. Naming follows verb_noun convention (run_shell, read_file, write_file, patch_file, list_files, grep, git_diff, git_status, run_tests, open_url, search_docs, remember_project_fact, update_todo_list). All tools have descriptions (10-200 chars, within baseline 10-1024 range). Input parameters are typed and described. However, output schemas are not documented, LLMs cannot predict response structure for downstream chaining. Error handling is minimal; tools lack recovery guidance. Destructive tools (write_file, patch_file, run_shell) have no confirmation or dry-run patterns. Parameter descriptions are functional but lack constraint details (e.g., timeout range, glob pattern syntax, diff format specifics). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present despite clear risk categories.
Get git diff for the current repo.
Get git status for the current repo.
Search for a string in files.
List files in a directory matching a glob pattern.
Fetch contents from a URL.
Patch a file using standard diff/patch. Validates diff format before applying. Rejects diffs with `:start_line:` / `:end_line:` markers in the REPLACE section (common model-generation artifact). Suggests `write_to_file` as fallback for large edits.
Read contents of a file.
Output schemas not documented. LLMs cannot predict response structure (fields, types, pagination) for downstream tool chaining. Baseline: 100% of A+ tools document return types.
Destructive tools (write_file, patch_file, run_shell) lack confirmation or dry-run patterns. Agents can cause irreversible data loss without safeguards.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk categories. Clients cannot distinguish safe from dangerous tools without parsing descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
Store a fact about the project in the local context index.
Run a shell command and return its output.
Run tests.
Search documentation within the repo.
Render a readable, mode-independent nested todo list.
Write content to a file (creates or overwrites).
Parameter descriptions lack constraint details. E.g., timeout range, glob pattern syntax, diff format validation rules not specified. Baseline: descriptions should state format, range, and allowed values.
Error handling minimal. Tools lack recovery guidance (e.g., 'file not found, try list_files() first'). Baseline: error responses must tell LLM what to do next.