Loading ToolBench
Agent orchestration platform — planner, registry, runtime, verifier. Exposes MCP tools for code analysis, file operations, memory recall, and governed MCP tool bridging.
Spine provides 10 well-named, verb-prefixed tools with clear descriptions (avg 156 chars, within 10-1024 baseline). All tools have input schemas with required fields properly declared. However, parameter descriptions are sparse or missing entirely, most params lack guidance on format, constraints, or valid values. Output schemas are undocumented; LLMs cannot predict response structure. Error handling is absent from descriptions. Tools like pkg_relevant_symbols and pkg_api_surface lack parameter descriptions explaining what 'query' means or how to formulate it. write_files has complex nested schema but no guidance on edit semantics. No tool declares permissions, audit requirements, or idempotency guarantees.
List the repository's files (relative paths) so you can decide what to read. Optional 'subdir' narrows the listing.
List the repo's relevant public API surface for a query, with file:line.
What breaks if you change a symbol — its transitive callers (impact set) with file:line and hop distance. Use before modifying shared code.
List the callers of a symbol by name, with file:line provenance.
Find existing code symbols (types/functions) relevant to a query, with file:line.
Read a file's contents by path relative to the worktree root.
Recall facts learned from past runs on this project — conventions, pitfalls, decisions, fix-patterns — before deciding how to implement something. Optional 'kind' filters to one of: convention, pitfall, decision, fix-pattern.
Parameter descriptions missing or incomplete. 'query' param in pkg_relevant_symbols, pkg_api_surface lacks guidance on format, scope, or example queries. 'symbol' in pkg_callers_of and pkg_blast_radius undescribed. LLMs cannot infer what constitutes a valid query or symbol name.
Output schemas completely undocumented. No tool declares what fields it returns, data types, or structure. LLMs cannot plan downstream calls or extract required IDs for chaining. E.g., pkg_relevant_symbols returns 'file:line' but response format is opaque.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
Run the worktree's tests and return pass/fail with captured output.
Finish: submit the changes written so far with a one-line summary. Ends the task.
Create new files (content) or edit existing ones (anchored find/replace). Returns what landed or a guard error to fix.
write_files has complex nested schema (files array with edits array) but no description of edit semantics, anchor matching rules, or error recovery. LLMs cannot reliably use anchored find/replace without explicit guidance on multi-line matching, whitespace handling, or partial-match behavior.
No error handling guidance. Tools lack descriptions of failure modes, recovery steps, or actionable error messages. E.g., what happens if read_file path does not exist? If pkg_relevant_symbols query matches nothing? Agents have no recovery path.
No permission or audit declarations. Tools like write_files and submit_changes are destructive but lack scope declarations (e.g., 'write:code', 'submit:changes'). No audit trail guidance. Agents cannot reason about least-privilege or compliance.