Loading ToolBench
Seeds, governs and navigates a memory_bank knowledge base: gated writes, routing without reading, drift against the code
Strong tool definitions with comprehensive descriptions and well-structured schemas. All 13 tools have clear, detailed descriptions (avg 180+ chars) explaining WHAT, WHEN, and WHY. Input schemas use Zod with proper types and constraints. Tool names follow verb_noun pattern (bank_route, bank_read, bank_create). Annotations present (readOnlyHint, idempotentHint). Key gaps: output schemas not formally documented in code; error handling guidance minimal; some parameter descriptions could be more prescriptive about constraints and formats.
What moved since a point in time. A git ref gives a true delta including deletions and renames; an ISO date falls back to modification times, which can only report what exists now. Returns changes grouped by kind (added, modified, deleted, renamed).
Creates a document from a template, writes the frontmatter the contract asks for, and registers it in the section index — one operation, so the registration step cannot be forgotten. The template comes from the bank's own flows/templates/ when it has one and from the set the server ships when it does not. If the section itself does not exist yet, it is created along with its index and its entry in the root index, and a warning says so. Refuses to write when the path is taken, when a derived_from target does not resolve, or when canonical_for is already owned by another document.
Finds documents whose frontmatter claims something the code does not support. Checks doc_kind, doc_function, status, delivery_status, and decision_status against the contract. Returns findings grouped by severity.
Edits a document in place. Can update the title, purpose, status, or a single section body. Refuses to edit when the path does not resolve. Section edits replace the entire section; use bank_read to see what is there first.
Traces the derived_from graph starting from one document. Returns the paths and titles of all documents reachable in the given direction (up to parents, down to children, or both). Depth limits how many steps to walk; the default is unlimited.
Output schemas not formally documented. Tool descriptions explain what is returned (e.g., 'Returns parsed frontmatter alongside the body'), but JSON Schema output types are not visible in code. LLMs cannot reliably plan downstream calls without knowing exact response field names and types.
Error handling guidance absent. Tools return errors (e.g., 'path does not resolve', 'canonical_for already owned') but do not guide LLM recovery. Missing actionable error messages like 'Try bank_search() to find the document' or 'Check bank_validate() for conflicts'.
Parameter constraints under-specified. 'docKind' and 'status' accept free-form strings with no enum or pattern. Descriptions mention examples ('e.g. adr') but do not enumerate valid values. LLMs will hallucinate invalid values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
Lists documents waiting in _inbox/, with their age, kind, title, and purpose. Returns entries sorted by age (oldest first).
Seeds an empty root: the governance set in dna/ (including the table that assigns each directory its layer), the four flow documents, a registered index for each of the eight sections, and draft stubs for the two documents the flow refers to — 24 files. Three further sections (epics, prd, prompts) are not created in advance; bank_create builds one the first time a document needs it. The document templates are not copied either: they ship with the server and bank_create reads them from there, so nothing is missing without them. Everything written becomes the bank's own — the starter is a starting point, not a schema the server keeps enforcing. A bank created this way validates clean; refuses to seed over existing documents unless force is passed.
Copies the flow templates from the server into flows/templates/, so the bank owns them and bank_create prefers them over the ones the server ships. Refuses to overwrite existing files unless force is passed.
Moves a document from _inbox/ to its final location, optionally updating its frontmatter. The document must exist in _inbox/; the destination must not. Registers the document in the section index and removes it from the inbox index. Refuses to promote when a derived_from target does not resolve or when canonical_for is already owned by another document.
Reads one document, or several in a single call, by bank-relative path — optionally a single level-two section of each. Pass an array whenever you need more than one: reading a handful of documents is one call, not one call each, and there is never a reason to shell out to cat instead. Section reads matter: canonical documents in real banks reach 190 KB. Returns parsed frontmatter alongside the body. A path that does not resolve is reported under failed without sinking the rest of the batch. A batch has a byte budget, spent in the order asked; whatever it does not reach comes back under skipped with its section list, so ask for those by section rather than repeating the whole read. Ten whole documents do not fit in one answer, and asking for them anyway returns a file path instead of the text.
Answers "what should I read about this". Ranks documents by their hand-written frontmatter (canonical_for, purpose, title, section headings) — not by body text — and returns paths with the reason each matched. Templates are never returned. Ask in whatever language the user did — bank documents are English, and the server translates the query rather than making you do it. Use this instead of reading README.md and walking section indexes.
Finds documents by literal term, not by frontmatter intent. Searches the body text of every document and returns paths ranked by match count and position. Ignores function words and expands Russian queries to English. Returns at most 25 results.
Checks the bank against its own contract and against structural rules. Returns findings grouped by severity and rule. Scope can narrow the check to one directory; rule can narrow it to one validation rule. A bank with no contract validates in degraded mode, checking only structural rules.
Batch operation limits not enforced or documented. bank_read accepts arrays but description warns 'Ten whole documents do not fit in one answer', no explicit max_items constraint in schema, no clear byte budget enforcement visible.
Dry-run pattern inconsistently applied. bank_create, bank_promote, and bank_edit support dryRun, but bank_init and bank_materialize_flows do not. Agents cannot safely preview destructive operations uniformly.