Loading ToolBench
A git-backed knowledge base server with read and manage modes. Serves OKF (Open Knowledge Format) markdown wikis with version control, supporting both read-only access to frozen commits and full editing with snapshot commits.
16 tools with clear verb-noun naming (kb_get, kb_add, kb_remove, etc.) and consistent descriptions (avg ~120 chars). All tools have input schemas with type and description fields. However, output schemas are entirely undocumented, no tool specifies what fields are returned, forcing LLMs to infer structure. Parameter descriptions are present but often terse (e.g., 'folder path relative to the repo root' lacks format/constraint details). Error handling is absent, no guidance on recovery, retryability, or actionable error messages. No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels. Missing pagination guidance for list tools (kb_list, kb_grep could return large result sets). Composition is sound, tools chain logically (kb_topics → kb_select_topic → kb_get). Security: no secrets in params, but no audit trail or permission gate documentation.
Create a new page in the working tree with frontmatter and body. Rebuilds the folder's index.md and validates the result. Manage mode only.
Fetch the full text of a page (markdown with frontmatter).
Search for a query string across all .md files in the knowledge base. Returns matching lines with file paths and line numbers.
Return the index.md (catalog) of a folder, listing its contents with descriptions.
List all .md files in a folder (non-recursive), with their types and one-line descriptions.
Create a new subfolder in the working tree. Manage mode only.
Rebuild all index.md files in the knowledge base (catalog generation). Manage mode only.
No output schemas documented for any tool. LLMs cannot infer what fields are returned (e.g., does kb_list return {files: [...]} or {items: [...], total: N}?). Forces LLMs to guess structure and risks parsing failures.
No error handling guidance. Tools like kb_remove (destructive) and kb_snapshot (git commit) provide no recovery hints, retryability classification, or actionable error messages. An LLM hitting 'git error' has no path forward.
No tool annotations despite clear risk levels. kb_remove is destructive, kb_snapshot is irreversible, kb_get is read-only, but no readOnlyHint or destructiveHint metadata. Agents cannot reason about side effects.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Delete a page from the working tree. Rebuilds the folder's index.md and validates. Manage mode only.
Return the knowledge manager rules (AGENT_RULES.md), which govern all manage-mode operations.
Activate a topic by name (switch the active knowledge base). Only available in multi-topic mode. After calling this, all subsequent read/write operations target the selected topic.
Switch the active branch (arm) to a different ref. Checks out the branch and updates the working tree. Manage mode only.
Commit the current working tree changes to the active branch with a message. Creates an immutable point in the git history. Manage mode only.
List all available topics (knowledge bases) in the registry, with one-line descriptions. Only available in multi-topic mode (registry of topics, no --topic pin).
Edit an existing page's frontmatter and/or body. Rebuilds the folder's index.md and validates. Manage mode only.
Validate the working tree against OKF v0.1 rules: frontmatter structure, reserved filenames, folder layout, type/tag consistency, etc. Returns a detailed report. Manage mode only.
List all git branches (arms) in the repo, with their latest commit messages and timestamps. Manage mode only.
List tools (kb_list, kb_grep) lack pagination parameters (limit, offset, cursor). A grep across a large knowledge base could return thousands of lines, exhausting context. No guidance on result limits.
Parameter descriptions are terse and lack format/constraint details. E.g., 'folder path relative to the repo root' does not specify allowed characters, max length, or whether '.' and '..' are valid. LLMs may pass invalid paths.