Loading ToolBench
A pluggable Model Context Protocol (MCP) server and headless CLI for the Open Knowledge Format (OKF).
okf-vault demonstrates solid definition quality with comprehensive tool coverage for a knowledge management domain. All 12 tools have clear, well-written descriptions (194-350 chars, exceeding the 194-char baseline average). Parameter schemas are present and properly typed with JSON Schema. However, some parameter descriptions are generic, and tool output schemas are not explicitly documented, they are inferred from implementations. The server follows good composition patterns (each tool has a single responsibility) and includes helpful system directives to guide agent behavior. No tool registration is visible in the provided source, so per-tool scores are capped at 50 as a conservative penalty for inference.
Create a top-level knowledge bundle (a namespace for related concepts). Note: writing a concept with okf_concept_upsert auto-creates intermediate directories, so you rarely need this unless seeding a brand-new root namespace.
DESTRUCTIVE. Soft-delete a bundle and ALL of its concepts. Avoid in the append/update memory model — prefer updating concepts via okf_concept_upsert. Use only on explicit request.
List the directory structure of a bundle. Use this when you need to understand the hierarchy of stored concepts before searching. Returns a tree of titles/links, not bodies.
List all top-level knowledge bundles (slug, title, description). Use this to discover which bundles exist before indexing or searching.
Update the title and description of an existing bundle.
Find all concepts that this one LINKS TO (outbound references). Extract okf:// URIs from 'body'. Useful for understanding dependencies and related knowledge.
Tool output schemas are not explicitly documented. While parameter schemas are well-defined, return types are inferred from code rather than formally declared. LLMs cannot parse code, they rely on explicit schema documentation to understand what fields to expect and plan downstream tool calls.
Some parameter descriptions are generic or lack implementation details. For example, 'offset' parameters describe pagination mechanics but do not state the max limit the LLM should assume. Descriptions should include constraints (min/max, format expectations) to prevent the LLM from passing invalid values.
Tool registration is not visible in the provided source code. Tool definitions are inferred from descriptions and type signatures in okf-vault.ts, but the explicit call to server.tool() or equivalent MCP registration is not shown. This prevents verification of schema JSON Schema compliance and makes scoring conservative.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 69 | <=2025-11-25 | v2 |
DESTRUCTIVE. Soft-delete a concept. Prefer okf_concept_upsert in the append/update memory model. Use only on explicit request.
Read a concept's full content (frontmatter + body markdown). Use the 'link' from okf_concept_search as input, or provide both 'bundle' and 'path'.
Retrieve the version history of a concept, showing all past snapshots with timestamps. Use this to audit changes or revert to an earlier state.
Find all concepts that LINK TO this one (inbound references). Reverse search from 'body' okf:// URIs. Useful for impact analysis and finding related concepts.
Search long-term memory for past context, user preferences, or system states. ALWAYS use this tool first when a user asks a question about past interactions, established projects, or personal preferences before answering. Returns a lightweight list of matches (link, title, summary snippet) — NOT full documents. Pick the most relevant 'link' and call okf_concept_get to read its full content.
Create a new concept or update an existing one in place (with automatic version snapshot). Intermediary directories are auto-created. Use this to consolidate architectural decisions, user preferences, and long-term memory.
Error handling is partially visible (toToolError helper maps domain errors to ToolErrorEnvelope), but error guidance is incomplete. Some recovery hints are present in descriptions (e.g., 'call okf_concept_search first'), but not all failure scenarios include actionable next steps. SearchQuerySyntaxError handling shows good error classification, but other error types lack explicit recovery guidance.
Destructive operations (okf_bundle_delete, okf_concept_delete) accept immediate execution without a confirmation or dry-run step. Descriptions warn agents to 'use only on explicit request', but no tool-level confirmation mechanism prevents accidental destructive calls if the agent misinterprets a user request.