Loading ToolBench
Self-contained, local-first semantic memory system with vector embeddings, chunking, entity relations, and semantic search
VKB demonstrates solid tool design with 17 well-named, verb-prefixed tools (vkb_ingest, vkb_query, vkb_delete, etc.). All tools have descriptions (avg ~180 chars, within 10-1024 baseline). Input schemas are present and mostly complete with type definitions and parameter descriptions. However, output schemas are NOT documented, callers cannot see what fields to expect from responses. Error handling is minimal; tools lack recovery guidance and actionable error messages. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels (WRITE, DESTRUCTIVE, READ_ONLY). Parameter descriptions are good but some lack format/constraint details (e.g., threshold range 0-1 is stated but not in all contexts). Composition is strong, tools chain well (query returns entity_id, get accepts entity_id). Naming follows verb_noun convention consistently.
Delete an entity and all its derived artifacts (chunks, sections, relations). Permanent.
Extract LLM-inferred relations and enrich entity metadata with tags. Runs in background.
Fetch entity or chunk by ID. Includes summaries, chunk list, sections, and all relations. Entity responses also include tag_context: co-tagged entities (entity_id, type, summary, tags, shared_tags) so callers can pull related context without extra round-trips.
Submit raw text or a URL/file path for ingestion. Returns job_id and entity_id immediately. Pipeline runs in background.
Submit multiple items for ingestion in a single call. Returns a summary with entity_id and job_id for each item. Identical content is automatically deduplicated (returns skipped=true for unchanged items). Prefer this over repeated vkb_ingest calls when ingesting a website crawl, file listing, or any batch of documents. All jobs run concurrently in the background — poll individual job_ids with vkb_job.
Output schemas not documented. Tools return complex objects (entity context, relations, summaries, tag_context) but callers cannot see field names, types, or structure. LLMs cannot plan downstream calls or extract data reliably.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Tools are marked with Risk levels (READ_ONLY, WRITE, DESTRUCTIVE) in comments but not exposed to MCP clients. Agents cannot distinguish safe from destructive operations.
Error handling lacks recovery guidance. No tool documents what errors can occur, whether they are retryable, or what the LLM should do next. E.g., vkb_query warns about empty results but does not structure error responses.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | <=2025-11-25 | v2 |
Ingest an OKF (Open Knowledge Format) bundle directory into vkb. Walks all .md files in the bundle, parses OKF frontmatter (type, title, description, resource, tags, timestamp), bulk-ingests every concept document, then asserts all OKF cross-links as permanent asserted relations. Cross-link relation types are derived from the nearest section heading (e.g. a link under "# Joins" -> "okf:joins"). Re-ingesting the same bundle is safe: unchanged files are deduplicated by content hash.
Poll a job by ID. Returns current pipeline stage, progress counters, and error detail if failed.
Lint an entity for structural issues: missing summaries, orphaned chunks, broken relations, stale embeddings. Returns array of findings.
Bulk lint: scan all entities and return aggregated findings grouped by code.
Run pending database migrations. Idempotent.
Fetch all relations for an entity or chunk (both incoming and outgoing). Returns paginated results with relation metadata.
Semantic search over the knowledge base. Returns top-k chunks with summaries, entity context, similarity scores, and relations. IMPORTANT — empty results do NOT mean the knowledge base lacks relevant content. An empty result set most likely means the similarity threshold is too high for the current query. When results is empty: retry with a progressively lower threshold (e.g. 0.5, then 0.3, then 0.1) before concluding no relevant content exists. The response includes a hint field when results is empty that suggests the next threshold to try. Default threshold is controlled by RELATION_THRESHOLD in config (typically 0.7–0.75). Many valid matches score in the 0.3–0.6 range.
Fetch raw text for an entity or chunk from the RawStore (L0 — filesystem read).
Re-ingest an entity from its preserved rawstore content. Clears all derived artifacts and re-runs the full pipeline.
Assert an explicit relation between any two entity or chunk IDs. Never pruned.
Re-embed stale chunks, re-weight relations, prune low-scoring relations, and re-summarize entities. Runs in background.
Get server status: database connection, worker pool health, job queue depth, and entity/chunk counts.
Destructive operations (vkb_delete, vkb_reingest) lack confirmation/dry-run support. Agents can permanently delete entities or re-ingest without a safety gate. No pattern:confirmation-request implemented.
Parameter descriptions lack format/constraint details in some tools. E.g., vkb_relate 'weight' accepts 0-1 but description does not state range. vkb_ingest 'meta' is 'arbitrary key-value' with no guidance on valid keys or structure.