Loading ToolBench
A personal, long-lived knowledge library with layered knowledge graph, wiki workspace, memory layer, and agent-driven curation. Exposes tools via MCP server for external clients.
BYO-LLM-WIKI exposes 27 tools with mostly complete descriptions and schemas. Naming follows verb_noun conventions (kg_search, memory_recall, skill_build). However, significant gaps exist: (1) Output schemas are undocumented, tools return JSON but the response structure is not formally specified, forcing LLMs to infer field names; (2) Error handling is minimal, no recovery guidance or actionable error messages visible; (3) Parameter descriptions lack constraints (e.g., kg_search 'limit' has no min/max stated); (4) Some tools are duplicated (kg_search, kg_get_entity, kg_stats appear twice with minor description variations); (5) Write tools (kg_upsert_entity, skill_build) lack confirmation/dry-run patterns despite modifying state. Tool descriptions average ~150 chars, within baseline (194 chars p50), but many lack WHEN-to-use context.
Add a typed, directed relationship between two entities.
Assign an entity to a topic (creates the topic if it does not exist yet).
Get the full evidence text of a chunk and its source, for grounding/citation.
Get a full entity profile: kind, aliases, summary, relations, and citing chunks.
Get a full entity profile: kind, aliases, summary, topic, typed relations, and the chunks that mention it.
List entities, most important first, optionally filtered by topic or kind.
List all topics (themes) with their parent and how many entities each contains.
Output schemas undocumented. Tools return JSON (via _dump()) but response field structure is not formally specified. LLMs cannot plan downstream tool calls or extract fields reliably.
Duplicate tool definitions. kg_search, kg_get_entity, kg_stats, skill_recall, skill_get, skill_list appear twice with minor description variations. LLMs waste reasoning cycles deciding between near-identical tools.
Parameter constraints missing. kg_search 'limit' (max 50), kg_neighbors 'depth' (max 3), kg_list_entities 'limit' (max 200) have bounds in code but not stated in parameter descriptions. LLMs may pass invalid values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
Merge a duplicate entity into the canonical one, re-pointing all edges. Use when two entities are the same thing (e.g. 'GPT-4' and 'GPT 4'). The dropped name is preserved as an alias of the kept entity.
Explore the subgraph around a node up to a given hop depth (for multi-hop reasoning).
Search the knowledge graph for nodes (entities/chunks/topics/...) matching a query.
Search the library for nodes whose text/name matches a query.
Set or replace the one-line canonical summary for an entity.
Get counts of every layer in the knowledge library. Returns counts of sources, chunks, entities, topics, syntheses and edges. Call this first to understand the size and shape of the library.
Counts of every layer in the knowledge library.
Create an entity or reinforce/enrich an existing one (idempotent). Use this to add a canonical concept/person/etc. or to attach a one-line summary. Prefer reusing an existing canonical name over creating near-duplicates.
Create or update a topic (theme) node, optionally nested under a parent topic. Topics are the hierarchical layer that groups related entities into coherent themes. Keep the count of topics small and meaningful.
Recall what the library already learned that is relevant to a query. Call this BEFORE answering or maintaining so you build on prior sessions (known facts, durable answers, user preferences, recorded gaps, corrections) instead of starting from scratch or repeating work.
Recall durable cross-session memories (facts, answers, preferences, gaps).
Record a durable memory so the library remembers it next session. Use for things worth carrying forward: a confirmed fact, a durable answer, a user goal/preference, a known gap to fill, or a correction. Near-duplicate memories are reinforced rather than duplicated, so it is safe to re-assert.
Answer a question grounded in the contextual RAG library, with citations. Requires the vector library to be built and OPENAI_API_KEY for embeddings.
Draft + evaluate a NEW agent skill from context (KG query or pasted text). This is a *proposing* action: the skill is gated to pending_review and is NOT auto-accepted — a human approves it in the BYO-WIKI review queue. Returns the gate decision and the new skill id.
Get a skill's full instructions/steps/triggers (after skill_recall).
Get the full instructions of an agent skill (its steps, triggers, and tools). Use after skill_recall to read the skill you intend to follow.
List accepted agent skills in the library (name + description).
List the accepted, ready-to-use agent skills in the library (name + description).
Find accepted agent skills whose purpose matches a task you are about to do. Call this when a request looks like a repeatable task — there may be a proven, human-approved skill that already knows how to do it. Returns the best-matching skills' names, descriptions, triggers and tools.
Find accepted, ready-to-use agent skills matching a task description.
Write tools lack confirmation/dry-run patterns. skill_build, kg_upsert_entity, kg_merge_entities modify state but offer no preview or confirmation step. Agents cannot safely test before committing.
Error handling minimal. Tools return JSON errors (e.g., {"error": "No entity matching..."}) but no recovery guidance. LLMs cannot determine if error is retryable, user-fixable, or fatal.