Loading ToolBench
Local memory infrastructure for AI agents. Typed vaults — semantic and procedural — with document indexing, skill management, and autonomous agent access via MCP, HTTP API, or CLI. Multi-vault architecture, zero cloud dependencies.
CtxVault MCP server has 8 tools with clear semantic purpose around vault management (query, write_doc, list operations) and skill management. Tool naming follows verb_noun convention well (query, write_doc, list_vaults, read_skill). Descriptions are present and moderately detailed (average ~140 chars), explaining WHEN to use each tool. However, critical gaps exist: (1) parameter descriptions are minimal or absent, most parameters lack detail on format, constraints, or expected values; (2) output schemas are not documented in the source code provided, response types are referenced (QueryResponse, WriteDocResponse, WarmupStatusResponse) but their structures are not shown; (3) error handling is basic, custom exceptions are caught but recovery guidance is limited; (4) no tool annotations (readOnlyHint, destructiveHint) despite having READ_ONLY and WRITE operations clearly marked; (5) input schemas in the code lack min/max constraints, enums, or format specifications. The server demonstrates solid domain modeling and idempotent write operations (overwrite flag), but falls short of production-grade polish in parameter documentation and schema visibility.
List all indexed documents inside a specific vault. Use this to understand what knowledge is available before performing a search.
List all available skills inside a specific vault. Use this to understand what skills are available before trying to fetch one.
List all available vaults. Use this before querying or writing to discover which vaults exist and choose the right one.
Search for relevant information in a CtxVault vault using semantic similarity. Use this when the user asks a question that might be answered by their personal knowledge base or documents. Returns the most relevant text chunks with their source files.
Retrieve a specific skill from a skill vault by name.
Check if the embedding model has finished initializing. Use this to verify warmup status before attempting queries or writes.
Parameter descriptions are missing or minimal. 'vault_name' and 'query' in the query() tool have short descriptions; 'file_path', 'content', 'generated_by' in write_doc lack detail on format or constraints. LLMs cannot infer what format file_path should take (relative? absolute? with extension?) or validation rules for content size.
Output schemas (QueryResponse, WriteDocResponse, WarmupStatusResponse) are referenced but not documented in the provided source code. The LLM cannot see what fields these responses contain, their types, or whether they include pagination, error details, or resource IDs needed for chaining. This violates the critical requirement that output schemas must be documented.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 61 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 43 | - | v1 |
Save new information or agent-generated content to a semantic vault for future retrieval. Use this only with semantic vaults, to persist important context, summaries, or notes that should be remembered across sessions. Supports .txt, .md, and .docx formats.
Create and store a new skill in a skill vault. Use this to persist procedural knowledge, instructions, or how-to guides that agents can retrieve and execute later. The skill will be indexed by name and description for fast lookup. Use this only with skill vaults.
No input schema constraints visible. Parameters like 'query' (string) and 'content' (string) lack min/max length, regex patterns, or format specifications. The 'overwrite' boolean defaults to false, which is safe, but 'file_path' in write_doc has no validation rules, LLMs could pass invalid paths like '../../etc/passwd' without sanitization guidance.
No tool annotations (destructiveHint/readOnlyHint/idempotentHint) present, despite explicit Risk markings (READ_ONLY vs WRITE). MCP spec recommends tool annotations to signal safety to clients. write_doc and write_skill should carry destructiveHint or explicit mutation markers.
Error handling is basic. Custom exceptions (VaultNotFoundError, EmptyQueryError, UnsupportedVaultOperationError) are caught and converted to ValueError, but messages lack recovery hints. E.g., 'Vault not found' should suggest 'Try list_vaults() to see available vaults.' No guidance on retryability or user-fixable vs fatal errors.
AGENT_ID permission model uses a command-line --agent flag, but tool descriptions do not explain how authorization works or what happens on permission failure. The check_access() function exists but is incomplete in the provided code snippet (line 'check_access' without invocation in write_doc). Unclear authorization semantics.
list_* tools (list_vaults, list_docs, list_skills) lack pagination parameters (limit, offset, next_cursor). If a user has 1000+ documents or skills, the entire list will be returned, exhausting context. Tool descriptions do not mention result limits or whether pagination is supported.