Loading ToolBench
MCP-first knowledge mesh for AI agents — master server reference implementation with wiki storage, semantic search, and real-time event streaming.
VaultMCP exposes 14 tools with complete input schemas and descriptions. Naming follows verb_noun convention (read, write, list, search, append_log, audit, ext_*). Descriptions are present and substantive (100-250 chars typical), explaining WHAT each tool does and WHEN to use it. However, several critical gaps reduce the score: (1) No output schemas documented, LLMs cannot infer response structure for planning downstream calls; (2) No error handling guidance, tools lack recovery hints or error categorization; (3) Parameter descriptions lack format/constraint details (e.g., 'path' accepts any string; no validation rules stated); (4) No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels; (5) Session parameter in write/append_log is under-documented, unclear if required or optional, what fields are mandatory. Composition is sound, tools are single-responsibility and chainable (e.g., list → read → write). Security concerns: session metadata passed as parameter rather than server-side injection; no explicit permission gate documentation.
Append a structured log entry to a page's audit log. Used by agents to record actions, decisions, or events.
Retrieve the audit log for a page, showing all writes and log entries with metadata about who made each change.
Declare a custom table for an extension. The table is created in a schema prefixed with the extension name.
Deregister an extension and clean up its tables and grants.
Request embedding of a page's content. Queues the page for the embedding worker to process asynchronously.
Emit a custom event from an extension. Events are broadcast to all subscribers via the SSE endpoint.
Execute a write operation (INSERT, UPDATE, DELETE) against an extension's tables.
No output schemas documented. LLMs cannot infer response structure (fields, types, pagination) for planning downstream tool calls. E.g., read() returns 'page content, version, metadata' but no schema shows field names or types.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels declared (READ_ONLY, WRITE, DESTRUCTIVE). Agents cannot distinguish safe retry candidates from irreversible operations.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
List all registered extensions and their metadata.
Execute a read-only SQL query against an extension's tables. Query must be SELECT-only and is validated before execution.
Register a new extension (external service) with the master. Extensions can declare custom tables, emit events, and execute queries.
List pages in the wiki with optional filtering by prefix, type, and update time. Supports pagination via cursor.
Read a page from the wiki by path. Returns the page content, version, metadata (type, owners, update timestamp, last writer info).
Search pages by query using lexical, semantic, or hybrid (RRF-fused) search modes. Semantic and hybrid modes require an embedding provider.
Write or update a page in the wiki. Validates content (encoding, size, frontmatter, secrets), enforces ownership rules, detects conflicts, and renders to disk after DB commit.
Parameter descriptions lack format/constraint details. 'path' parameter accepts any string with no validation rules stated (length, character restrictions, path traversal guards). 'session' object parameter is under-documented, unclear if required, which fields are mandatory, or how to construct it.
No error handling guidance. Tools lack recovery hints (e.g., 'User not found. Try search_users() first.'), error categorization (retryable vs fatal), or actionable error messages. Agents cannot self-correct on failures.
Session metadata passed as tool parameter rather than server-side injection. Agents log all parameters, session IDs, server_id, and other metadata could leak into traces and prompt history.