Loading ToolBench
MCP-compliant Obsidian documentation server with RAG and live sync
Vault MCP demonstrates decent foundational structure with 6 tools offering semantic search, document retrieval, and indexing capabilities. All tools have descriptions (10-180 chars, within acceptable range) and clear verb-noun naming conventions. However, several tools lack complete input schemas, parameter descriptions are sparse or missing, and output schemas are undocumented. The server implements a working RAG-based document retrieval pattern but falls short of production-grade tool composition and error handling guidance. Tools are well-separated by concern (list/get/search/reindex/read/get_sections) but parameter metadata is incomplete, forcing LLMs to infer constraints and retry on invalid inputs.
Retrieves the full, raw content of a specific document from disk.
Get the full sections that enclose the given character range. This is the primary tool for targeted context around a RAG search. It identifies the tightest bounding sections and returns complete content.
Retrieves a list of all file paths currently indexed in the vector store.
Read and return the entire content of a document. WARNING: This can return a large amount of text and should be used as a last resort when more targeted methods are insufficient.
Performs an intelligent re-indexing of the vault by comparing the current state against the last known state using a Merkle tree.
Performs a semantic search for relevant document chunks using RAG query engine or vector store search.
search_documents parameter descriptions incomplete: 'instruction' parameter lacks detail on embedding model tuning; 'terse' boolean lacks guidance on when to use; no mention of expected result count or pagination strategy
Output schemas are not documented in tool definitions. LLMs cannot see what fields to expect from search_documents results, reindex_vault response, or get_enclosing_sections output. This forces agents to guess at response structure and risks extraction errors.
get_enclosing_sections requires numeric character indices (start_char_idx, end_char_idx) but these are not user-friendly and tie the tool to a specific internal representation. No guidance on how agents obtain these indices or what happens with invalid ranges.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | B | 78 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 46 | - | v1 |
search_documents 'limit' parameter lacks min/max constraints. No statement of default value or whether 'limit' is required. Unbounded numeric params invite LLM overreach (requesting 10,000 results).
Error handling is minimal across all tools. get_document specifies a 404 for missing files, but no guidance on retryable vs user-fixable errors, no recovery hints, and no suggestion of alternative actions for LLMs (e.g., 'Call search_documents to locate file first').
read_full_document description includes a warning ('can return large amount of text') but lacks guidance on when to use it vs search_documents. No token limit or result capping strategy documented; agents may request full documents and exhaust context window.
Tool composition risk: search_documents and read_full_document both retrieve document content but with different tradeoffs. No explicit guidance on when to prefer one over the other, risking agent confusion and redundant calls.