Loading ToolBench
MCP server for Obsidian vault operations including reading, writing, searching, and deleting markdown files
Mixed-quality MCP server with good foundation but significant gaps. All 13 tools are explicitly registered with names, descriptions, and schemas visible in server.py. Naming follows verb_noun conventions consistently (list_, get_, create_, update_, append_, patch_, delete_). Descriptions present for all tools but vary in quality, most are 20-100 chars, meeting minimum threshold but below the 50-200 char optimal range for LLM reasoning. Input schemas present and properly typed (strings, integers, booleans, arrays) for all tools, with parameter descriptions generally provided. Critical gaps: (1) No output schemas documented, LLMs cannot infer response structure from return statements; (2) Error handling is present but not recovery-guiding, errors return dicts with 'success' boolean and error messages, but don't suggest next steps; (3) No confirmation pattern for destructive delete_file_tool despite security sensitivity; (4) Parameter constraints minimal, enums not used; (5) No pagination support for list/search results that could grow large. Composition is sound, tools follow single responsibility (read, write, delete separated). Security: tool implementations validate paths and use proper exception handling, but no explicit permission-gate pattern visible. Path traversal protection via validate_relative_path utility is present.
Append content to the end of a file.
Read multiple markdown files at once.
Create a new markdown file.
Delete a markdown file from the vault.
Get today's daily note (or specific date).
Read the content of a single markdown file.
Get recently modified markdown files.
No output/return schemas documented. Tools return dicts with 'success', 'error', file contents, or match counts, but LLMs cannot infer response structure. Expected fields: file_paths, count, content, matches, context, etc. This forces LLMs to guess at downstream field names and risks failed tool chaining.
Descriptions are too brief (20-75 chars on average, target 50-200). E.g., 'Search for text across all markdown files in the vault' lacks context on when to use vs get_file_content_tool, what format results take, or performance characteristics. Low-context descriptions reduce LLM precision in tool selection.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 47 | - | v1 |
List markdown files in a specific directory.
List all markdown files in the vault root directory.
Insert content at a specific location relative to a heading.
Find files containing a specific tag.
Search for text across all markdown files in the vault.
Replace entire file content (preserves frontmatter if not in new content).
No confirmation or dry-run pattern for destructive delete_file_tool despite requiring 'confirm' parameter. While the confirm=True safety check is present in code, the tool description does not warn users or LLMs about irreversibility, and there is no dry-run mode. Agents risk permanent data loss.
Error handling returns dicts with success/error but no recovery guidance. E.g., 'File not found' does not suggest checking vault path, listing available files, or alternative tools. Agents have no next-step hint and may retry unnecessarily or dead-end.
No pagination or result limits documented for search_vault_tool, get_recent_files_tool, or batch_get_files_tool. search_vault matches are capped at 5 per file in code, but this is not documented in the description. Agents may assume unbounded results and waste context. Large vaults could exceed token budgets.
Parameter enums missing. E.g., patch_file_tool accepts position as free-form string ('before' or 'after') but does not declare an enum constraint. LLMs may hallucinate values like 'between', 'middle', 'start'. Enums are self-documenting and prevent invalid input.
Tool composition: create_file_tool has optional add_frontmatter=True default. Design is reasonable, but there is no corresponding tool to remove/edit frontmatter if the user changes their mind. Agents may need to read, edit, and rewrite entire files.