Loading ToolBench
A minimal but real MCP server for a Markdown knowledge base
Vault MCP demonstrates solid naming conventions (verb-first: search, read_note, create_note, update_note, list_directory) and comprehensive parameter descriptions. All 8 tools have descriptions exceeding 20 characters, with most in the 150-300 char range. Input schemas are present and typed for all tools. However, output schemas are entirely undocumented, no tool specifies what fields or structure the response contains. Error handling is minimal; tools lack recovery guidance or actionable error messages. The 'gotcha_check' and 'vault_health' tools have domain-specific value but descriptions could better explain when to invoke them vs alternatives. Parameter defaults are sensible (limit=10, depth=1, auto_index=true) and do not cause data loss.
Create a note with consistent frontmatter, optionally linking it into the folder index. Refuses to overwrite. A gotcha without a severity callout in its content is rejected — the marker is not decoration, it is what makes the note retrievable. Args: path: vault-relative target, e.g. "GLOBAL/gotchas/docker/mount-stale.md". title: state the CONCLUSION, not the topic. content: the body, without frontmatter. note_type: gotcha | insight | adr | snippet | analysis | project-index | index tags: list of tags. severity: for gotchas — danger | warning | tip | info auto_index: append a line to the folder's _INDEX.md.
Return known traps relevant to a context, before you make the mistake. Pass what you are about to work on: project, technologies, paths. Only notes carrying a severity callout are returned — that marker is what makes them retrievable. Args: context: e.g. "acme-portal, docker, terraform, deploy to QA". limit: maximum warnings (capped at 20).
Structural overview of a folder without reading any content. Use this to orient before reading. Reading a folder's `_INDEX.md` is usually cheaper still. Args: path: vault-relative folder, "" for the root. depth: how many levels to descend (capped at 3).
Read one note: frontmatter, body, outgoing links, and its callouts. Args: path: vault-relative, e.g. "GLOBAL/gotchas/docker/mount-stale.md".
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., does search return note_id, path, title, content, or all of these?). This forces agents to guess response structure.
Error handling is absent. Tools do not return actionable error messages or recovery guidance. E.g., if create_note rejects a gotcha without a severity callout, the error message should explain what severity values are valid and how to fix it.
Parameter 'note_type' in create_note accepts a free-form string with a description listing valid values ('gotcha | insight | adr | snippet | analysis | project-index | index') but no enum constraint. LLMs will hallucinate invalid types.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 67 | 2026-07-28+ | v2 |
Full-text search across the vault, ranked, capped. Args: query: one or two precise terms — not a sentence. limit: maximum hits (capped at 25). folder: restrict to a subtree, e.g. "PROJECTS/acme".
Server and vault status. Call this first when anything looks wrong. A note count of 0 with a non-empty vault directory is the classic stale-mount symptom — the container's bind mount points at nothing while the host path is full.
Update a note, preserving its frontmatter. Args: path: vault-relative note path. content: the text to write. mode: "append" (default) or "replace_body".
Hygiene report: gotchas without a callout, broken links, orphans, missing indexes. `gotchas_without_callout` is the one to act on first — every entry is a note that exists and will never surface when it is needed.
Parameter 'severity' in create_note is free-form string with description listing valid values ('danger | warning | tip | info') but no enum constraint. Should be an enum to prevent invalid values.
Tool 'gotcha_check' description does not explain when to call it vs 'search'. Both retrieve notes; the distinction (gotcha_check filters by severity callout, search is full-text) is buried in the description and not obvious to LLM selection logic.