Loading ToolBench
Local knowledge substrate for owned markdown/Obsidian vaults, exposed through MCP, REST, and CLI with multimodal OCR/ASR/CLIP search
exomem presents a vault-management system with 25 tools covering file I/O, search, and note operations. Strengths: all tools have names and descriptions present; most parameters have type definitions and descriptions; clear verb-noun naming convention (create_, get_, list_, delete_, etc.). Weaknesses: descriptions vary significantly in quality and length; many descriptions are brief (under 50 chars) and lack action guidance; output schemas are documented in tool-definition code but not visible in the provided source; no error handling patterns visible (recovery guidance, actionable errors); tool compositions lack dependency hints; some tool pairs (e.g., add vs create_file, search vs semantic_search vs claim_search) blur responsibility boundaries; no tool annotations (readOnlyHint, destructiveHint) visible in MCP registration.
Alias for create_file (legacy)
Append content to the end of a file
Search for factual claims in vault documents
Get server coordination status and availability
Create a new directory in the vault
Create a new markdown file in the vault
Delete a directory from the vault
Delete a file from the vault
Vague tool name 'set_take' does not clearly convey 'replace entire file content'. LLMs will confuse it with set_frontmatter_field or other mutation tools. Should be renamed to 'set_file_content' or 'replace_file'.
Duplicate/overlapping tool 'add' (legacy alias for create_file) violates single-responsibility. Both add and create_file accept the same parameters. LLMs waste reasoning deciding between them. Remove 'add' or clearly distinguish it.
Three search variants (search, semantic_search, claim_search) lack dependency hints and mutual-exclusivity documentation. When should an agent call search vs semantic_search? No guidance provided. Add descriptions like: 'Call this for keyword-based search. For semantic/embedding-based search, call semantic_search instead.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
Edit file content by line range
Read a file or directory listing
Extract YAML frontmatter from a markdown file
Search vault images by caption/text (CLIP, requires media extra)
Create a wiki-style link between notes
List contents of a vault directory
List files that link to a given file
List files in the vault trash
Apply multiple edits to a file in sequence
Create a timestamped note
Preserve a file snapshot in vault history
Restore a file from trash
Full-text search the vault (keyword/BM25 + optional semantic)
Semantic search via embeddings (requires embeddings extra)
Set or update a YAML frontmatter field
Replace file content entirely
Search transcripts of audio/video files (requires media extra)
No visible error handling patterns (recovery guides, actionable errors, categorization). Tools like delete_file should document: 'If file not found, call list_directory to verify path. Cannot be undone.' Bare failures with no guidance force agent guessing.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) visible in MCP registration. This violates current spec (2026-07-28) requirement. Tools must declare their impact: get*, list*, search* should have readOnlyHint=true; delete_*, and multi_edit should have destructiveHint=true.
Descriptions for several tools are under 40 characters and lack context for LLM selection: 'coordination_status': 'Get server coordination status and availability' (47 chars), does not explain WHEN to call it or WHAT it returns. 'Create a new directory in the vault' (36 chars), vague.
Output schemas not fully documented in source. For list_directory, search, semantic_search, what fields are returned? Pagination? Total count? Without output documentation, agents cannot plan downstream calls or extract required IDs (path, rank, score, etc.).
Tools like note (create_timestamp_note), link (create_link) and preserve lack clarity on prerequisites and parameter relationships. Does 'link' require both from_path and to_path to exist? Does 'note' auto-generate the filename? Undocumented dependencies force agent trial-and-error.
Parameter 'recursive' in list_directory lacks guidance on what structures are returned when true. Does it flatten nested dirs or return tree structure? Format matters for agent parsing.
No validation hints in parameter descriptions. Parameters like 'path' have no length limits, character restrictions, or path-traversal guards documented. LLMs cannot infer validation rules and may pass './../etc/passwd' unchecked.