Loading ToolBench
memex is a semantic search CLI for markdown vaults with 12 tools. Strengths: all tools have descriptions (most 100+ chars, well above the 34-char baseline); clear action verbs (find, search, explore, read, rename, index); parameters have type definitions and descriptions. Weaknesses: NO visible input/output schemas in the source code provided; no error handling guidance documented; no tool annotations (readOnlyHint/destructiveHint); limited parameter constraints (no enums, ranges, or patterns visible); some parameter descriptions lack format/constraint details; no documented output schemas to guide LLM planning. The tool definitions appear inferred from CLI code rather than explicitly registered with full schemas. Risk levels (READ_ONLY, WRITE, DESTRUCTIVE) are noted but not exposed as tool annotations.
Show or edit configuration.
Show a note's outlinks, backlinks, and similar notes. Outlinks: notes referenced via [[wikilinks]]. Backlinks: notes that link to this note. Similar: nearest neighbors by embedding distance (excludes linked notes).
Find notes by name, alias, or path using fuzzy matching. Matches against filenames, frontmatter aliases, and paths. Ranked: exact > substring > fuzzy. No embeddings needed — instant.
Index vault files. Runs automatically before search/explore. Incremental — only re-indexes files whose mtime changed.
Show or follow memex logs.
Read a note with ![[embeds]] recursively inlined. Resolves wikilink-style references including #heading and #^block-id targets. Requires Obsidian to be running with CLI enabled. Non-markdown embeds (images, PDFs, etc.) are left as-is.
No visible output schemas documented. Tools return results but LLMs cannot see what fields to expect, preventing downstream tool chaining and context planning. E.g., does search() return note_id, path, content, metadata? Does find() return the same structure?
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) visible in schema. Risk levels are noted in the metadata but not exposed as machine-readable MCP tool annotations, preventing MCP clients from validating safe vs. dangerous operations.
Parameters lack constraint documentation. 'limit' params have no min/max bounds (default=10 for find, default=5 for search, why different?). 'vault' is optional but behavior when omitted is vague ('searches all'). 'model' in vault:add has no enum or format guidance.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Rename a note file and update all [[wikilinks]] pointing to it. Handles path links ([[subdir/note]]), title links ([[note]]), aliases ([[note|Display]]), heading refs ([[note#section]]). Ambiguous links (multiple files share a name) are skipped with warning.
Search notes using embedding similarity. Embeds the query and ranks all indexed notes by cosine distance. Searches all configured vaults unless -v is given. Use natural language queries (questions work well). Returns paths by default; --full includes note content.
Add directories to a vault (creates it if new). A vault is a named group of directories that share an embedding model. Adding paths to an existing vault appends them.
Show vault details: paths, model, note counts, DB size.
List all configured vaults.
Remove a vault or a specific path from it. Without --path, removes the entire vault. With --path, removes only that directory (deletes the vault if it was the last path).
No error handling guidance. If index() fails on a file, what does the LLM see? If rename() encounters an ambiguous link, does it fail the whole operation or return partial success? Error responses should guide recovery ('Try manually resolving the ambiguous links: ...').
Tool definitions appear inferred from CLI code rather than explicitly registered with full JSON Schema. No explicit tool registration function visible (e.g., server.register_tool()). This caps schema visibility at 45/100 across all tools.
vault:list has empty input schema ({}). While appropriate for a parameterless tool, the description 'List all configured vaults' is only 33 chars, below the 50-char baseline for clarity. Should expand: 'List all configured vaults, including their paths, embedding model, and note count.'
logs tool description is vague ('Show or follow memex logs'). What format? Where are logs stored? How many lines returned by default? Is follow mode streaming or polling? LLM cannot infer.
config tool description ('Show or edit configuration') is ambiguous. Does it read/write a config file? What are valid keys? What are valid values? No enum or format constraints visible. Could accidentally write malformed config.
Destructive tool vault:remove lacks confirmation pattern. 'Remove a vault' with 'name' and optional 'path' could delete user data silently if the LLM misunderstands the semantics. No mention of dry-run, confirmation prompt, or backup guidance.