Loading ToolBench
Local semantic search MCP server over ~/.claude/reports vault using sentence-transformers embeddings and flat JSON+.npy storage
Three tools with complete schemas and strong descriptions (avg 180 chars, well above 10-char minimum). All parameters typed and described. Tool names follow verb_noun pattern (semantic_search, reindex, suggest_links). Output schemas documented in descriptions. Key gaps: no error handling guidance (e.g., what to do if vault is empty or corrupted), no parameter constraints (top_k unbounded), no idempotency hints for reindex (full rebuild could be expensive if called repeatedly), and no mention of prerequisites or dependencies between tools. Descriptions are clear but could be more LLM-optimized with explicit 'when to use' guidance.
Rebuild the semantic search index from the current state of ~/.claude/reports. Full rebuild every call (no incremental logic, by design). Takes well under 2 minutes at the current corpus size. Returns {files_processed, files_skipped, chunks_total, duration_s}.
Semantically search the ~/.claude/reports vault and return the most relevant chunks for a natural-language query. Results below a similarity threshold are filtered out rather than returned as weak/irrelevant matches — an empty list means no relevant report was found, not that the index is broken. Returns a list of {file, heading, snippet, score}, most relevant first.
Suggest [[wiki-link]] candidates between semantically-close files in ~/.claude/reports that aren't currently linked to each other. Uses per-file centroid embeddings (mean-pooled from each file's existing chunk vectors) compared file-to-file. Already linked pairs are excluded via fail-open wiki-link resolution. Suggest-only: this tool never writes to any file. To actually insert a confirmed link, use the obsidian-mcp server's edit-note tool. Returns a list of {file_a, file_b, score}, most similar first.
No error recovery guidance. semantic_search and suggest_links raise RuntimeError if index not found, but description does not tell LLM to call reindex() first, this is only in the exception message, not the tool description.
top_k parameter unbounded. No min/max constraints documented. LLM could pass top_k=10000, causing performance issues. Rubric baseline: numeric params must specify range (e.g., 1 - 100).
reindex() has no idempotency hint or warning. Description says 'full rebuild every call' but does not warn that repeated calls are expensive (2 minutes, model download on first run). LLM may not realize cost of retries.
No tool composition guidance. suggest_links says 'use obsidian-mcp's edit-note tool to insert links' but does not explain how to chain tools or what parameters edit-note expects. Breaks the tool-chain pattern.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | <=2025-11-25 | v2 |
semantic_search filters results below similarity threshold but does not document the threshold value or how to interpret scores. LLM cannot reason about result quality or adjust queries based on score ranges.