Loading ToolBench
Ultrasync MCP presents a semantic indexing server with 9 tools, all READ_ONLY or WRITE operations on a code index. Naming is strong and action-oriented (compute_hash, index_file, delete_file, etc.). Descriptions are present and reasonably detailed (avg ~120 chars), explaining WHAT each tool does and its purpose. However, schemas are incomplete: input parameters lack formal type declarations in several cases, and output schemas are entirely undocumented. Error handling guidance is absent, tools do not explain how to recover from failures or what to do if an operation fails. No tool annotations (readOnlyHint, destructiveHint) are declared despite clear risk levels assigned in metadata. The server targets semantic code search and embeddings, which is well-scoped, but tool composition could be tighter (e.g., index_file and reindex_file are separate when a single idempotent index_file with force=true would suffice). Parameter descriptions are adequate but lack explicit format constraints and validation rules.
Add a code symbol directly to the JIT index. Use this when search() can't find inline JSX, UI elements, or nested code that isn't extracted as a standalone symbol.
Compute a 64-bit hash for a key string. Useful for computing key hashes for GlobalIndex lookups.
Delete a file and all its symbols from the index. Use when a file is deleted from the codebase or you want to remove it from search results entirely.
Delete a single symbol from the index by key hash. Use to remove manually added symbols (via add_symbol) or individual extracted symbols.
Get statistics about the current file registry.
Get JIT index statistics. Returns file count, symbol count, blob size, vector cache usage, and database location. Includes vector waste diagnostics.
Output schemas entirely undocumented. No tool describes what fields are returned, their types, or structure. LLMs cannot plan downstream tool calls or extract needed data without knowing response format.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk metadata (READ_ONLY, WRITE, DESTRUCTIVE). Agents cannot infer tool safety or retry semantics without annotations.
Error handling guidance absent. Tools do not explain recovery paths when operations fail. For example, index_directory says 'can be resumed if interrupted' but provides no mechanism or guidance for agents to resume.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | D | 56 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 18 | - | v1 |
Index files in a directory incrementally with JIT indexing. Only indexes files that have changed since last index. Progress is tracked and can be resumed if interrupted.
Index a single file on-demand with JIT indexing. Creates embeddings for the file and its symbols, storing them in the persistent JIT index. Skips files that haven't changed unless force=True.
Invalidate and reindex a file in the JIT index. Use when a file has changed significantly and you want to force a complete reindex.
Redundant tool design: index_file with force=false and reindex_file perform overlapping operations. A single index_file tool with force parameter and idempotent semantics would reduce cognitive load and improve composition.
Input parameter constraints lack explicitness. 'pattern' in index_directory described as 'Glob pattern for files' but no format specification (e.g. 'must be a valid glob expression'). 'symbol_type' in add_symbol has enum values but no validation guidance.
Stats tools (get_registry_stats, get_stats) return undefined structure. What fields does 'statistics about the current file registry' include? Count? Size? Timing? Without schema, agents cannot extract or use the data reliably.