Loading ToolBench
Full AI context layer over MCP — tree-sitter code-map, document RAG (PDF/Office/HTML/email + OCR + reranker), shared agent memory, on-demand web crawl, git history + blame + per-symbol diff. 300+ languages, 10+ coding-agent harnesses, content-addressed Fjall + LanceDB.
Two tools with comprehensive schemas and detailed descriptions. The 'code' tool has an exceptionally long description (1200+ chars) that reads like a reference manual rather than an LLM-optimized guide, it buries key use cases in technical jargon. The 'admin' tool similarly over-documents modes inline. Both tools have well-typed input schemas with parameter descriptions, but descriptions could be more concise and action-oriented. Output schemas are not explicitly documented in the visible code. Error handling guidance is minimal. Naming is clear (verb_noun pattern), but the tools are complex with many modes, which increases cognitive load for LLM selection.
Operate basemind itself: index health, re-indexing, cache footprint and cleanup, usage telemetry, and context-shrinking text tools. `mode` is required. `status` reports this workspace's indexed state — file count, per-language breakdown, total bytes, on-disk blob count, schema version, and whether a scan is still running; ask it first when a query comes back empty or says 'no index' / 'no indexed files'. `repo` returns repository identity: workdir, current branch, full + short HEAD sha. `rescan` re-indexes the working tree in-process so symbols, calls, and outlines you just wrote become searchable — run it after editing code; `paths` scopes it incrementally, `full: true` forces a complete re-index and overrides `paths`. It holds an exclusive lock (other queries block; <1s for ~100 files) and returns scanned / updated / removed counts. `cache_stats` reports basemind's resource footprint: on-disk bytes per component (blobs / views / lance / git-cache / telemetry / git-history), the `du`-accurate `total_bytes`, blob and orphan counts, and the serving process's current + peak RSS. `gc` reports how much of the blob store is orphaned and reclaimable — non-destructive, `removed` is always 0 while the blob store is machine-global. `cache_clear` deletes one cache component (`component`: blobs|views|lance|git-cache|telemetry|all, or views:<name>); `blobs` needs `confirm=true`, and `views`/`all` are refused in-process because they back the live index — stop the server and run `basemind cache clear` instead. `telemetry` aggregates recorded calls into a usage summary — total calls, per-tool histogram, response bytes, heuristic `est_tokens_saved` — over `window` (`today` default, `1h`, `24h`, `all`), optionally filtered to one `tool`. `compress` shrinks content so it costs less context: with `path` it returns an indexed file's structural outline (imports + signatures, verbatim, never bodies and never paraphrased); with `text` it runs a lexical prose pass — supply exactly one of the two. `delta` emits a compact +N/-M line diff from `old` to `new` so re-reading something you already saw costs only what changed; either side over 50,000 bytes or 2,000 lines bails to a full re-read (`bailed=true`). `checkpoint` extracts decisions, errors, and changed files from session `text` into a credential-safe summary to persist or re-inject instead of the whole transcript — the changed-file list comes from this server's git working tree, not from the text. `waste` flags redundant reads, repeated queries, and oversized reads in a JSON-Lines tool-call `log`. Parameters that belong to another mode are rejected, not ignored.
Tool descriptions are excessively long (1200+ chars) and read as reference manuals rather than LLM-optimized guides. The 'code' tool description buries key use cases in technical jargon and mode documentation. Descriptions should be 50 - 200 chars, with mode details moved to parameter descriptions or separate documentation.
Output schemas are not documented in the visible source code. LLMs cannot plan downstream calls or extract required fields without knowing what each tool returns. Add explicit return type documentation for each mode.
Both tools use a 'mode' parameter to multiplex many sub-operations (code has 13 modes, admin has 11). This violates single-responsibility principle and forces LLMs to reason about mode selection before understanding what the tool does. Consider splitting into separate tools (e.g., code_outline, code_search, code_grep) or at minimum, provide a discovery tool that lists available modes.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 65 | 2026-07-28+ | v2 |
Read this repository's code map instead of opening files: grep it, find where something is defined, see who calls this, locate a file by name, and pull one symbol's body. Everything is served from the index — paths, lines, columns and signatures — at a fraction of the tokens a Read or a shell `rg` costs. `mode` is required. `outline` returns one file's structure (symbols with kind + row/col + signature, plus imports) — read this INSTEAD of opening the file, then fetch only the span you need; `l2: true` adds call sites and doc comments when an L2 blob exists for the file's current content. `symbols` searches every indexed symbol NAME for `name` as a case-sensitive SUBSTRING (optional `kind` filter) — the definition finder, and the right answer to "where is X defined"; `total` counts matches up to a per-call cap (`limit*64`, min 2000) and sets `total_is_partial` when it stops there. `grep` is regex content search (Rust `regex` syntax) over EVERY indexed file — use it for a pattern, a string literal or a comment, and prefer `symbols` for a plain identifier; `limit` caps hits, never files, so `total_matches` is exact, and `language` / `path_contains` narrow the sweep. `files` enumerates indexed paths (`path_contains` / `language` filters). `find` is fuzzy filename search (fzf/fd-style subsequence, case-insensitive, ranked by score) — reach for it instead of `find` / `fd` / `ls -R`. `definition` resolves the reference at `path`:`line`:`column` to the definition it BINDS to — scope-resolved, not name-matched, so it never conflates same-named symbols, and it follows cross-file imports for JS/TS; `line` is 1-based, `column` 0-based bytes; a position with no resolved binding returns no `definition` rather than an error. `references` finds every call site whose callee identifier contains `name` — NAME-ONLY, no scope resolution, so `Foo::bar()` and `bar()` both match `name="bar"`; that is the fast, complete floor for "what calls this". `callers` answers the same question for ONE specific definition: it resolves `path` + `name` (+ optional `kind`) first, echoes it as `definition`, then runs that same name scan — so `total` agrees with `references` on an unambiguous name — and additionally marks each hit `resolved` when scope/import resolution PROVED it binds to that definition (`resolved_total`). `resolved: false` is not evidence a hit isn't a caller; trust `total` for completeness before a refactor and filter on `resolved` when you want precision. `implementations` finds the types that implement / extend / inherit `trait_name` (case-sensitive substring; Rust, Python, TS/TSX, JS class + interface extends/implements — Go structural satisfaction is not detected). `dependents` is the reverse import lookup: indexed files whose imports mention `module` (heuristic substring against each recorded module path). `expand` returns one symbol's RAW SOURCE BODY resolved by `path` + `name` (+ `kind` to disambiguate an overload) — the inverse of an outline entry, and the second half of the outline-then-expand pattern. `semantic` searches code by MEANING rather than spelling: `lane` picks `hybrid` (default; RRF fusion of vector, BM25 and exact-symbol lanes, degrading gracefully when a lane is unavailable), `semantic` (vector only) or `keyword` (BM25 only); it returns POINTERS (path + line/byte range + symbol), never bodies — fetch one with `chunk`. `chunk` returns a single chunk's source by `path`, disambiguated with `chunk_id` or `byte_start`. Caps: `symbols` / `grep` / `references` / `callers` / `implementations` default `limit` 100, max 1000; `files` / `find` default 200, max 5000; `semantic` default 10, max 100. The index scanners bound their work at `scan_cap = limit * 8` and flag a cut result with `total_is_partial`. `cursor` pages (`references` / `callers` / `implementations` cursors are stable across rescans; the in-memory ones set `cursor_invalidated`), `max_tokens` budgets the returned list and sets `budgeted`, and `format: "toon"` returns compact tabular rows. Parameters that belong to another mode are rejected, not ignored.
Error handling guidance is absent. No descriptions explain what to do if a query returns empty, an index is stale, or a permission is denied. Add recovery hints like 'If no results, try rescan() to update the index' or 'If permission denied, check your workspace access.'
Parameter descriptions lack concrete constraints. E.g., 'limit' has no min/max bounds stated in the description (schema shows max 1000, but LLMs don't parse JSON Schema). 'line' and 'column' descriptions don't clarify 1-based vs 0-based indexing clearly enough for LLM reasoning.