Loading ToolBench
Local MCP sidecar that lets a small model on your own machine answer coding-agent questions about version-pinned upstream repos, with compact cited answers.
Strong tool definitions with clear naming (verb_noun pattern), comprehensive descriptions (avg 180 chars), and well-structured input schemas using Zod. All 8 tools have descriptions and typed parameters. Key strengths: grep_repo and read_file have excellent constraint documentation (line limits, byte caps, regex guidance); ask_project/ask_local include confidence semantics and debug modes. Weaknesses: output schemas are not formally documented in the code (inferred from descriptions); error handling guidance is minimal; no tool annotations (readOnlyHint/destructiveHint); update_project lacks confirmation/dry-run pattern for destructive operation.
Ask a narrow question about the user's CURRENT working repo (not a third-party library). Pass the absolute path of the local repo root. No clone, no version pin; uses the working tree as-is, including uncommitted changes. Same response shape and `confidence` semantics as ask_project.
Pinned-version library lookup. Requires version/tag/branch/SHA. Ask with exact symbol/path; say definition/declaration if usage sites are not enough. Response includes a `confidence` field (ok | partial | low) describing how well each citation matches what the agent actually read; verify against `sources` when confidence is partial or low.
Search the cloned upstream repo for a regex pattern. Returns file:line:text matches. Use this to find where a symbol, option, or phrase appears. Be specific - use the exact identifier (e.g. 'stopWhen' or 'streamText'). Use `read_file` after to read context around a hit.
List catalog projects and cached <project>@<version> clones.
Read a line range from a file in the cloned repo. Use after grep_repo to see context around a hit. Capped at 120 lines / 8 KB per call. Each returned line is prefixed with its ABSOLUTE file line number, e.g. ` 21| const guestId = ...;` - cite those numbers directly in your answer, do not count from the snippet start.
Output schemas not formally documented. Tool descriptions infer return structure (e.g., 'Returns file:line:text matches' for grep_repo, 'Response includes confidence field' for ask_project), but no explicit JSON Schema or TypeScript interface is visible in the code. LLMs cannot reliably parse unstructured responses.
update_project (WRITE operation) lacks confirmation/dry-run pattern. Destructive tools should support a confirmation step or dry-run mode to prevent accidental re-clones or branch overwrites. No error recovery guidance if refresh fails mid-operation.
Tool annotations missing. No readOnlyHint, destructiveHint, or idempotentHint annotations in tool definitions. This prevents clients from understanding which tools are safe to retry, which modify state, and which are read-only. Current spec (2026-07-28) expects these hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Force refresh/re-clone for a project version or moving branch.
Fetch and parse the content of a web page. Capped at 6 KB. Strips HTML tags and returns plain text. Blocks private/loopback addresses by default unless LOCAL_CONTEXT_ALLOW_PRIVATE_FETCH=1.
Search the web for information when the repo cannot answer. Returns up to 8 results with title, snippet, and URL.
Error handling lacks recovery guidance. Tools return errors (e.g., 'blocked: hostname resolves to private address') but do not guide the LLM on next steps. Pattern: 'User not found. Try search_users() with a partial name.' Missing from most tool error paths.
list_projects description is minimal (inferred from name only). No explanation of what 'cached <project>@<version> clones' means or when to call this tool. Baseline: descriptions should be 50-200 chars with WHEN/WHY context.