Loading ToolBench
Context-budgeted code search for AI agents. Ranked, deduplicated results that fit a token ceiling and never truncate silently. CLI and MCP server.
ctxgrep defines two well-named, focused tools with strong descriptions and comprehensive input schemas. Both tools follow verb_noun naming (search, count) and include detailed parameter documentation with constraints (regex validation, path traversal prevention, token budgets). Descriptions are 150 - 250 chars, within the 10 - 1024 baseline. However, output schemas are not explicitly documented in the visible code, the MCP registration shows input schemas but no return type definitions. Error handling descriptions are present but lack recovery guidance (e.g., 'invalid regex rejected' but no suggestion for what to do next). No tool annotations (readOnlyHint, destructiveHint) are visible despite both tools being read-only operations.
Count how many matches and files a pattern would produce, without returning any content. Costs almost no tokens. Use this first when you are unsure how broad a pattern is, then narrow it before calling search.
Search code under the server root and return the most relevant matches that fit an explicit token budget. Results are ranked (definitions above call sites, source above vendored and test code), capped per file so one dense file cannot crowd out the rest, and ALWAYS report how many matches were omitted. Use instead of shelling out to grep whenever a pattern might match more than a handful of lines.
Output schemas not documented. Input schemas are complete with types and descriptions, but return types for search() and count() are not visible in the MCP tool registration. LLMs cannot plan downstream operations without knowing what fields to expect.
Error messages lack recovery guidance. Descriptions mention 'invalid regex is rejected' and 'traversal outside root is refused' but do not suggest what the LLM should do next (e.g., 'Try a simpler pattern' or 'Verify the path is within the root').
Tool annotations missing. Both tools are read-only operations but lack readOnlyHint annotation. This prevents clients from optimizing caching or safety policies.
No pagination or result-limiting guidance in count() description. While search() documents max_tokens and max_per_file, count() does not explain whether it returns a single count or per-file breakdowns, or if results are capped.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
Parameter descriptions could be more prescriptive. 'globs' parameter says 'Include/exclude globs, e.g. ["*.rs", "!target/**"]', the example is helpful but should be paired with a formal constraint (e.g., 'Glob syntax follows .gitignore rules').