Loading ToolBench
Lightning-fast code context fetcher for LLMs - scans codebases and returns relevant context for queries
Single tool with generally good schema coverage and acceptable naming. The tool name 'fetch_context' uses a verb-noun pattern and clearly conveys intent. Input schema is well-defined with proper types and reasonable descriptions. However, output schema is undocumented (returns plain string), and parameter descriptions, while present, lack specificity about constraints and valid ranges. Error handling and recovery guidance are absent from the implementation visible in source code. The tool addresses a legitimate use case (codebase context extraction for LLMs) but lacks production-grade refinements around output structure, error classification, and edge case handling.
Scan a codebase and return relevant context for a query. Args: query: The question or query to answer using the codebase. path: Path to the codebase (local path or GitHub URL). Defaults to current directory. budget: Token budget for the context (e.g. "50k", "100k"). Defaults to "50k". include: List of glob patterns to include (e.g. ["*.py"]). exclude: List of glob patterns to exclude (e.g. ["tests/*"]). force_large: Allow processing files larger than 1MB or 20k lines.
Output schema undocumented. Tool returns a plain string (result.to_text()) with no documented structure, fields, or format specification. LLMs cannot plan downstream usage or validate expected content.
Parameter descriptions lack constraint details. 'budget' parameter accepts arbitrary strings parsed via parse_token_budget() but the description does not enumerate valid formats (50k, 100k, 1m, etc.) or acceptable ranges. LLMs may pass invalid values like '999999k' or '5' without guidance.
No error handling or recovery guidance visible in tool code. If parse_token_budget() fails, if builder.build() times out, or if a path is inaccessible, no error classification or actionable recovery steps are returned to the LLM. Raw exceptions will not guide LLM behavior.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | B | 74 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 57 | - | v1 |
Glob pattern parameters (include/exclude) lack format guidance. The description states 'e.g. ["*.py"]' but does not clarify whether patterns follow .gitignore syntax, glob(3), or another standard. This ambiguity invites incorrect patterns and wasted calls.
No pagination or result limiting mechanism documented. If context extraction produces very large results, they will be returned in full, risking context window exhaustion. Tool description does not mention maximum output size or truncation behavior.
Path parameter accepts both local paths and GitHub URLs but no validation or format specification is documented. LLMs may pass malformed URLs, invalid paths, or paths that trigger security issues (e.g., symlink traversal). No guidance on when to use local vs remote paths.