GrepAI provides 10 read-only codebase tools with complete JSON schemas and reasonable descriptions. All tools have proper input schemas with types and descriptions, meeting baseline requirements. However, descriptions are generic (avg ~80 chars, below the 194-char production baseline) and lack LLM-optimized guidance on WHEN to use each tool vs. alternatives. No tool annotations (readOnlyHint, idempotentHint) despite all being read-only. Error handling is absent, no recovery guidance, no actionable error messages. Output schemas are not documented. Tool composition is sound (each does one thing), but parameter naming could be more explicit (e.g., 'pattern' vs 'search_pattern', 'path' vs 'file_path' to avoid ambiguity across tools).
Analyze file structure.
Find function, class, or variable definitions in the codebase.
Find all references/usages of a symbol in the codebase.
Get directory tree structure.
Get metadata about a file or directory.
Get workspace information.
List files and directories in a path. Supports glob patterns.
No tool annotations despite all tools being read-only. Missing readOnlyHint and idempotentHint annotations that would help LLMs understand safety properties.
Descriptions are generic and lack LLM-optimized guidance. Average length ~80 chars vs. production baseline of 194 chars. Missing WHEN to use each tool and how it differs from similar tools (e.g., find_definition vs find_references).
No output schemas documented. LLMs cannot plan downstream calls or extract required fields. E.g., search_codebase returns 'matching lines with context' but structure is undefined.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | <=2025-11-25 | v2 |
Read the contents of a file. Supports partial reads with offset and limit.
Preview search and replace.
Search for a pattern (text or regex) across all files in the codebase. Returns matching lines with context.
No error handling or recovery guidance. Tools will fail silently or return raw exceptions. LLMs cannot self-correct or know what to try next.
Parameter naming ambiguity: 'path' used across multiple tools (list_files, get_file_info, get_directory_structure, analyze_file) without consistent semantics. 'pattern' in search_codebase vs 'file_pattern' in list_files creates confusion.