Loading ToolBench
GitHub Research CLI — 14 tools for searching, browsing, and managing repos, code, issues, discussions, releases.
14 tools with complete schemas and descriptions, but significant quality gaps. All tools have descriptions (10-150 chars, mostly adequate) and input schemas with types. However, descriptions lack LLM-optimized guidance on WHEN to use each tool vs. similar ones. Parameter descriptions are minimal (5-30 chars). Output schemas are undocumented, LLMs cannot predict response structure. Error handling is absent; no recovery guidance. Destructive tool (delete_issue) lacks confirmation pattern. Composition issues: index_* tools write to local RAG system (side effects not clearly documented); download_files writes to filesystem (risk not highlighted). No tool annotations (readOnlyHint, destructiveHint). Naming is clear and verb-first, but parameter naming inconsistencies (e.g., 'repo' vs 'owner/repo' split across tools).
Create a new issue.
Permanently delete an issue via GraphQL (irreversible).
Download specific repo files to a local directory.
Read file from GitHub repo.
Read a single issue (title, state, body).
Browse repository file tree (one level).
Fetch discussions matching a query and index into RAG.
Output schemas undocumented. LLMs cannot predict response structure for any tool. No documentation of returned fields, types, or pagination. Breaks tool chaining and forces agents to guess field names.
No error handling or recovery guidance. Tools provide no actionable error messages. E.g., if search_repos returns 0 results, no guidance on fallback strategies. If index_issues fails, no recovery path documented.
Destructive tool (delete_issue) lacks confirmation pattern. 'confirm' parameter defaults to false, but no dry-run or explicit confirmation workflow. Agents could accidentally delete issues without safeguards.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | <=2025-11-25 | v2 |
Fetch issues matching a query and index into RAG.
Fetch all releases and index into RAG.
List repository issues (default: open only).
Show how recently a repo was pushed to.
Search code across GitHub.
Search GitHub repositories.
Update an existing issue (also closes/reopens).
index_* tools (index_issues, index_discussions, index_releases) have side effects (write to local RAG filesystem) not clearly documented in descriptions. LLMs may not understand these are stateful operations with external dependencies (rag-cli binary).
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). LLMs cannot distinguish safe reads from destructive writes without explicit hints. Increases risk of misuse.