Loading ToolBench
GitHub Code Explorer using Model Context Protocol
Three tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. All tools have names starting with action verbs (search_, get_) which is good. Descriptions exist but are minimal (10-50 chars, below the 50-200 char baseline for LLM optimization). Parameter descriptions are present but lack constraints, ranges, and format guidance. No output schemas documented. Error handling returns generic messages without recovery guidance. Tools are read-only (low risk) but lack the structure expected of production-grade agents.
Get the contents of a file from a GitHub repository
Search for code across GitHub repositories
Search for GitHub repositories
Output schemas not documented. LLMs cannot plan downstream calls or extract required fields. search_code and search_repositories return raw GitHub API responses with pagination metadata, but no schema is provided to guide the agent on what fields to expect.
Parameter descriptions lack constraints and format guidance. 'page' and 'per_page' have no min/max bounds stated (baseline: 1-100 for per_page). 'q' search query has no guidance on GitHub search syntax or length limits. LLMs will pass invalid values without clear error recovery.
Error responses are generic ('error.message' from axios). No recovery guidance. Example: 'Request failed with status code 401' tells the agent nothing about whether to retry, ask the user, or check credentials. Should return 'GitHub API authentication failed. Ensure GH_TOKEN environment variable is set.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 42 | 2026-07-28+ | v2 |
Tool descriptions are too short (10-50 chars vs. baseline 50-200). 'Search for code across GitHub repositories' lacks WHEN to use it, what it returns, or how it differs from search_repositories. Descriptions should answer: What? When? What does it return?
No pagination guidance in tool descriptions. search_code and search_repositories accept page/per_page but do not document max results, total count return, or when pagination is required. LLMs may request 100 results per page without understanding token cost.