Loading ToolBench
MCP server for enterprise code AI tooling that searches indexed repositories, fetches files from GitHub, and performs semantic code searches using embeddings
Three tools with complete schemas and descriptions, but significant gaps in parameter clarity and output documentation. Tool names follow verb_noun convention (search, fetch, search). Descriptions are adequate (100-150 chars) but lack actionable context for LLM selection. Parameters have types and descriptions, but lack constraints (enums, ranges, formats). No documented output schemas. Error handling present but generic. Missing idempotency guidance and chaining field documentation.
Fetches raw file contents from a GitHub repository for the provided organization, repository, and file path.
Searches across indexed enterprise repositories and documentation using a natural language query and specified technology stack. Returns the most relevant code examples, files, and documentation to assist with development tasks and problem-solving.
Searches the Qdrant code embeddings collection for semantically similar code snippets.
No documented output schemas. Tools return text content wrapped in generic 'content' array, but LLMs cannot plan downstream calls without knowing field structure (e.g., does search return file paths, line numbers, relevance scores?).
Parameters lack constraints. 'Stack' accepts free-form comma-separated strings with no validation or enum guidance. 'keywords' in search_code_embeddings has no length/format constraints. LLMs will hallucinate invalid tech stacks.
Missing chaining IDs in responses. If search returns relevant files, the response should include org/repository/filePath so agent can immediately call fetch_file_from_git without extra lookup. Current implementation forces discovery detour.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
Error messages are generic ('Failed to fetch from webhook', 'GitHub request failed'). No recovery guidance. LLM cannot distinguish retryable (rate limit) from fatal (invalid token) errors.
No pagination support. search and search_code_embeddings return unbounded results. Large result sets will exhaust context window. Missing limit/offset parameters and total count in response.