Loading ToolBench
MCP server for efficient code search using semantic embeddings and AI reranking
Single tool 'search_codebase' has a well-structured schema with all required parameters typed and described. Description is comprehensive (500+ chars) and includes detailed guidance on query formulation with good/bad examples. However, the description is excessively long for LLM consumption (violates 10-1024 char guideline), and output schema is not documented in the visible code. No error handling guidance or recovery patterns visible. Tool naming is clear (verb_noun pattern), but lacks information about what fields are returned or how pagination works.
Semantic code search with AI reranking. Finds relevant code by meaning, not keywords. ## CRITICAL: Write Specific Queries Vague queries return noise. Specific queries find exactly what you need. GOOD (specific intent + context): • "retry logic with exponential backoff in HTTP client" • "JWT token validation and refresh flow" • "database connection pooling configuration" • "error handling for file upload timeout" • "how user permissions are checked before API access" BAD (too vague - avoid these): • "authentication" → TOO BROAD. Try: "password hashing during user login" • "error handling" → TOO BROAD. Try: "error handling when payment fails" • "config" → TOO BROAD. Try: "database connection config loading" • "utils" or "helper" → MEANINGLESS. Describe what the util DOES. ## Query Formula [WHAT] + [WHERE/WHEN/HOW] = Good Query • "caching" → "Redis caching for API response memoization" • "logging" → "structured logging setup for request tracing" ## Tips • Include the PROBLEM you're solving, not just keywords • Mention specific technologies if relevant (Redis, JWT, WebSocket) • Follow up with file read for full context
Description exceeds recommended length (500+ chars vs 10-1024 guideline max ~200 for clarity). Verbose guidance on query formulation should be moved to tool documentation or examples, not embedded in the description.
Output schema not documented. LLMs cannot plan downstream operations without knowing what fields search_codebase returns (e.g., file paths, line numbers, relevance scores, code snippets).
No error handling guidance. Tool description lacks recovery hints for common failures (e.g., 'codebase_path not found', 'query too vague', 'no results'). LLMs cannot self-correct without actionable error messages.
max_results parameter lacks validation constraints. Description says 'max: 10' but does not specify minimum or explain why 10 is the ceiling. Unbounded defaults invite LLM misuse.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
No pagination or result limiting strategy documented. If search returns many results, how does the tool handle truncation? Does it return a next_cursor or total_count? LLMs need this to plan multi-step searches.