Loading ToolBench
A Model Context Protocol server for analyzing large codebases using Gemini 2.0
Single tool 'deepview' with incomplete schema, vague description, and missing parameter details. Tool name lacks action verb clarity. Input schema visible in code shows two parameters (question, codebase_file) with basic type info but insufficient constraint documentation. Description is generic and lacks clarity on when to use this tool vs alternatives. No output schema documented. Error handling returns bare error strings rather than actionable recovery guidance. The tool conflates two concerns: querying a codebase AND loading files dynamically, which should be separated.
Ask a question about the codebase using Gemini.
Tool name 'deepview' lacks action verb. Should be 'query_codebase', 'analyze_codebase', or 'ask_about_codebase' to clarify intent before reading description.
Input schema present but lacks critical constraints: 'question' parameter has no length bounds, format pattern, or guidance on what constitutes a valid question. 'codebase_file' is optional but description does not explain fallback behavior clearly.
Tool description (56 chars) is below 10-char minimum for clarity. 'Ask a question about the codebase using Gemini.' does not explain WHEN to use this tool, WHAT kinds of questions it handles well, or what the user should expect back. Does not mention that it requires a codebase file to be loaded first.
No output schema documented. Code shows tool may return a string (response.text) OR a dict with 'error' key. LLM cannot plan downstream operations or parse results reliably. Baseline for A+ tools: 100% have documented return types.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 38 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 31 | - | v1 |
Error handling returns bare dictionaries ('error': 'string') or strings without recovery guidance. Example: 'No codebase loaded. Please provide a codebase file.' tells the LLM to call the same tool again with a file param, but this is implicit, not stated. Should return: 'No codebase loaded. Call this tool with codebase_file parameter pointing to a valid file path, or ensure the codebase was loaded at server startup.'
Tool combines two responsibilities: (1) querying a codebase via Gemini, and (2) dynamically loading codebase files. This violates the single-responsibility principle. Should split into 'load_codebase' and 'query_codebase' so agents can compose them independently.
Global state 'codebase_content' is shared across all tool calls. This makes the tool stateful and non-idempotent: calling deepview(question='X') twice with different codebase_file params may return different results depending on the order and internal state. Agents rely on repeated calls producing consistent results.
Parameter 'codebase_file' accepts a string file path but does not validate it, document format constraints, or explain path resolution (absolute vs relative, current working directory context). LLMs may pass invalid paths leading to silent failures.
No mention of model selection or token limits. Tool constructs a user prompt containing the entire codebase, which could easily exceed Gemini's context window (200k tokens). No warning about codebase size limits or truncation behavior.
Tool description does not disclose that it requires GEMINI_API_KEY environment variable. Agents have no way to know this dependency or recover if the key is missing (though server will raise ValueError at startup).