Loading ToolBench
MCP server providing integration with Google's Gemini CLI - an unofficial tool for AI research and analysis
This server has 4 tools with explicit schemas and descriptions visible in src/gemini_mcp/tools.py. However, multiple critical issues prevent a higher score: (1) Tool names lack clarity, 'gemini_prompt', 'gemini_research', 'gemini_analyze_code', 'gemini_summarize' are overly generic and don't follow verb_noun convention (e.g., should be 'send_prompt_to_gemini', 'research_topic', 'analyze_code', 'summarize_content'). (2) Descriptions are present but shallow (avg ~60 chars), leaving the LLM with insufficient context about WHEN to use each tool and what distinguishes them. (3) Parameter descriptions exist but lack specificity, 'Additional context to prepend to the prompt' lacks format/constraint details. (4) No output schemas documented, the tool definitions show input schemas only; no indication of response structure, fields, or pagination. (5) No error handling strategy visible, no guidance on what to do if Gemini CLI is unavailable, input validation fails, or file access is denied. (6) Security checks pass (allowed_directories concept present, file access gated), but no per-tool permission declarations. The implementation does show good practices (env var for gemini_path, Path.resolve() for traversal prevention, async/await pattern), but definition quality is mediocre by production standards.
Use Gemini to analyze code files
Send a prompt to Gemini CLI and get a response
Use Gemini to research a topic with optional file context
Use Gemini to summarize content from files or text
Tool names lack action verbs and are overly generic. 'gemini_prompt', 'gemini_research', 'gemini_analyze_code', 'gemini_summarize' do not follow verb_noun pattern (e.g., get_, create_, send_, analyze_). This forces LLMs to read full descriptions to disambiguate, increasing tool selection errors.
No output schemas documented. Input schemas are present and well-formed, but there is no specification of what fields, types, or structure the LLM should expect in responses. This forces the LLM to infer the response format, leading to parsing errors and missed opportunities for chaining tools.
Tool descriptions are too brief (avg 52 chars; baseline 194 chars). They lack WHEN context, i.e., when to use gemini_prompt vs gemini_research, or what distinguishes 'review' from 'explain' in gemini_analyze_code. Insufficient descriptions reduce LLM's ability to select the right tool.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 45 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 32 | - | v1 |
gemini_summarize has no required parameters ('required': []). Both 'content' and 'files' are optional, creating an ambiguous contract, the LLM could omit both and trigger a runtime error. Parameter validation is missing.
Parameter descriptions lack format and constraint details. E.g., 'Additional context to prepend to the prompt' does not specify encoding (UTF-8?), max length, or format (plain text, markdown, structured). 'List of file paths to include as context' does not state whether they are absolute/relative paths, max file count, or supported file types.
Enum fields (analysis_type, summary_type) lack descriptions for each option. An LLM invoking 'analysis_type': 'security' has no explanation of what output to expect. Best practice: 'security', [description of what security analysis includes].
No error handling guidance visible. No indication of: (1) What errors can occur (Gemini CLI unavailable, file not found, permission denied). (2) Whether errors are retryable. (3) What the LLM should do next (ask user, try alternative tool, give up). This violates the recovery-guide pattern.
No per-tool permission declarations. While the code includes allowed_directories logic (good), the tool definitions do not declare what permissions (e.g., 'read:files', 'execute:gemini') are required for each tool. This limits audit clarity and least-privilege configuration.