Loading ToolBench
MCP Server for Gmail integration with customer data management
This Gmail MCP server has fundamental definition quality issues that would not pass production code review. While all 7 tools are explicitly registered with schemas and descriptions (tools.ts), the quality falls short in several critical areas: (1) descriptions are generic and lack actionable guidance for LLM selection, (2) input schemas lack proper parameter descriptions for most fields, (3) no output schemas are documented, (4) error handling guidance is absent, (5) security-critical tools (authenticate_gmail, send_bulk_email) lack permission declarations and validation details. The server lands in the 'Fair' range with significant gaps typical of early-stage community tooling.
Authenticate with Gmail using OAuth2. Starts a local server and opens authorization in browser.
Complete Gmail authentication with the authorization code (only needed if using manual flow)
Search Gmail messages using Gmail search syntax (e.g., 'from:[email protected]').
Send bulk emails to customers currently loaded in the store.
Send a single email to a specific address.
Start a local web server with a drag-and-drop interface for easy customer data upload (CSV/XLSX).
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract expected fields from responses. Critical for tools like search_emails (what fields returned?) and send_bulk_email (success count? failed recipients?).
Tool descriptions lack actionable context. 'Authenticate with Gmail using OAuth2' does not explain WHEN to use this vs complete_authentication, WHAT to do if auth fails, or what credential formats are accepted. Baseline: 194 chars avg, most A+ tools 50-200 chars. Most here are 50-100 chars and generic.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 44 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 41 | - | v1 |
Directly upload customer data from a local file path (alternative to the web UI).
Security-critical parameters lack validation constraints. useManualFlow, code, filePath have minimal descriptions. No mention of: allowed code formats, expected token lifetime, file path restrictions (e.g., no path traversal), or what happens on auth failure. Baseline: high-security tools must document format and constraints explicitly.
Missing parameter descriptions reduce clarity. 'body' in send_bulk_email and send_email has descriptions, but 'port' in start_upload_server lacks guidance on valid range (1-65535? privileged ports?). 'emailColumn' in upload_customer_data does not explain behavior if column is missing or has NULL values.
No error handling guidance. What does LLM do if authenticate_gmail fails? If send_bulk_email hits rate limits? If upload_customer_data receives a malformed CSV? No recovery paths, no retry guidance, no fallback suggestions. Baseline: A+ tools classify errors as retryable/user-fixable/fatal.
Destructive and sensitive operations (send_bulk_email, send_email, delete via search) lack confirmation/dry-run patterns. An agent could send 10,000 emails with a single mistaken call. No mention of preview mode, approval gates, or undo mechanisms.
No pagination or result limits documented for search_emails. If a query returns 10,000 results, does the tool cap at maxResults=10? Does it return a next_cursor? Baseline: tools returning lists must support pagination and enforce limits. Missing in description.
send_bulk_email placeholder mechanism ('e.g. {name}, {email}') is documented but not formally constrained. What happens if a placeholder references a missing column? What about escaping? Does {email} auto-validate email format? Docs should specify exactly which columns are valid placeholders (derived from uploaded data schema).
No scope/permission declarations. Tools require Gmail API access (oauth2), file system access (upload), local server startup. LLMs have no visibility into what permissions are needed or what an agent with these tools can do. Baseline: each tool should declare required scopes (read:email, write:email, etc.).
Prompt definitions are skeletal. 'compose_bulk_email' lacks description body, argument types, and return schema. 'analyze_email_engagement' has no arguments or description. These prompts will not be useful to LLMs without fuller documentation.