Loading ToolBench
The Gmail MCP server has 10 tools with explicit schemas and descriptions visible in the source code. However, significant quality gaps reduce the overall score: (1) tool names lack clarity and action specificity, 'send_introduction_email' conflates two concerns (composition), 'check_gmail_config' is vague about what 'check' means; (2) descriptions are minimal (9-47 chars), below the 50-200 char optimum for LLM reasoning; (3) parameter descriptions are sparse or missing in some tools (e.g., 'query' in list_unread_emails has minimal context); (4) no output schemas documented anywhere, the API response structure is opaque; (5) error handling is generic (McpError wraps with only error messages, no recovery guidance, no classification); (6) parameters lack constraints (e.g., 'maxResults' is unbounded, 'html' is unexplained boolean); (7) no idempotency guarantees for write tools, critical for agents retrying on ambiguous failures; (8) OAuth/SMTP credentials stored in environment, correct injection pattern, but no scope declarations or permission gates documented.
Archive (remove from INBOX) an email by ID (requires OAuth2 config)
Check if Gmail configuration is properly set up
Move an email to trash by ID (requires OAuth2 config)
Retrieve a full email by ID (requires OAuth2 config)
List available email templates
List Gmail labels (requires OAuth2 config)
List recent unread emails (requires OAuth2 config)
Tool descriptions are critically short (9 - 47 characters, well below the 50 - 200 char optimum). At 9 chars, 'Check if Gmail configuration is properly set up' provides insufficient context for LLM tool selection. Agents cannot reason about when to call 'check_gmail_config' vs. 'send_email' without fuller description.
No output schemas documented. The CallToolRequest handler returns McpError wrapping with only a message string, no typed response structure. LLMs cannot plan downstream tool calls (e.g., after list_unread_emails, what fields are available to extract a message ID?). This breaks tool chaining.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 43 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 52 | - | v1 |
Send an email using Gmail
Send a professional introduction email
Send an email using a stored template with variables
No pagination support. list_unread_emails and list_labels accept maxResults but lack offset/cursor or total_count. Large result sets will truncate without visibility into remaining items, forcing wasteful re-queries.
Parameter descriptions missing or vague. 'query' in list_unread_emails lacks explanation of Gmail search syntax (is it 'from:user' or 'subject:xyz'?). 'html' in send_email has no guidance on when to use it. Parameters lack type constraints (e.g., maxResults is unbounded, should be 1 - 100).
Error handling does not guide recovery. All errors throw McpError(ErrorCode.InternalError, ...) with a message only. No classification (retryable vs. user-fixable vs. fatal), no actionable hints (e.g., 'OAuth token expired. Check GMAIL_OAUTH_TOKEN env var' for OAuth failures, or 'Email not found. Call list_unread_emails to find valid IDs' for missing email IDs).
Destructive operations (delete_email, archive_email) lack confirmation or dry-run support. An agent invoking delete_email can permanently remove messages without a confirmation step, risking data loss on logic errors.
'send_introduction_email' conflates two concerns: composing a templated intro message AND sending it. Per pattern:tool, each tool should do one thing. Split into 'compose_introduction_email' (returns body) and 'send_email' (takes body + recipient), allowing reuse and agent composition.
Tool name 'check_gmail_config' is vague. Does it verify credentials? Check OAuth scope? Validate SMTP settings? Rename to 'verify_gmail_credentials' or 'test_gmail_connection' to clarify intent and let LLMs infer the right moment to call it.
No idempotency guarantees or duplicate detection for write tools. If an agent retries send_email on a transient error, the email may be sent twice. Document which tools are idempotent (archive_email probably is; send_email is not) and provide deduplication strategies for non-idempotent ones.
No scope or permission declarations. Tools should declare required permissions (e.g., 'read:gmail', 'send:email', 'admin:delete'). Missing scopes prevent least-privilege agent configuration and clear audit trails.