Loading ToolBench
The Email MCP server has three tools with explicit definitions and working schemas, but exhibits significant quality gaps. All tools have descriptions and input schemas visible in the source, but descriptions lack LLM-optimization guidance, parameter descriptions are minimal, and output schemas are undocumented. The naming follows action-verb conventions (send_email, setup_credentials, get_credentials_status), but descriptions are verbose and lack clarity on WHEN to use each tool vs alternatives. Error handling returns structured dicts but lacks actionable recovery guidance. No tool annotations (destructiveHint, readOnlyHint, idempotentHint) are present. The server's reliance on OAuth file-based credential management is secure (no secrets in parameters), but the setup_credentials tool's file copying pattern is unusual and not well-justified. Overall: functional but below production quality.
Check the status of OAuth credentials and token. Returns: Dictionary with status information
Send an email using Gmail API with OAuth 2.0 authentication. Args: to: List of recipient email addresses subject: Email subject line body: Email body content (can include HTML if html=True) cc: Optional list of CC recipients bcc: Optional list of BCC recipients html: Whether to treat the body as HTML (default: True) Returns: Dictionary with status and message ID if successful
Set up OAuth 2.0 credentials for Gmail API by copying from a source path or creating a new file. Args: source_path: Optional path to an existing OAuth certificate.json file Returns: Dictionary with status and file path
Output schemas not documented. Tool descriptions omit what fields the response contains. LLMs must infer response structure from source code inspection, not from tool metadata.
Parameter descriptions are generic or absent. 'source_path: Optional path to an existing OAuth certificate.json file' is minimal. Missing context: format requirements, file size limits, validation rules, what happens if the file is malformed.
No error guidance for recovery. Responses like {"success": False, "error": "..."} lack actionable next steps. E.g., 'Credentials file not found' should suggest 'Call setup_credentials() with source_path=...' or point to OAuth setup docs.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 52 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 48 | - | v1 |
No tool annotations. send_email and setup_credentials are destructive but lack destructiveHint=True. get_credentials_status is read-only but lacks readOnlyHint=True. Annotations help LLMs reason about side effects and retry safety.
send_email accepts 'to' as a required parameter but offers no way to accept human-friendly identifiers. The Gmail API requires valid email addresses. If an LLM has 'Jack' from chat context, it must somehow resolve to an email first, but no lookup tool exists. Limits composability.
setup_credentials tool design is questionable. Copying OAuth certificates from arbitrary source_path is unusual and poses a security risk if the source file is compromised. Better to guide the user through Google OAuth flow once and store the token securely server-side.
Description of send_email mentions 'html=True' default but this is not emphasized. The LLM might not realize HTML tags in the body will be rendered; if the user says 'send an email with the text <b>bold</b>', the LLM may not know this will be rendered as formatted text.