Loading ToolBench
MCP Server for Gmail Integration - provides tools to fetch unread emails and create draft replies via the Gmail API
The server defines 2 tools with explicit FastMCP registration and valid schemas. However, multiple critical gaps limit effectiveness: (1) parameter descriptions are present but generic, lacking format/range constraints or validation guidance; (2) output schemas are documented in docstrings but not formally structured in JSON Schema; (3) no error handling guidance or recovery patterns; (4) missing tool annotations (destructiveHint for create_draft_reply, readOnlyHint for get_unread_emails); (5) no pagination support despite max_results parameter suggesting need for result limiting. Tool naming is clear and action-verb prefixed, which is good. Compositions are atomic (two separate tools, not conflated), which follows the pattern. But the server falls short of production quality due to incomplete schemas and missing LLM-facing error recovery.
Create a draft reply in an existing Gmail thread. Args: thread_id (str): Gmail thread ID message_id (str): Original Gmail message ID reply_body (str): Email body content Returns: str: Human-readable confirmation message Gmail Scope: gmail.compose
Fetch unread emails from the user's Gmail inbox. Args: max_results (int): Maximum number of emails to return (default: 100, max: 500) Returns: List[dict]: A list of email summaries containing: - sender - subject - snippet - message_id - thread_id Gmail Scope: gmail.readonly
Output schema not formally declared in JSON Schema format. Tool docstrings describe return structure (list of dicts with sender/subject/snippet/message_id/thread_id for get_unread_emails; string confirmation for create_draft_reply) but no structured schema visible to LLMs in registration metadata.
No error handling or recovery guidance. If Gmail auth fails, quota is exceeded, or invalid thread_id is passed, the tools will return raw exceptions or generic failures. LLMs receive no guidance on how to self-correct or retry.
Missing tool annotations. create_draft_reply should declare destructiveHint=true (modifies Gmail state by creating draft). get_unread_emails should declare readOnlyHint=true (no side effects). These hints help LLMs reason about safety and idempotence.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 42 | - | v1 |
Parameter descriptions lack format/range constraints. max_results says 'default: 100, max: 500' in docstring but not formally as JSON Schema constraints (minimum/maximum keywords). LLMs cannot reliably enforce these bounds.
No pagination support visible. get_unread_emails accepts max_results but returns a flat list with no next_cursor, total_count, or offset. If >100 unread emails exist, the agent cannot iterate to fetch more without multiple sequential calls.
Gmail API error cases not handled explicitly. fetch_unread_emails and create_gmail_draft_reply can fail if credentials are missing, Gmail API quota is exceeded, thread_id is invalid, or user lacks compose scope. No try-catch or informative error messages visible in app/gmail_client.py.
Credentials handled via OAuth flow in gmail_auth.py but no validation that credentials have required scopes (gmail.readonly for get_unread_emails, gmail.compose for create_draft_reply). If scopes are missing, API calls will fail with cryptic 403 errors.