Loading ToolBench
Gmail Secretary MCP server for interactive email and calendar processing
The server defines 27 tools with substantial functionality but has significant quality gaps. Most tools have names starting with action verbs (list_, search_, get_, create_, mark_, move_, modify_, send_, respond_) which is good. However, critical issues emerge: (1) Input schemas are visible in the provided tool list but lack complete type information and constraints in the schema representations shown. For example, search_emails lists parameters but the actual JSON Schema structure is not fully detailed in the source excerpt. (2) Descriptions vary widely in quality, some are good ('Search emails using full-text search with optional filters') while others are vague ('Process email with combined actions', 'Create draft reply' with empty input schema {}). (3) Parameter descriptions are uneven: search_emails has detailed param descriptions, but many tools either have minimal descriptions or parameters with no description at all (e.g., create_draft_reply has empty input {}). (4) Tool composition shows a concerning pattern: multiple tools addressing overlapping concerns (triage_inbox, triage_priority_emails, triage_remaining_emails, quick_clean_inbox, prioritize_inbox all perform classification/triage on emails, this violates single-responsibility principle). (5) Output schemas are not documented in the provided code excerpt, we cannot see what these tools return, which fails the critical pattern:tool requirement. (6) Error handling and recovery guidance is not visible in tool definitions. Tools marked with ⚠️ MUTATION note require user confirmation, but the mechanism for this is not visible in the schema definitions themselves, making it unclear how the confirmation workflow is enforced.
Apply labels from triage results. For high-confidence (>90%) classifications: Auto-applies labels, marks read if specified in actions, archives if specified in actions. For lower confidence, applies labels but skips destructive actions unless explicitly approved.
Find emails needing response.
Create a new calendar event. ⚠️ MUTATION: Requires user confirmation before execution.
Create draft reply.
Execute approved batch cleanup - queue emails for move to Secretary/Auto-Cleaned. ⚠️ MUTATION: Only call after user has approved the cleanup candidates.
Check free/busy calendar availability.
Critical: Multiple tools perform overlapping triage/classification (triage_inbox, triage_priority_emails, triage_remaining_emails, quick_clean_inbox, prioritize_inbox). Violates single-responsibility principle. LLMs will struggle to select between them, and the agent design is confusing.
High: create_draft_reply has empty input schema ({}) with no parameters documented. The tool is unusable without knowing what inputs it requires. Does it need a UID? A folder? Body text?
High: Output schemas are not visible in the provided source code. The rubric requires documented return types for all tools. Without knowing what fields each tool returns, LLMs cannot plan downstream calls or extract required data (e.g., UIDs for subsequent operations).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | <=2025-11-25 | v2 |
Get a daily briefing with priority emails and calendar events.
Get full details of an email by its UID.
Get all emails in a conversation thread.
Format triage results for user display. Takes raw triage results and formats them for human review, showing counts by category and sample emails.
Get unread emails from a folder.
List calendar events.
List all email folders available in the mailbox.
Mark an email as read. ⚠️ MUTATION: Requires user confirmation before execution.
Mark an email as unread. ⚠️ MUTATION: Requires user confirmation before execution.
Add or remove Gmail labels from an email. ⚠️ MUTATION: Requires user confirmation before execution.
Move an email to a different folder. ⚠️ MUTATION: Requires user confirmation before execution.
Fast pattern-based prioritization of inbox emails (NO LLM). Processes ALL unread emails using pattern matching and signal analysis. High-confidence items get labeled immediately. Unclear items get Secretary/Unclear label for later LLM triage. Run this FIRST before triage_inbox. Handles bulk email efficiently.
Process email with combined actions.
Identify cleanup candidates.
Respond to a meeting invitation. ⚠️ MUTATION: Requires user confirmation before execution.
Search emails using full-text search with optional filters.
Send a new email. ⚠️ MUTATION: Requires user confirmation before execution. NEVER call this without showing the user the draft first!
Get server status including database, engine, and enrollment status.
Smart inbox triage with LLM classification
Identify priority emails.
Process remaining emails.
High: process_email has an overly vague description ('Process email with combined actions') and a generic dictionary input ('Dictionary of actions'). This violates the principle that parameter names and descriptions must be self-documenting. An LLM cannot determine what actions are valid without detailed documentation.
Medium: Many read-only tools have minimal descriptions (< 60 chars, e.g., 'List calendar events', 'Check free/busy'). The baseline is 194 chars. These lack context about when to use each tool and what problem they solve.
Medium: Confirmation workflow for mutation tools is documented in descriptions (⚠️ MUTATION notes) but not enforced in the schema or via a formal pattern. It is unclear how the system prevents an LLM from calling send_email without user approval, or what mechanism ensures confirmation is actually obtained.
Medium: No error handling guidance in tool definitions. If search_emails returns no results, or send_email fails, or a permission error occurs, there is no documented recovery path for the LLM. Error handling must tell the agent what to do next.
Medium: Some parameters lack descriptions or constraints. For example, the 'folder' parameter appears in many tools but is not consistently described (e.g., what are valid folder names? Is it case-sensitive? What's the default?). Enum constraints should replace free-form strings.
Low: Tool names could be more distinctive. Several read-only tools are named generically (get_daily_briefing, list_calendar_events, check_emails_needing_response) but do not clearly differentiate when to use one vs. another similar tool. The naming baseline from production tools emphasizes verb_noun clarity.