Loading ToolBench
Model Context Protocol server for a personal Gmail (consumer Google account) inbox and calendar, via the Gmail and Google Calendar APIs. Stdio transport, loopback OAuth, OS-keyring token cache.
Strong foundation with 20 well-named tools, complete input schemas, and clear descriptions. Tool names follow verb_noun convention (gmail_list_*, gmail_read, gmail_send, etc.). All tools have descriptions (avg 156 chars, within 10-1024 baseline). Input schemas are properly typed with JSON Schema. However, output schemas are not documented, LLMs cannot predict response structure. Parameter descriptions are present but some lack detail on constraints (e.g., maxBodyChars range not explained in description). Error handling guidance is minimal. No tool annotations for destructive operations despite many write/delete tools. Pagination is well-implemented (cursor-based with limit constraints).
Delete an event. If you organize it and sendUpdates=all (default), attendees receive a cancellation. Use sendUpdates=none for solo events.
Create an event. Times use {dateTime, timeZone} where dateTime is local-form (no offset) and timeZone is an IANA name. If attendees are provided, invites are sent. isOnlineMeeting attaches a Google Meet link.
List the user's calendars (primary + subscribed) with id, name, default flag, and whether they're editable.
List events in a time range. Recurring series are expanded into individual instances, ordered by start time. Defaults to the primary calendar.
Read full details of a single event including description, attendees, and recurrence.
Output schemas not documented. LLMs cannot predict response structure for any tool. Responses must be inferred from tool names and descriptions alone, increasing hallucination risk.
Destructive tools (gmail_send, gmail_delete, gmail_reply, gmail_send_draft, gmail_calendar_create_event, gmail_calendar_update_event, gmail_calendar_cancel_event) lack tool annotations (destructiveHint=true). Agents cannot distinguish safe reads from irreversible writes.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
Respond to a meeting invite as accept, tentativelyAccept, or decline. Optionally include a comment and choose whether to notify the organizer.
Update an event's subject, time, location, description, or attendees. Works on single instances of a recurring series (each instance has its own id).
Create a draft without sending. Returns a draftId for gmail_send_draft.
Delete a message. By default moves to Trash (recoverable). Pass hardDelete=true to permanently delete — not recoverable.
List the mailbox's labels (system labels like INBOX/SENT/TRASH and custom labels) with id, name, type, and unread/total counts.
List the most recent messages carrying a label, newest first. Defaults to INBOX. Supports unreadOnly. Returns up to `limit` (max 100) plus a `nextCursor`; pass it back as `cursor` to page through more.
Mark a message read (remove UNREAD) or unread (add UNREAD). Pass isRead=false to mark unread.
Add or remove labels on a message. Removing INBOX archives the message; adding STARRED stars it. Use gmail_list_labels for ids.
Fetch a single message by id, including its decoded body (text preferred, html fallback) and threading headers.
Reply to a message by id, keeping it in the same thread. Sends immediately. replyAll=true to include all original recipients.
Search messages with Gmail query syntax (from:, subject:, has:attachment, newer_than:7d, is:unread, etc.). Returns up to `limit` (max 100) plus a `nextCursor`; pass it back as `cursor` to page through more.
Send a new email immediately. Saved to Sent. Use gmail_create_draft to review before sending.
Send a previously created draft by id.
Restore a message from Trash back to the mailbox.
Return the signed-in Google account's Gmail profile (email address, message/thread totals).
Parameter constraint descriptions incomplete. maxBodyChars has min/max in schema (0-1000000) but description does not explain the truncation behavior or why 20000 is the default. LLMs cannot reason about appropriate values.
No error handling guidance. Tools do not document what errors are retryable, user-fixable, or fatal. LLMs cannot determine recovery strategy on failure.
gmail_modify_labels description is vague ('Add or remove labels'). Does not explain the semantic difference between removing INBOX (archive) vs removing a custom label, or when to use this vs gmail_mark_read.