Loading ToolBench
MCP server for Microsoft Exchange Online integration via Microsoft Graph API
Exchange Online MCP server has 16 tools with complete input schemas and descriptions. Naming follows verb_noun convention (list_, search_, get_, create_, update_, delete_). Descriptions are present and substantive (100-300 chars typical), exceeding the 10-char minimum. However, several critical gaps reduce the score: (1) Parameters accept both names and IDs but lack explicit enum constraints or validation guidance (e.g., folder parameter accepts 'Inbox' or folder ID with no constraint documentation). (2) Output schemas are documented in code comments but not formally exposed in tool definitions, LLMs cannot see the return structure. (3) Error handling is minimal, no recovery guidance, no categorization of retryable vs. fatal errors. (4) No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels (READ_ONLY, WRITE, DESTRUCTIVE). (5) Destructive tools (delete_email, delete_event) lack confirmation/dry-run patterns. Average tool quality is fair; most tools are well-named and described but lack schema rigor and error guidance expected of production tools.
Archive an email (move to Archive folder). Args: message_id: The message ID to archive. Returns: JSON object with the archived message details. Examples: - archive_email(message_id="AAMkADU1...") - Archive an email
Create a draft email message. Args: subject: Email subject line. body: Email body content (plain text or HTML). to_recipients: List of recipient email addresses. cc_recipients: List of CC recipient email addresses (optional). body_type: Content type - 'text' or 'html' (default: 'html'). importance: Message importance - 'low', 'normal', or 'high' (default: 'normal'). Returns: JSON object with the created draft message details. Examples: - create_draft(subject="Meeting", body="Let's meet tomorrow", to_recipients=["[email protected]"]) - create_draft(subject="Report", body="<h1>Report</h1>", to_recipients=["[email protected]"], body_type="html")
Create a calendar event. Args: subject: Event title. start_date: Start date/time (ISO format: YYYY-MM-DDTHH:MM:SS). end_date: End date/time (ISO format: YYYY-MM-DDTHH:MM:SS). calendar: Calendar name or ID (default: primary calendar). description: Event description (optional). location: Event location (optional). attendees: List of attendee email addresses (optional). is_all_day: Whether this is an all-day event (default: false). Returns: JSON object with the created event details. Examples: - create_event(subject="Meeting", start_date="2024-01-15T10:00:00", end_date="2024-01-15T11:00:00") - create_event(subject="Conference", start_date="2024-02-01", end_date="2024-02-03", is_all_day=True, attendees=["[email protected]"])
Output schemas not formally documented in tool definitions. Code comments describe return structure (format_message, format_event, etc.) but LLMs cannot access these. Tool definitions lack explicit 'returns' schema blocks.
Destructive tools (delete_email, delete_event) lack confirmation/dry-run patterns. No recovery guidance in error responses. Agents can permanently delete without safeguards.
Parameters accepting both names and IDs (folder, calendar, event_id) lack explicit validation constraints or enum guidance. Descriptions say 'Folder name or folder ID' but do not specify format, length, or valid values. LLMs may pass invalid identifiers.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | <=2025-11-25 | v2 |
Delete an email (move to Deleted Items folder). Args: message_id: The message ID to delete. Returns: JSON object with success status. Examples: - delete_email(message_id="AAMkADU1...") - Delete an email
Delete a calendar event. Args: event_id: The event ID to delete. calendar: Calendar name or ID (optional, for context). Returns: JSON object with success status. Examples: - delete_event(event_id="AAMkADU1...") - Delete an event
Get a specific email message by ID. Args: message_id: The message ID to retrieve. Returns: JSON object with full email details including body, attachments info, and metadata. Examples: - get_email(message_id="AAMkADU1...") - Get full email details
Get a specific calendar event by ID. Args: event_id: The event ID to retrieve. calendar: Calendar name or ID (optional, for context). Returns: JSON object with full event details. Examples: - get_event(event_id="AAMkADU1...") - Get event details
Get free/busy schedule information for one or more users. Args: emails: List of email addresses to get schedule for. from_date: Start date/time (ISO format: YYYY-MM-DDTHH:MM:SS). to_date: End date/time (ISO format: YYYY-MM-DDTHH:MM:SS). interval: Availability view interval in minutes (default: 30). Returns: JSON array of schedule information for each user. Examples: - get_schedule(emails=["[email protected]"], from_date="2024-01-15T09:00:00", to_date="2024-01-15T17:00:00") - get_schedule(emails=["[email protected]", "[email protected]"], from_date="2024-01-15", to_date="2024-01-16")
List all calendars in the mailbox. Returns: JSON array of calendars with id, name, and properties. Examples: - list_calendars() - Get all calendars
List calendar events with optional filters. Args: calendar: Calendar name or ID. Default: primary calendar. from_date: Filter events starting after this date (ISO format: YYYY-MM-DD). to_date: Filter events starting before this date (ISO format: YYYY-MM-DD). limit: Maximum number of results (default 25, max 100). Returns: JSON array of calendar events with details. Examples: - list_events() - Get upcoming events from primary calendar - list_events(from_date="2024-01-01", to_date="2024-01-31") - Events in January - list_events(calendar="Work", limit=50) - Get 50 events from Work calendar
List all mail folders in the mailbox. Returns: JSON array of mail folders with id, name, and item counts. Examples: - list_mail_folders() - Get all folders (Inbox, Sent, Archive, etc.)
Mark an email as read or unread. Args: message_id: The message ID to update. is_read: True to mark as read, False to mark as unread. Returns: JSON object with the updated message details. Examples: - mark_email_read(message_id="AAMkADU1...", is_read=True) - Mark as read - mark_email_read(message_id="AAMkADU1...", is_read=False) - Mark as unread
Move an email to a different folder. Args: message_id: The message ID to move. destination_folder: Destination folder name (e.g., 'Archive', 'Drafts') or folder ID. Returns: JSON object with the moved message details. Examples: - move_email(message_id="AAMkADU1...", destination_folder="Archive") - Archive an email - move_email(message_id="AAMkADU1...", destination_folder="Drafts") - Move to drafts
Search for emails with various filters. Args: folder: Folder name (e.g., 'Inbox', 'Archive') or folder ID. Default: Inbox. sender: Filter by sender email address. subject: Filter by subject (contains search). from_date: Filter emails received after this date (ISO format: YYYY-MM-DD). to_date: Filter emails received before this date (ISO format: YYYY-MM-DD). is_read: Filter by read status (true/false). has_attachments: Filter by attachment presence (true/false). limit: Maximum number of results (default 25, max 100). Returns: JSON array of matching emails with key fields. Examples: - search_emails() - Recent emails from Inbox - search_emails(sender="[email protected]") - Emails from specific sender - search_emails(subject="meeting", is_read=False) - Unread emails about meetings - search_emails(from_date="2024-01-01", has_attachments=True) - Recent emails with attachments
Send an email message. Args: subject: Email subject line. body: Email body content (plain text or HTML). to_recipients: List of recipient email addresses. cc_recipients: List of CC recipient email addresses (optional). body_type: Content type - 'text' or 'html' (default: 'html'). importance: Message importance - 'low', 'normal', or 'high' (default: 'normal'). Returns: JSON object with success status and message ID. Examples: - send_email(subject="Hello", body="Hi there", to_recipients=["[email protected]"]) - send_email(subject="Report", body="<h1>Report</h1>", to_recipients=["[email protected]"], body_type="html", importance="high")
Update a calendar event. Args: event_id: The event ID to update. subject: New event title (optional). start_date: New start date/time (optional, ISO format: YYYY-MM-DDTHH:MM:SS). end_date: New end date/time (optional, ISO format: YYYY-MM-DDTHH:MM:SS). description: New event description (optional). location: New event location (optional). calendar: Calendar name or ID (optional, for context). Returns: JSON object with the updated event details. Examples: - update_event(event_id="AAMkADU1...", subject="Updated Meeting") - update_event(event_id="AAMkADU1...", start_date="2024-01-15T14:00:00", end_date="2024-01-15T15:00:00")
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels declared in metadata (READ_ONLY, WRITE, DESTRUCTIVE). Clients cannot infer safety properties from tool definitions.
Error handling minimal. No recovery guidance (e.g., 'User not found. Try search_users() first.'), no error categorization (retryable vs. fatal), no actionable error messages. Agents cannot self-correct on failures.