Loading ToolBench
Model Context Protocol server for Outlook via Microsoft Graph API
This server uses an aggregate domain tool pattern (verb dispatch) which fundamentally violates single-responsibility and naming clarity principles. All 4 tools (calendar, mail, account, system) accept a generic 'operation' parameter as a free-form string, forcing the LLM to reason about verb dispatch at runtime rather than selecting tools by name. This pattern creates massive naming ambiguity: there is no tool named 'create_event', 'list_events', 'get_message', or 'search_messages', only generic 'calendar' and 'mail' tools that require the LLM to know the internal verb taxonomy. Descriptions are verbose but focus on listing verbs rather than clarifying WHEN to use each tool and what each operation does. Input schemas lack detailed parameter descriptions for the operation parameter itself (no enum of allowed verbs visible in schema, only in prose description). No output schemas documented. Error handling and recovery guidance absent. The pattern trades discoverability and LLM-friendly naming for implementation convenience.
Account management for Microsoft accounts connected to the Outlook MCP server. Aggregate domain tool dispatched by operation verb. Verbs: help, add, remove, list, login, logout, refresh
Calendar operations for Microsoft Outlook via Microsoft Graph. Aggregate domain tool dispatched by operation verb. Verbs: help, list_calendars, list_events, get_event, search_events, create_event, update_event, delete_event, respond_event, reschedule_event, create_meeting, update_meeting, cancel_meeting, reschedule_meeting, get_free_busy
Mail operations for Microsoft Outlook via Microsoft Graph. Aggregate domain tool dispatched by operation verb. By default only read verbs are registered. Additional read verbs (get_conversation, list_attachments, get_attachment) are registered when MailEnabled is configured, and write verbs (create_draft, create_reply_draft, create_forward_draft, update_draft, delete_draft) are registered when MailManageEnabled is configured. Always-on verbs: help, list_folders, list_messages, get_message, search_messages
System diagnostics and authentication utilities for the Outlook MCP server. Start with system.about when troubleshooting; it returns build identity and host environment metadata without requiring authentication. Aggregate domain tool dispatched by operation verb. Always-on verbs: help, status. Optional: complete_auth (when auth_code method is active)
Aggregate domain tool pattern (verb dispatch) violates single-responsibility principle. Tools 'calendar', 'mail', 'account', and 'system' are dispatcher tools that accept a generic 'operation' string parameter and route to internal verbs (create_event, list_events, search_messages, etc.). This forces LLMs to reason about verb taxonomy rather than selecting tools by name, eliminating the primary semantic signal LLMs use to choose tools.
Input schemas lack structured parameter documentation. The 'operation' parameter is documented only as prose in the tool description (e.g., 'Verbs: help, list_calendars, list_events...'), not as an enum constraint in the JSON Schema. This means LLMs cannot parse valid options programmatically and must rely on fuzzy string matching, leading to hallucinated or invalid operation names.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 38 | 2026-07-28+ | v2 |
No output schemas documented. Tools do not declare what fields or structure they return, forcing LLMs to guess what data is available and how to extract it. This prevents proper composition, an LLM cannot know if calendar returns 'event_id' or 'id', blocking downstream tool chaining.
Operation parameters ('operation' field) lack clear description of their constraints and valid values in the schema. The schema shows only type:string with description listing verbs in prose, but no enum array. LLMs treating this as free-form text may generate invalid operation names like 'List_Events' (wrong case) or 'create event' (space instead of underscore).
Tool descriptions are overly long (300+ chars) and serve more as API documentation (verb listing) than LLM selection guides. They do not clearly state WHEN to use each tool, what the most common operations are, or how to decide between similar tools (e.g., when to use 'create_event' vs 'create_meeting' within the calendar tool).
Account parameter documentation warns 'Never assume a default account. Microsoft account label from the connected accounts list. LLM MUST consider disconnected accounts before acting on a specific account.' This is instruction-to-human text, not parameter guidance. It doesn't explain what valid values are, where to get account labels, or how to list them programmatically.
No error handling or recovery guidance visible. No documentation of what errors each tool can return, how to interpret them, or what the LLM should do next (retry, ask user, fail gracefully). Destructive tools (calendar, mail, account all marked DESTRUCTIVE) lack confirmation or dry-run patterns.
Destructive operations (delete_event, cancel_meeting, delete_draft, remove account) are exposed without confirmation or dry-run. An agent error could cause irreversible data loss. No idempotent hints or undo mechanisms documented.