Loading ToolBench
An MCP server for Microsoft Outlook integration, providing access to email and calendar functionality through the Microsoft Graph API
The Outlook MCP server defines 3 tools with reasonable structure and schema completeness. All tools have action-verb names (search_*, create_*) and descriptions present. Input schemas are visible and mostly well-formed with type definitions and parameter descriptions. However, output schemas are not documented, the tool responses are not formally declared, making it unclear what fields LLMs should expect downstream. Parameter descriptions are adequate but lack some specificity around constraints (e.g., max_results upper bounds not enforced in descriptions for all tools). Error handling is not visible in the tool definitions. The server demonstrates competent baseline patterns but lacks the polish expected of A-grade tools.
Create a new calendar event.
Search for calendar events based on various criteria.
Search for emails in your mailbox based on various criteria.
Output schemas not documented. Tool responses are not formally declared in the source code. LLMs cannot reliably infer what fields to expect (e.g., what fields does search_calendar_events return per event? What is the shape of the response?). This forces LLMs to guess at response structure and breaks tool chaining when downstream tools require specific fields.
Parameter constraints are incompletely documented. The 'max_results' parameter in search_calendar_events and search_emails declares a max of 100 in the description, but other constraints (min value, default behavior if omitted) are not explicit. Parameter descriptions lack actionable guidance on what happens if constraints are violated.
No error handling or recovery guidance in tool definitions. If a search returns 0 results, what should the LLM do next? If create_calendar_event fails due to a scheduling conflict, what is the error response and how should the agent recover? Tool descriptions do not include error cases or recovery suggestions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 64 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 58 | - | v1 |
Parameter descriptions do not consistently explain dependencies. For example, create_calendar_event has both 'start_datetime' and 'is_all_day', if is_all_day is true, should start_datetime be a date only (YYYY-MM-DD) or a full datetime? The relationship between these parameters is undocumented, forcing LLMs to reason about valid combinations.
Missing pagination guidance in output responses. search_calendar_events and search_emails accept 'max_results' but there is no indication of whether pagination tokens, cursors, or offset/limit patterns are supported for retrieving more results. Large result sets could truncate without warning.