Loading ToolBench
Windows-only MCP server that exposes local Outlook (COM) capabilities to Claude via stdio. Supports email listing, sending, calendar event management without requiring Azure App Registration or Microsoft Graph.
This server has 8 tools with generally complete input schemas and descriptions, but several patterns indicate mediocre production readiness. All tools have descriptions and input schemas are present for all 8 tools, which is above baseline. However, descriptions are sparse (most 60-120 chars, below the 194-char production average), parameter descriptions lack detail on validation rules, and error handling/recovery guidance is absent. No tool annotation hints (readOnlyHint/destructiveHint/idempotentHint) are visible. Output schemas are not documented. The tools themselves are well-named with action verbs (list_, send_, create_, scan_) and cover a coherent domain (Outlook email/calendar), but the implementation lacks the depth expected for a 70+ grade. Most parameters have types but lack constraint details (ranges, patterns, dependencies). This lands solidly in the 'fair' to 'poor' range, above baseline for having schemas at all, but well below production-grade tool definitions.
Create a calendar event/meeting in local Outlook. Use ISO datetimes.
List all Outlook accounts/stores configured
List emails from a local Outlook folder. Supports inbox/sent/drafts with filtering options.
List emails matching a regex pattern
List calendar events in a date range from local Outlook.
List all folders in Outlook with unread count
Scan all folders for unread emails
Output schemas not documented. Tool descriptions state WHAT is returned (e.g., 'List emails from a local Outlook folder') but do not specify the structure of returned objects (fields, types, pagination). LLMs cannot determine what fields to extract for downstream tool calls or how to compose results.
Parameter descriptions lack validation rules and constraints. E.g., 'top' parameter states 'Maximum number of emails to return' but does not explicitly state minimum=1, maximum=200 in description text (though JSON Schema has the constraints). 'query' parameter lacks guidance on whether it is substring match, regex, or exact match. 'profile' parameter does not explain what happens if the profile does not exist.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 28 | - | v1 |
Send an email using local Outlook. Supports text or HTML body and attachments.
No error handling or recovery guidance in tool descriptions. E.g., 'send_email' does not state what happens if a recipient address is invalid, if attachments do not exist, or if Outlook is not running. 'create_event' does not explain what happens if requiredAttendees are not valid Outlook contacts. LLMs cannot self-correct without guidance.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) visible. This forces LLMs to infer whether a tool is safe to call multiple times, modifies state, or reads safely. Risk assessment tools like 'send_email' and 'create_event' should carry destructiveHint=true to signal irreversibility.
Parameter descriptions for 'profile' are inconsistent and vague. Multiple tools accept a 'profile' parameter, but the description says 'Outlook profile/account name (optional)' without explaining: what is the expected format? What happens if the default profile is used? Can agents list valid profiles? The 'list_accounts' tool exists, but tools don't reference it as a discovery step.
No pagination documented for list tools. 'list_emails', 'list_events', 'scan_unread_emails', and 'list_emails_by_pattern' all accept 'top' to limit results, but there is no 'offset' or 'cursor' parameter and no mention of whether results are paginated or truncated. Large result sets could blow the context window without pagination guidance.
Generic parameter name 'pattern' in 'list_emails_by_pattern' lacks constraint documentation. The description says 'Regular expression pattern to match against Subject' but does not explain: is it case-sensitive? Does it support all regex features, or a subset? What happens if the regex is invalid? The parameter name shadows a common variable name, which may confuse LLMs.
Composition issue: 'scan_unread_emails' and 'list_emails' with 'unreadOnly=true' overlap significantly. LLMs may not know which to prefer. The tools should either be consolidated or have clear distinctions in their descriptions (e.g., 'scan_unread_emails searches all folders in one call; use list_emails for a specific folder').