Loading ToolBench
MCP server for Microsoft 365 administrative tasks with multi-tenant support
The server provides 14 tools with generally present descriptions and schemas, but exhibits significant inconsistencies in quality. All tools have descriptions (10 - 336 characters, baseline 194), and input schemas are defined with proper types and parameter descriptions. However, output schemas are not documented, the server does not specify what fields are returned, their types, or structure, forcing LLMs to infer output shape. Tools like list_users and list_distribution_lists accept rich filtering options and mention pagination behavior in descriptions, but no explicit pagination parameters (page, offset, limit, next_cursor) are defined in the input schemas. Destructive operations (delete_office365_group) lack confirmation/dry-run patterns. Parameter naming is generally good (verb_noun, use of action verbs), but some tools conflate multiple responsibilities (schedule_meeting combines finding times and scheduling into two separate tools, good, but send_email handles both plain text and attachments in one call). Security concerns: credentials are not exposed as parameters (good), but error handling is minimal, no recovery guidance, no categorization of retryable vs. fatal errors, no validation error messages to help LLMs self-correct. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite having read-only and destructive operations.
Add a member to a distribution list in an M365 tenant.
Create a new Office 365 group in an M365 tenant.
Delete an Office 365 group from an M365 tenant.
Find available meeting times for a group of attendees in an M365 environment.
Get members of a specific distribution list in an M365 tenant with filtering and selection options.
List all configured M365 tenant profiles. Returns: A formatted string listing all available profiles
Output schemas not documented. No tool specifies what fields are returned, their types, or structure. LLMs must infer output shape, risking misplanning of downstream calls.
Pagination not formalized in schema. Tools mention pagination behavior in text descriptions ('Without top parameter: Automatically retrieves ALL users') but do not declare explicit page/offset/limit/next_cursor parameters in the input schema. This forces LLMs to guess how pagination works.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 62 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
List distribution lists (groups) in an M365 tenant with comprehensive filtering and selection options. NOTE: This tool lists the distribution lists themselves, NOT their members. To get the users that belong to a specific distribution list like '[email protected]', you would need a separate group membership expansion tool. Args: profile: The M365 tenant profile to use. If not specified and only one profile is configured, that profile will be used automatically. select: Properties to include (e.g., ['displayName', 'mail', 'mailEnabled']) filter: OData filter expression (e.g., "mailEnabled eq true and securityEnabled eq false") search: Search string (e.g., 'displayName:marketing') orderby: Properties to sort by (e.g., ['displayName']) top: Maximum number of results to return (if not specified, returns ALL distribution lists) count: Whether to include total count in response expand: Related objects to expand (e.g., ['members']) Returns: JSON string with distribution lists data and metadata. Does NOT include member lists. Pagination Behavior: - Without 'top' parameter: Automatically retrieves ALL distribution lists in the tenant - With 'top' parameter: Returns only the specified number of results with pagination info
List messages in a user's inbox in an M365 environment with filtering and selection options.
List users in an M365 tenant with comprehensive filtering and selection options. Args: profile: The M365 tenant profile to use. If not specified and only one profile is configured, that profile will be used automatically. select: Properties to include (e.g., ['displayName', 'mail', 'userPrincipalName']) filter: OData filter expression (e.g., "endswith(mail,'@contoso.com')") search: Search string (e.g., 'displayName:admin') orderby: Properties to sort by (e.g., ['displayName']) top: Maximum number of results to return (if not specified, returns ALL users) count: Whether to include total count in response expand: Related objects to expand (e.g., ['manager']) Returns: JSON string with users data and metadata Pagination Behavior: - Without 'top' parameter: Automatically retrieves ALL users in the tenant - With 'top' parameter: Returns only the specified number of results with pagination info
Read data from an Excel worksheet in OneDrive/SharePoint in an M365 environment.
Remove a member from a distribution list in an M365 tenant.
Schedule a meeting in the calendar of a user in an M365 environment.
Send an email on behalf of a user in an M365 environment.
Update data in an Excel worksheet range in OneDrive/SharePoint in an M365 environment.
Destructive tool (delete_office365_group) lacks confirmation or dry-run pattern. No safety gate prevents accidental permanent deletion. No error categorization (retryable vs. fatal) to guide agent recovery.
No tool annotations. None of the 14 tools declare readOnlyHint, destructiveHint, or idempotentHint metadata, despite having clear read-only (list_*, get_*) and destructive (delete_*) semantics. This prevents clients from enforcing safety policies.
Error handling does not guide recovery. No tool specifies what errors are retryable, what remediation steps the LLM should take, or how to interpret error responses. Example: 'authentication failed' does not tell the agent whether to retry, reload credentials, or alert the user.
Parameter naming inconsistency. Filters accept 'filter' (OData), 'search' (text search), 'orderby' (array), 'top' (limit), but no explicit 'skip' or 'offset' for pagination. Callers must infer offset behavior or rely on stateful pagination.
list_configured_profiles returns minimal documentation, no indication of what profile structure contains (tenant_id? client_id? visibility?), forcing LLMs to call it purely to discover available options with no upfront understanding of the data.
Parameter 'expand' in list_* tools mentions examples ('manager', 'members') but does not specify what expansions are valid. LLMs may hallucinate expansion names, causing API errors.
send_email description is terse (29 chars: 'Send an email on behalf of a user in an M365 environment.'). Does not explain when to use it vs. other email tools, what happens on success/failure, or whether it supports templates.