Loading ToolBench
Server has reasonably detailed tool definitions with KQL-aware descriptions and parameter schemas, but several quality gaps prevent a higher score. All 6 tools have non-empty descriptions (good baseline), and most parameters include types and descriptions. However, output schemas are not explicitly documented, parameter descriptions vary in actionability, and error handling guidance is absent from tool definitions. The search_messages tool has exceptional documentation of KQL syntax, but others are more generic. No tool descriptions explain what happens on error or how to recover.
Check the authentication status of the Microsoft Graph connection. Returns whether the user is authenticated and shows their basic profile information.
Create a new chat conversation. Can create 1:1 chats or group chats with multiple participants.
Delete a message from a channel.
Delete a message from a chat.
Download images or other hosted content from messages. Returns base64 encoded content.
Retrieve replies to a specific message in a channel.
Retrieve recent messages from a specific channel in a Microsoft Team. Returns message content, sender information, and timestamps.
No output schemas documented. Tool definitions describe input parameters but omit what fields are returned. LLMs cannot reason about downstream tool chains or data extraction without knowing return structure.
No error handling guidance in tool descriptions. Descriptions state what tools do but omit recovery paths (e.g., 'if user not found, try search_users'). This forces LLMs to guess on failure.
Parameter descriptions vary in specificity. 'Query string' and 'Search query' are generic; 'Search query string. Supports KQL syntax...' (search_messages) is exemplary. Standardize across all tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 16 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 48 | - | v1 |
Retrieve recent messages from a specific chat conversation. Returns message content, sender information, and timestamps.
Get information about the current authenticated user.
Find recent messages where the current user was @mentioned across all Teams channels and chats.
Get information about a specific user by ID or email address.
List all channels in a specific Microsoft Team. Returns channel names, descriptions, types, and IDs for the specified team.
List all recent chats (1:1 conversations and group chats) that the current user participates in. Returns chat topics, types, and participant information.
List all members of a specific Microsoft Team with their roles and display names.
List all Microsoft Teams that the current user is a member of. Returns team names, descriptions, and IDs.
List users in the organization with optional filtering by name or email.
Send a reply to a message in a channel thread.
Search for messages across all Microsoft Teams channels and chats using the Microsoft Search API. The query string supports KQL (Keyword Query Language) syntax for advanced filtering: from:<name> — messages sent by a person (e.g. from:bob) to:<name> — messages sent to a person mentions:<userId> — messages that mention a specific user ID (without dashes) IsMentioned:true — messages that mention the current user hasAttachment:true|false — filter by attachment presence IsRead:true|false — filter by read status sent>=YYYY-MM-DD — messages sent on or after a date sent<=YYYY-MM-DD — messages sent on or before a date Examples: "quarterly report" from:alice sent>=2025-01-01 hasAttachment:true from:bob project update sent>=2025-02-01 Use get_chat_messages or get_channel_messages for browsing a specific conversation.
Search for users by name or email to get their IDs for @mentioning in messages. Useful for finding the correct user ID before composing a message with mentions.
Send a message to a specific channel in a Microsoft Team. Supports text and markdown formatting, mentions, and importance levels.
Send a message to a specific chat. Supports text and markdown formatting, @mentions, file attachments, and importance levels.
Update the content of a previously sent channel message.
Update the content of a previously sent chat message.
get_user accepts 'user ID or email address' as one parameter but does not separate them into user_id vs user_email. Overloaded parameters force LLMs to reason about type disambiguation.
search_messages returns messages but no visible documentation of result structure (fields like timestamp, sender, channel_id, message_id). Downstream tools (e.g., reply to message, get context) need these fields.