Loading ToolBench
Office 365 calendar assistant powered by Claude Code and Microsoft Graph
Office Assistant has well-structured tool definitions with clear naming (verb_noun pattern) and comprehensive parameter schemas. All 7 tools have descriptions and typed input schemas. However, output schemas are not documented, error handling guidance is absent, and several parameter descriptions lack actionable constraints (e.g., timezone format, datetime validation rules). Descriptions average ~150 chars, which is good, but lack dependency hints and recovery guidance. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels. The create_event tool is complex with 14 parameters but lacks documentation of mutually exclusive constraints (recurrence_end_date vs recurrence_count).
Create a new calendar event.
Find available meeting times that work for a group of people. Microsoft Graph checks each attendee's calendar and suggests times where everyone is available. Returns suggestions sorted by confidence.
Get free/busy schedule for one or more people. Returns each person's availability status across the time range, broken into slots of the specified interval.
Get the authenticated user's display name, email address, and timezone. Use this to understand who is currently logged in and what timezone they work in. Call this before other tools if you need the user's timezone for date calculations.
List all calendars the authenticated user has access to. Returns each calendar's name, ID, owner, and whether it can be edited. Use this when the user asks "which calendars do I have?" or wants to know about shared calendars.
Output schemas not documented. Tools return data but LLMs cannot plan downstream calls without knowing response structure (e.g., what fields does get_free_busy return? Is it a list of slots with status? Does it include confidence scores?). This violates pattern:tool and forces LLMs to guess.
create_event has 14 parameters with undocumented mutual exclusivity. recurrence_end_date and recurrence_count cannot both be set; recurrence_days_of_week is required only for weekly patterns. These constraints must be explicit in parameter descriptions to prevent LLM errors.
No error handling guidance. Tools lack descriptions of failure modes, recovery steps, or actionable error messages. E.g., if find_meeting_times returns no candidates, should the LLM retry with a longer window or different attendees? No guidance provided.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
List calendar events for a date range.
List available meeting rooms in the organisation. Returns room names, email addresses, and capacity. Use the email address with ``room_emails`` in ``create_event`` to book a room. Only works with work/school Microsoft 365 accounts.
Timezone parameter descriptions lack format specification. 'IANA timezone (e.g. Europe/London)' is informal. Should state: 'IANA timezone identifier (e.g., Europe/London, America/New_York). Invalid timezones will return an error.' This prevents LLM hallucination of invalid zones.
No tool annotations. create_event is destructive (WRITE risk) and should carry destructiveHint=true. get_free_busy, list_calendars, list_events, list_rooms are read-only and should carry readOnlyHint=true. Annotations guide agent safety policies.