Loading ToolBench
A Model Context Protocol server providing tools for CRUD operations for the mac-os calendar
Server has 4 tools with generally adequate naming and descriptions, but significant gaps in schema completeness, parameter documentation, and error handling. Tool names follow verb_noun convention (list_calendars, create_event, etc.), which is good. Descriptions are present and moderately detailed (range 70-280 chars), meeting the 10-1024 char baseline. However, parameter schemas show inconsistent typing: list_calendars has empty input {}, list_events and create_event use proper datetime/string types, but complex object parameters like 'recurrence_rule' lack detailed subfield documentation. Output schemas are entirely undocumented, callers cannot predict response structure. Error handling is minimal (generic try-catch returning string messages rather than structured errors with recovery guidance). The server uses macOS-specific Calendar framework, limiting portability.
Create a new calendar event. Before using this tool, make sure to: 1. Ask the user which calendar they want to use if not specified (check calendars://list) 2. Ask if they want to add a location if none provided 3. Ask if they want to add any notes/description if none provided 4. Confirm the date and time with the user 5. Ask if they want to set reminders for the event
List all available calendars.
List calendar events in a date range. The start_date should always use the time such that it represents the beginning of that day (00:00:00). The end_date should always use the time such that it represents the end of that day (23:59:59). This way, range based searches are always inclusive and can locate all events in that date range.
Update an existing calendar event. Before using this tool, make sure to: 1. Ask the user which fields they want to update 2. If moving to a different calendar, verify the calendar exists using calendars://list 3. If updating time, confirm the new time with the user 4. Ask if they want to add/update location if not specified 5. Ask if they want to add/update notes if not specified 6. Ask if they want to set reminders for the event
list_calendars tool has empty input schema {} with no parameters documented, preventing LLMs from understanding tool inputs. Baseline requires every parameter to have type and description.
Output schemas are completely undocumented across all tools. Callers cannot predict response structure (string vs object vs array), preventing proper downstream tool chaining and forcing LLMs to parse unstructured text.
Complex object parameter 'recurrence_rule' in create_event and update_event lacks detailed subfield documentation. Description mentions 'RecurrenceRule' type but provides only prose-form field list; no schema constraints (required fields, enum values for frequency, valid range for interval). LLMs cannot validate or construct valid recurrence rules.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 57 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 56 | - | v1 |
Error handling returns generic string messages ('Error listing calendars: {str(e)}') with no error classification, recovery guidance, or actionable detail.
list_events and create_event lack explicit pagination support and result limits. list_events could return arbitrarily many events; create_event accepts array reminder_offsets with no documented size limit.
Parameter 'calendar_name' in list_events, create_event, and update_event is optional but underdocumented. Description does not explain behavior when omitted (defaults to current calendar? all calendars?). No validation constraint; LLM could pass invalid calendar names causing silent failures.
No idempotency guarantees or dry-run support for destructive operations (create_event, update_event are WRITE operations).
Permission/credential model is implicit, not explicit. Tool definitions do not declare required scopes (read:calendar, write:calendar).