Loading ToolBench
Generic MCP server for mail (JMAP) + calendar (CalDAV) + contacts (CardDAV). Works with Fastmail out of the box.
The server implements 31 tools across mail (JMAP), calendar (CalDAV), and contacts (CardDAV) domains. All tools are explicitly registered with descriptions and input schemas using Zod validation. Naming follows verb_noun conventions consistently. Descriptions are generally adequate (100-300 chars) and include usage hints. However, there are notable gaps: (1) Output schemas are NOT documented, tools return JSON-stringified responses without declaring what fields to expect, forcing LLMs to infer structure; (2) Some parameter descriptions lack specificity (e.g., 'Mailbox ID' without format hints); (3) No error handling guidance in tool descriptions, errors are not classified as retryable vs user-fixable; (4) Several tools require ordered dependencies (e.g., 'list_mailboxes first to get ID') documented in text rather than enforced in schema; (5) Tool annotations (readOnlyHint, destructiveHint, idempotentHint) are present and correct, which is a strength. Overall, this is a solid B+ implementation, comprehensive coverage, good naming and schemas, but lacking output documentation and error recovery guidance.
Create a new calendar collection (CalDAV)
Create a new contact (CardDAV)
Create a new contact collection (CardDAV)
Create a new calendar event (CalDAV)
Create a mailbox/folder (label) (JMAP)
Delete a calendar collection (CalDAV)
Delete a contact (CardDAV)
Output schemas not documented. Tools return JSON-stringified responses (e.g., `asText(await sdk.listMailboxes())`) but nowhere in the tool registration is the structure of that output declared. LLMs cannot plan downstream calls or extract specific fields without reverse-engineering the response format.
Dependency documentation is textual, not enforced. Tools like list_emails and move_email include instructions 'You MUST call list_mailboxes first' and 'Call list_mailboxes first' in descriptions, but the schema does not validate that a mailboxId lookup was performed. This invites LLMs to guess or hallucinate IDs.
No error handling guidance. Tool descriptions do not explain what errors might occur, whether they are retryable, or what the LLM should do next. For example, send_email could fail due to invalid addresses, missing subject, or network issues, but the description provides no recovery hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 64 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Delete a contact collection (CardDAV)
Delete an email (moves to Trash) (JMAP)
Delete a calendar event (CalDAV)
Delete a mailbox/folder (label) (JMAP)
Get a download URL for an attachment (JMAP)
Get a contact by id (CardDAV)
Get an email by id (JMAP)
List attachments for an email (JMAP)
Get a calendar event by id (CalDAV)
List calendars (CalDAV)
List contact collections (CardDAV)
List contacts in a contact collection (CardDAV)
List emails from a mailbox (JMAP). You MUST call list_mailboxes first to get the mailbox ID — pass the id field, not the name. Returns {items, position, total, nextPosition}; nextPosition is null at the end.
List events in a calendar (CalDAV)
List mailboxes/folders (JMAP)
Mark an email read/unread (JMAP)
Move an email to another mailbox (JMAP). Call list_mailboxes first to get the target mailbox ID.
Search emails by full-text query (JMAP). Returns {items, position, total, nextPosition}; nextPosition is null at the end.
Send an email (JMAP)
Update calendar properties (name, description, color, timezone) (CalDAV)
Update a contact (CardDAV)
Update a contact collection (CardDAV)
Update a calendar event (CalDAV)
Update mailbox properties (JMAP)
Parameter descriptions lack specificity. Many describe only the field name without explaining valid formats or constraints. E.g., 'Mailbox name' (create_mailbox), is it required? Max length? Special chars? 'IANA timezone' (create_calendar), what happens if invalid? These vague descriptions force LLMs to trial-and-error.
Pagination results structure undefined. list_emails and search_emails mention they return {items, position, total, nextPosition} in the description text, but this structure is not formally documented as the output schema. Tools like list_calendars and list_contacts do not even mention pagination behavior in descriptions.
Destructive tools lack confirmation step. delete_mailbox, delete_email, delete_event, and delete_contact are marked with destructiveHint: true, which is correct, but there is no dry-run or explicit confirmation mechanism. Agents have no way to preview the impact before committing.
Parameter type constraints not enforced via schema. For example, create_calendar's 'color' parameter specifies a regex pattern '^#[0-9A-Fa-f]{6}$', but update_calendar's 'color' parameter does not show this constraint in the schema definition visible in the source. Inconsistency between create and update parameter validation.