Loading ToolBench
jmap-mcp demonstrates strong schema definition and comprehensive parameter documentation across 11 tools. All tools have descriptions between 50-400 chars (within ideal range). Parameter schemas are complete with types, descriptions, and constraints (min/max, enums, formats). Critical strength: email search/manipulation tools include detailed pagination guidance and property selection advice. Key weakness: no explicit error handling guidance or recovery patterns in descriptions; no tool annotations (readOnlyHint/destructiveHint); destructive operations (delete_emails) lack confirmation/dry-run support. Output schemas are not explicitly documented in tool definitions themselves, requiring LLM to infer from JMAP spec. Average tool quality: 73/100.
Permanently delete emails. This moves them to the Trash mailbox if a Trash folder exists, otherwise permanently removes them.
Retrieve changes to emails since a previous state. Returns created, updated, and destroyed email IDs. Use with search_emails queryState to track incremental changes. Optionally fetch full email details for changed messages.
Retrieve full details for specific email IDs. Specify properties to optimize response size. When requesting 'bodyValues', you MUST also request either 'textBody' or 'htmlBody' (or both) to get the actual content — bodyValues alone won't return readable text. Returns notFound array for IDs that don't exist. Includes state for use with get_email_changes.
Retrieve mailboxes (folders). Paginated results. Use position and limit to navigate pages.
Get updates to a previous search query. Returns added and removed email IDs since the previous query state. Use the same filter parameters as the original search_emails call. Useful for maintaining an incremental view of search results.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). LLM cannot distinguish safe read operations from destructive writes without relying solely on naming conventions. delete_emails, move_emails, mark_emails, send_email should declare risk profiles.
Destructive operations (delete_emails) lack confirmation/dry-run support or explicit error recovery guidance. Description states 'moves to Trash' but does not warn LLM about permanent deletion if Trash does not exist, or how to recover from accidental deletion.
Output schemas not documented in tool definitions. While parameter input schemas are comprehensive, there is no explicit declaration of response fields (e.g., search_emails returns {ids, total, position, hasMore, queryState} structure). LLM must infer from JMAP RFC or experiment.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 69 | 2026-07-28+ | v2 |
| 2026-03-09 | C | 64 | - | v1 |
Retrieve threads by ID. Each thread contains a list of email IDs in conversation order.
Set keyword flags on emails (read/seen status, flagged status, etc.). Pass true/false for 'seen' and 'flagged' to set or unset those flags.
Move emails to a different mailbox (folder). Removes from all other mailboxes.
Reply to an existing email. Automatically sets correct To/CC, subject (Re: prefix), and threading headers (In-Reply-To, References). Use replyAll=true to include all original recipients. The identityId parameter is optional — if omitted, the server uses the default sending identity.
Search emails with filters (text, sender/recipient, dates, keywords). All filters are AND'd together. Returns only email IDs — use get_emails to fetch full content. For listing emails, request only the properties you need (e.g., ['id', 'subject', 'from', 'receivedAt', 'preview'] for a summary). Results are paginated: each response includes `total` (total matching emails), `position` (current offset), and `hasMore` (boolean). To get the next page, call again with `position` set to the current `position + ids.length`. Do NOT fetch all pages unless explicitly asked — the first page is usually sufficient. Also returns `queryState` for incremental sync via get_search_updates.
Send a new email. Requires either textBody or htmlBody (or both). The identityId parameter is optional — if omitted, the server uses the default sending identity.
No error recovery guidance in tool descriptions. When a tool fails (e.g., 'email not found', 'mailbox does not exist'), descriptions do not advise LLM on recovery steps (call search_emails, check mailbox list, etc.).
get_threads description is minimal (14 words). Does not explain when to use get_threads vs. searching by thread context, or clarify what 'conversation order' means.
identityId parameter in send_email and reply_to_email marked optional with minimal guidance. Description says 'if omitted, the server uses the default sending identity' but does not explain how LLM discovers available identities or when to override the default.