Loading ToolBench
Model Context Protocol server for Fastmail providing access to email messages, contacts, and calendar events
Server has 5 tools with explicit registration, typed schemas, and descriptions. However, several critical gaps reduce overall quality: (1) No output schemas documented, LLMs cannot predict return structure or chain tools effectively. (2) Parameter descriptions exist but lack detail on constraints, formats, and valid ranges. (3) No error handling guidance visible in code. (4) Missing required parameters on most tools (only messages-get marks 'message_id' as required). (5) Tool names are clear and action-oriented (list_, search_, get_), which is strong. Average description length ~90 chars is below the 194-char baseline for A+ tools. No tool annotations (readOnlyHint, etc.) despite all being read-only operations.
List contacts from address book
List calendar events
Get full details of a specific message
List email messages with optional filtering
Search email messages with advanced filtering including date ranges
No output schemas documented for any tool. LLMs cannot predict return structure, breaking tool chaining and context planning.
Parameter descriptions lack constraint details (format, min/max, enum values, regex patterns). E.g., 'date_start' and 'date_end' state 'YYYY-MM-DD format' but lack description text warning about invalid formats. 'sort_by' lists values in description ('receivedAt, sentAt, subject') but should use an enum constraint instead.
Missing 'required' declarations in schema. Only 'messages-get' marks 'message_id' as required; 'account_id' appears to be required for all tools but is not declared, forcing LLMs to guess and risking invalid calls.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 50 | - | v1 |
No error handling documentation or recovery guidance visible in code. No tool returns error classification (retryable, user-fixable, fatal) or actionable error messages.
Tool annotations missing. All five tools are read-only operations but lack 'readOnlyHint: true' annotation, preventing agents from understanding safety properties and planning accordingly.
'messages-search' has 11 optional parameters but provides no guidance on defaults, mutual exclusivity, or which combinations make sense. LLMs may pass conflicting filters (e.g., both 'read: true' and 'read: false') without knowing the behavior.
Descriptions are generic and brief (avg ~90 chars, below 194-char baseline). E.g., 'contacts-list' is 'List contacts from address book', lacks context for when to use it vs. other discovery tools, what structure is returned, or how to chain it with other operations.