Loading ToolBench
Cosmic-grackle demonstrates solid definition quality with complete schemas, clear descriptions, and thoughtful parameter design. All 8 tools have explicit input schemas with typed properties and descriptions. Tool names follow verb_noun convention (contacts_list, contacts_search, contacts_create, etc.). Descriptions are substantive (100-200 chars typical). Input validation is comprehensive with length checks for all string fields. However, output schemas are not documented, LLMs cannot see what fields to expect from responses. Error handling is present but generic (McpError::invalid_params). No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels. The contacts_update tool has complex parameter relationships (phone_from/phone_to, email_from/email_to) that are well-documented but could benefit from explicit mutual-exclusivity guidance.
Create a new contact in the macOS Contacts app.
Delete a contact from the macOS Contacts app.
Get a specific contact by ID.
List all contacts or a limited number of contacts from the macOS Contacts app.
Search for contacts by name query.
Update an existing contact in the macOS Contacts app.
List all contact groups in the macOS Contacts app.
Output schemas not documented. LLMs cannot see what fields contacts_list, contacts_search, contacts_get, contacts_create, contacts_update, or groups_members return. This forces agents to guess at response structure and breaks tool chaining.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). contacts_delete is marked DESTRUCTIVE and contacts_create/contacts_update are WRITE, but the tool definitions lack explicit annotations. Agents cannot infer safety properties from the schema alone.
contacts_update has complex parameter relationships (phone_from/phone_to, email_from/email_to, url_from/url_to, postal_from/postal_to) with omit-one semantics. While documented in descriptions, no explicit mutual-exclusivity constraint or enum-based state machine. LLMs may pass invalid combinations.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | <=2025-11-25 | v2 |
Get all members of a contact group by group name.
Error messages are generic (McpError::invalid_params with a message). No recovery guidance. When contacts_search fails, the agent does not know whether to retry, adjust the query, or call a different tool.
contacts_list and groups_list lack pagination parameters (limit, offset, cursor). If a user has thousands of contacts, the tool returns all of them, risking context window exhaustion. No documented result cap.