MCP server that connects to the Vitally API to provide customer information, including listing accounts as resources, reading account details, searching for users, and querying account health scores
Vitally MCP has basic tool definitions with schemas and descriptions present, but falls short of production quality. All 6 tools have descriptions (10-80 chars, mostly adequate) and input schemas with type definitions. However, parameter descriptions are minimal or absent, most params lack guidance on format, constraints, or dependencies. Output schemas are undocumented; tools return raw API responses without field documentation. Error handling is absent, no recovery guidance, no categorization of retryable vs fatal errors. The create_account_note tool (WRITE operation) lacks confirmation or dry-run support. No tool annotations (readOnlyHint/destructiveHint) despite clear risk profiles. Naming is verb-first and clear (search_, get_, create_), but parameter naming inconsistencies exist (accountId vs email vs externalId). Overall, definitions are skeletal, sufficient to invoke but insufficient for LLM-driven composition or error recovery.
Vitally tool to create a note for a specific account
Vitally tool to retrieve conversations for a specific account
Vitally tool to get the health score for a specific account
Vitally tool to retrieve tasks for a specific account
Vitally tool to search for available tools by keyword
Vitally tool to search for users by email or external ID
Output schemas undocumented. Tools return raw API responses (VitallyConversation, VitallyTask, VitallyNote interfaces defined in code but not exposed to LLM). LLMs cannot plan downstream calls or extract required fields without documented return types.
Parameter descriptions missing or minimal. search_users has 3 params (email, externalId, emailSubdomain) with no guidance on mutual exclusivity, format, or when to use each. LLMs cannot disambiguate.
No error handling or recovery guidance. callVitallyAPI() throws raw errors ('API call failed: 404') with no actionable next steps. LLMs receive no signal to retry, ask user, or call alternative tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
Destructive tool (create_account_note) lacks confirmation or dry-run. No idempotent hint or confirmation_request pattern. Agent could accidentally create duplicate notes on retry.
No tool annotations. create_account_note should declare destructiveHint=true; read-only tools should declare readOnlyHint=true. LLMs cannot infer safety properties.