Loading ToolBench
MCP server for KallyAI API - AI phone assistant integration. Make restaurant reservations, schedule appointments, and handle phone calls through Claude AI.
This server demonstrates solid definition quality with well-structured tool schemas, comprehensive parameter descriptions, and clear naming conventions. All 6 tools are explicitly registered with input schemas and detailed descriptions. Tool names follow verb_noun pattern (create_call, list_calls, get_call, get_transcript, get_subscription, get_statistics). Descriptions are substantial (194-500+ chars) and explain WHAT the tool does, WHEN to use it, and include error handling guidance. Parameter schemas use proper JSON Schema with types, patterns, enum constraints, and min/max bounds. However, there are gaps: (1) output schemas are documented in description text but not formally in JSON Schema, (2) some optional parameters lack clarity on dependencies, (3) error responses are described but not formally structured in the schema, (4) missing explicit indication of idempotency for tools like get_* operations.
Make an AI phone call to a business to accomplish a task such as making a reservation, scheduling an appointment, or asking questions. This tool creates a phone call where an AI assistant will speak with a human at the specified phone number to complete the requested task. The AI can handle restaurant reservations, medical appointments, hotel bookings, and general inquiries.
Retrieve detailed information about a specific phone call by its ID. This tool returns comprehensive information about a call including status, duration, highlights, and next steps.
Retrieve usage statistics for the authenticated user's account. This tool returns information about minutes and calls allocated, used, and remaining, along with overall usage percentage and subscription status.
Retrieve the authenticated user's subscription status and plan details. This tool returns information about the user's current subscription including plan type, minutes included, expiration date, and management URL.
Retrieve the conversation transcript for a specific call. This tool returns a timestamped transcript of the conversation between the AI assistant and the human, showing who said what and when.
Output schemas are documented in prose within tool descriptions but not formally declared as JSON Schema outputSchema in tool registration. LLMs benefit from structured output schema declarations to plan downstream tool calls and extract typed fields reliably.
Error handling is documented in prose (e.g., 'Returns quota_exceeded (402)') but not formally structured as error response schemas. LLMs cannot parse unstructured error guidance, they need categorized errors (retryable, user-fixable, fatal) and actionable next steps in a machine-readable format.
Parameter dependencies are not explicitly documented. For example, appointment_date and appointment_time are optional but together form a semantic unit for appointment_time; time_preference_text is mutually clarifying with appointment_time. LLMs will pass incompatible combinations unless dependencies are stated.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 65 | 2025-06-18+ | v2 |
Retrieve a paginated list of previous phone calls made through KallyAI. This tool returns a history of all calls made by the authenticated user, including their status, business information, and creation timestamps.
get_* tools (get_call, get_transcript, get_subscription, get_statistics) have toolAnnotations.idempotentHint=true set for create_call, but idempotency is not explicitly declared for these read-only tools. While they are read-only and safe to retry, the schema should include idempotentHint for clarity.
Example values in descriptions (e.g., '+14155551234' for phone numbers, '2026-01-28' for dates, 'morning before 11am' for preferences) risk being reused literally by LLMs instead of adapted to context. Replace with enum/pattern constraints or remove examples.