Loading ToolBench
Strong tool naming (all verb-first, clear actions) and comprehensive parameter schemas with proper types, ranges, and enums. Descriptions are detailed and actionable (avg ~150 chars). However, output schemas are not explicitly documented in the source, and error handling guidance is minimal. Tool composition is clean (single responsibility per tool), and parameter constraints are well-specified. Missing: explicit recovery guidance in error scenarios, documented return types, and confirmation patterns for destructive operations (send_sms).
Counts how many SMS parts a message will be split into, accounting for the alphabet and encoding.
Fetches the account with its plan, usage, and limits.
Fetches one phone with its current online status and SIM subscriptions.
Fetches one message by id with its current delivery state.
Shows a send (batch) with the status of every recipient. Use it to follow up on a send_sms call that was still pending.
Lists the phones paired with the account, their online status, and their SIM subscriptions.
Output schemas not documented. Tool descriptions state what is returned (e.g., 'Returns the outcome per recipient') but do not specify the structure of response objects, field types, or pagination cursors. LLMs cannot plan downstream tool calls without knowing what fields to extract.
send_sms is a destructive operation (sends real SMS) but lacks confirmation or dry-run capability. No error guidance for common failures (invalid phone number, rate limit, device offline). LLMs may retry blindly, causing duplicate sends.
Descriptions for get_device, list_devices, and get_account are brief (60-65 chars) and lack context on when to call them vs. alternatives. E.g., get_device description does not explain when to use it instead of list_devices.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 69 | 2025-06-18+ | v2 |
Lists sent and received SMS, newest first by default. Filter by direction, status, phone, send, text, or time range. Follow next_cursor for more.
Sends a text message from the user's Android phone to one or more numbers. Returns the outcome per recipient once the phone reports it, or the pending state if it takes longer than wait_seconds.
Waits for a text message to arrive on the user's phone, optionally from a given number or containing given text. Good for one-time codes and replies. Returns when a match arrives or the timeout passes.
list_messages accepts optional filters (direction, status, device_id, send_id, search, since, until) but does not document mutual exclusivity or precedence. If both 'send_id' and 'device_id' are provided, which takes priority? Undocumented dependencies cause silent misuse.
No error classification or recovery guidance. Tools do not document which errors are retryable (network timeout), user-fixable (invalid phone number), or fatal (account suspended). LLMs cannot decide whether to retry, ask the user, or abort.