Loading ToolBench
Disposable email inbox service for testing. Create inboxes, receive emails, extract verification links.
TestMailbox has 5 tools with clear verb-noun naming (create_inbox, check_inbox, wait_for_email, get_email_links, delete_inbox). All tools have descriptions (avg 180 chars, within 10-1024 range). Input schemas are present with type definitions for all parameters. However, parameter descriptions are minimal (avg 40 chars vs baseline 72), and output schemas are not formally documented. Error handling returns dict with 'error' key but lacks recovery guidance. No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels. Descriptions are functional but lack LLM-optimization details (e.g., wait_for_email doesn't explain regex syntax or timeout behavior clearly).
List all messages in a disposable inbox. Args: inbox_id: The inbox ID returned from create_inbox Returns: dict with inbox_id, count, and list of messages (newest first)
Create a new disposable email inbox. Returns an inbox with a random email address like [email protected]. Inboxes auto-expire after 1 hour. Returns: dict with id, email, created_at, expires_at
Delete a disposable inbox and all its messages. Args: inbox_id: The inbox ID to delete Returns: dict with status
Extract all links from the latest email in an inbox. Perfect for finding verification links, password reset URLs, etc. Args: inbox_id: The inbox ID Returns: dict with inbox_id and list of links
Wait for an email matching subject/from pattern to arrive in the inbox. Polls the inbox every second until a matching email arrives or timeout. Args: inbox_id: The inbox ID to watch subject_pattern: Optional regex to match against email subject from_pattern: Optional regex to match against sender address timeout: Maximum seconds to wait (1-120, default 30) Returns: The matching message dict, or error if timeout
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. delete_inbox is DESTRUCTIVE, check_inbox/wait_for_email/get_email_links are READ_ONLY, create_inbox is WRITE. LLMs cannot infer safety properties without explicit hints.
Output schemas not formally documented. Tools return dicts with fields like 'inbox_id', 'count', 'messages', 'links', 'error' but no JSON Schema definition. LLMs cannot plan downstream calls without knowing response structure.
Error handling lacks recovery guidance. Errors return {'error': 'message'} but do not suggest next steps. E.g., 'Inbox not found' should suggest 'Call create_inbox() to create a new inbox.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Parameter descriptions are minimal (avg 40 chars vs baseline 72). 'inbox_id' described as 'The inbox ID to watch' lacks context on format, origin, or lifetime. 'timeout' lacks explicit range validation text (1-120 is in description but not emphasized).
wait_for_email accepts regex patterns but does not document regex syntax, flags, or error handling for invalid patterns. LLMs may pass invalid regex and receive cryptic errors.