Loading ToolBench
13 tools with reasonable naming and parameter schemas, but inconsistent description quality and output schema documentation. Most tools have good parameter constraints via Zod schemas (converted to JSON Schema). However, several tools lack context-specific guidance, and the server is missing output schema documentation for critical tools. Error handling is minimal. Tool names follow verb_noun pattern well (filter_emails, send_email, etc.), but descriptions vary significantly in depth and actionability. Pagination support is present but not uniformly documented across list tools.
Moves messages to the Archive folder (Gmail) or equivalent.
Batch triage update: flags + folder moves in one call.
Creates a new email folder.
Deletes a user-created email folder.
Creates a new email draft without sending it.
Filter messages in a single logical AND clause. • Use *only* the keys provided in the schema. • The filter is **AND‑ed** across all supplied keys; "OR" logic is not supported here. • For OR / NOT / advanced date math, call **search_emails_native** instead. Examples --------- ✓ Get unread inbox mail: { "folderName": "Inbox", "unread": true, "limit": 20 } ✓ All starred messages across every folder, newest first: { "starred": true, "limit": 50 } Pagination ----------- If the response includes "NEXT_PAGE_TOKEN: …", pass that value back in "pageToken" to fetch the next page.
Output schemas not documented. Tools like send_email, draft_email, read_emails, and others do not have explicit documentation of their return types. LLMs cannot infer what fields are available in responses, forcing them to guess at field names and potentially make invalid downstream tool calls.
list_emails_for_triage has null/missing parameter descriptions. Three parameters (unreadOnly, starredOnly, olderThan) have description: null, which violates the requirement that every parameter must have a non-empty description. This forces the LLM to infer meaning from names alone.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 69 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Finds a folder by name, or creates a new one if no match is found.
Pages through the inbox for triage.
Retrieves all available email folders (previously called labels).
Retrieves the full content and metadata of specific emails by their IDs.
Search emails. IMPORTANT: ONLY SINGLE WORD QUERIES ARE ALLOWED. just basic text search. Use pagetoken to paginate.
Composes and sends a new email immediately.
Updates an existing folder's name.
Descriptions too brief or generic for several tools. send_email, draft_email, read_emails, create_folder, update_folder, delete_folder, and batch_archive_emails have descriptions under 100 characters (e.g., 'Composes and sends a new email immediately.' is 51 chars). These lack context about prerequisites, error conditions, or when to use one tool vs. another.
No error handling guidance in tool descriptions. Error responses are not documented. For example, send_email does not explain what happens if an email address is invalid, a recipient rejects the message, or the API rate limit is hit. LLMs have no recovery path.
Pagination design inconsistency. filter_emails returns NEXT_PAGE_TOKEN, but search_emails uses pageToken as both input and output; list_emails_for_triage does the same. No tool explicitly states the total count of results or whether a next_page_token indicates more results. This forces LLMs to guess when to stop paginating.
No dry-run or confirmation for destructive operations. delete_folder and batch_archive_emails are destructive but do not offer a confirmation step. Agents cannot preview or validate before execution, risking accidental data loss.
search_emails constraint 'ONLY SINGLE WORD QUERIES' is enforced in description but not in schema. No regex pattern or enum constraint prevents multi-word queries. The description reads as a disclaimer but the schema does not validate, forcing runtime rejection.
batch_triage_emails accepts mutually exclusive parameters without documentation. setUnread, setStarred, moveToFolderId, addFolderIds, removeFolderIds are all optional but some combinations may be invalid (e.g., both moveToFolderId and addFolderIds). The schema does not enforce mutual exclusivity, and descriptions do not clarify the semantics.
No field-name normalization guidance. read_emails returns messageIds (plural array), but some tools may accept message_id or message ID. No response example or schema clarifies the exact field names agents should use in downstream calls.