Loading ToolBench
Multi-account mail infrastructure: IMAP + Microsoft Graph, web GUI, MCP agent API
Lenovmail demonstrates solid tool design with clear verb-based naming (list_, search_, get_, set_, move_, delete_, send_), comprehensive parameter schemas, and good descriptions. All 10 tools have non-empty descriptions (avg ~120 chars) and typed input schemas. However, output schemas are not documented in the visible code, parameter descriptions lack constraint details (ranges, formats, enums), and error handling guidance is minimal. The server correctly implements scope-based access control and audit logging, but lacks recovery hints in error messages. Naming is consistent and follows verb_noun convention well.
Delete one or more messages permanently.
Fetch one attachment's bytes, capped at 1MB.
Fetch one message with its text/HTML body and attachment metadata.
Fetch every message in a conversation (thread), ordered by date.
List the mail accounts reachable with this token, with status and message counts.
List an account's folders (inbox, sent, etc.) with message counts.
Move one or more messages to a folder.
Output schemas not documented in source code. LLMs cannot plan downstream tool calls or extract required fields without knowing response structure.
Parameter descriptions lack constraint details. 'limit' accepts 1-100 but description does not state range. 'cursor' is opaque with no format guidance. Numeric bounds and format patterns must be explicit.
Error handling lacks recovery guidance. ToolError exceptions are raised but do not suggest next steps (e.g., 'User not found. Try list_accounts() first'). Errors should be actionable for LLM retry logic.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | 2026-07-28+ | v2 |
Search messages in one account. `query` uses full-text search (subject, body, sender, recipients). Optional filters: folder_id, unread, flagged, has_attachments. Continue to the next page with `cursor`.
Send a message through the outbox queue. If the token requires approval, the message stops at `pending_approval` status until a human approves it in the GUI.
Set or clear message flags (read, flagged, etc.) on one or more messages.
Destructive operations (delete_messages, send_message) lack confirmation/dry-run pattern. Agents can permanently delete or send emails without a safety gate. Consider adding a 'dry_run' parameter or separate 'confirm_delete' step.
set_flags tool description does not clarify which flags are settable or what happens when both 'read' and 'flagged' are omitted. Undocumented parameter relationships cause silent misuse.