Loading ToolBench
SMTP for AI agents. No middleman. Self-hosted mail server (AI Mail Exchange) with built-in MCP.
aimx provides 7 tools with explicit schemas and descriptions. Naming follows verb_noun conventions (mailbox_list, mailbox_create, hook_create, send). All tools have descriptions (10-250 chars). Schemas are properly defined with types and per-parameter descriptions. However, several gaps prevent a higher score: (1) Output schemas are not documented, tools return structured objects but the response structure is not declared, forcing LLMs to infer it; (2) Error handling lacks recovery guidance, tools note what they do but don't explain what to do if permission is denied or a mailbox doesn't exist; (3) Some parameters could be more constrained (e.g., 'event' enum is good, but 'timeout_secs' range [1, 600] is mentioned in description, not in schema); (4) Composition is strong (tools are single-purpose), but tool chaining metadata is missing (e.g., hook_create doesn't mention that mailbox_list must be called first to discover valid mailbox names).
Create a new hook on a mailbox. Non-root callers must own the target mailbox. Atomically rewrites config.toml and swaps in-memory Config snapshot. Hooks on catchall mailboxes are forbidden.
Delete a hook by its effective name. Non-root callers must own the mailbox the hook is attached to. Atomically rewrites config.toml and swaps in-memory Config snapshot. Returns ENOENT with no information leak if hook exists on a mailbox the caller doesn't own.
List hooks visible to the caller. Root sees every hook on every mailbox; non-root sees only hooks on mailboxes it owns. Returns JSON array of HookListRow objects.
Create a new mailbox. Non-root callers create mailboxes owned by themselves; root can specify an owner. Atomically writes config.toml and swaps in-memory Config snapshot.
Delete a mailbox. Non-root callers must own the target mailbox. Atomically rewrites config.toml and swaps in-memory Config snapshot.
List mailboxes visible to the caller. Root sees every mailbox; non-root sees only mailboxes it owns.
Output schemas are not documented for any tool. Tools return structured objects (HookListRow for hook_list, presumably email message_id for send), but the response schema is never declared. LLMs cannot predict what fields are available, forcing them to hallucinate field names.
Destructive operations (mailbox_delete, hook_delete) lack confirmation or dry-run support. No pattern to prevent irreversible mistakes. LLM-invoked deletions are one-way, no recovery.
Error handling provides no recovery guidance. Tools describe what they do but not what happens when things fail. E.g., if mailbox_create fails because the name already exists, the error message is not documented, LLM doesn't know whether to retry, ask user, or suggest a different name.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | <=2025-11-25 | v2 |
Send an email. Pure UDS client that never reads config.toml. The daemon parses the From: header and resolves the sender mailbox against its in-memory Config.
Tool chaining dependencies are undocumented. hook_create accepts a mailbox name, but mailbox_list is the tool to discover valid names. Similarly, hook_delete requires a hook name that hook_list provides. No description hints guide the LLM to call dependent tools first.
Parameter constraints are described in prose but not enforced in schema. 'timeout_secs' [1, 600], 'hook name' regex [a-zA-Z0-9_][a-zA-Z0-9_.-]{0,127}, these should be in JSON Schema min/max and pattern fields, not just description strings. LLMs read schema constraints more reliably than descriptions.