Loading ToolBench
MCP server for Qorami — check an email before an AI agent sends it (send / request_human_confirmation / do_not_send).
The Qorami MCP server has 3 tools with documented purposes and input schemas. Tool naming follows verb_noun conventions (qorami_health, verify_email, check_action_status), and descriptions are present but vary in quality. However, there are significant gaps in schema completeness, parameter documentation, and output structure definition. The tools are straightforward and well-intentioned for a guardrail use case, but they lack the rigor expected of production-grade agent tools. Descriptions are adequate but could be more LLM-optimized for clarity around when to use each tool and what to expect back.
After request_human_confirmation, poll this with the actionId until nextAction.type becomes "send" (approved) or "do_not_send" (blocked).
Check whether the Qorami MCP server is configured correctly. This never spends credits: it checks API reachability and, when a key is set, reads usage/balance.
Before sending ANY email, call this to get permission from Qorami. Returns a decision the agent MUST obey: send (allowed), request_human_confirmation (a human must approve first — then poll check_action_status), or do_not_send (blocked). When not allowed, "suggestions" says what to fix.
Output schemas are not documented. Tools describe responses in natural language ('Returns a decision') but provide no structured schema showing what fields the LLM will receive, field types, or whether responses are paginated. This forces LLMs to infer response structure, leading to parsing errors and missed chaining opportunities.
Parameter descriptions lack constraint details. The 'policyProfile' enum in verify_email lists allowed values, but descriptions for 'recipient', 'subject', and 'body' do not state format expectations, length limits, or encoding. The description for check_action_status parameter 'actionId' does not explain format or origin.
verify_email and check_action_status descriptions do not explain the state machine flow clearly enough for LLMs to understand when to use each tool or how they interact. The description mentions 'actionId returned by verify_email' but does not state that check_action_status is always paired with verify_email when human confirmation is needed.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
Error handling and recovery are not documented. The descriptions mention decisions ('send', 'request_human_confirmation', 'do_not_send') but do not guide what the LLM should do if the service is unreachable, if an actionId is invalid, or if polling times out. There is no recovery guidance (e.g., 'Retry after 5 seconds', 'Call qorami_health to diagnose', 'Escalate to human').
qorami_health input schema is empty ({}). While this is correct for a no-argument health check, the description does not clearly state what constitutes success, what the return structure is, or what the LLM should do if it fails.
Idempotency is not declared. The verify_email tool, when called multiple times with identical inputs, may return different decisions if the policy or human review state changes. The descriptions do not state whether it is safe to retry or whether retrying is idempotent, which is critical for agent error recovery.