Loading ToolBench
Open-source AI Shopping Agent — AI buys products online using Lithic virtual cards via MCP. Makes real payments via Stripe and PayPal.
ClawPay exhibits significant structural and naming issues across its 9 tools. While input schemas are visible and well-formed, descriptions lack specificity about irreversible consequences, parameter descriptions are sparse or missing, and tool names violate single-responsibility principles. The server handles payment and shopping operations, high-risk domains, but does not clearly guide LLMs on error recovery, idempotency, or the destructive nature of operations. 'browse_and_buy' combines multiple concerns (browsing, adding to cart, purchasing). Most parameter descriptions are minimal (6-20 chars) vs. the baseline of 72 chars. Output schemas are not documented. No security hints or audit trails are visible.
Browse an online store, add items to cart, and complete purchase using a Lithic virtual card. Requires Playwright installed.
Get Stripe account balance.
List recent payment transactions.
Create and confirm a payment in cents.
Refund a payment intent.
Send money via PayPal Payouts to an email address or phone number.
Tool 'browse_and_buy' violates single-responsibility principle, combines browsing, cart management, and purchase completion in one tool. Should split into separate tools (search_products, add_to_cart, checkout) so agent can compose them.
Irreversible operations (pay, send_paypal, browse_and_buy, refund) lack explicit 'IRREVERSIBLE' or 'WRITE' warnings in descriptions. LLMs need to know which operations cannot be undone. Descriptions should state: 'This operation is irreversible, no undo possible.'
Parameter descriptions are sparse or missing. Example: 'send_paypal' has 'recipientEmail' (no description visible), 'recipientPhone' (no description), 'amount' (terse). Baseline is 72 chars per parameter; most here are 0-20 chars. This prevents LLMs from understanding constraints and valid formats.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 50 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 37 | - | v1 |
Set up Lithic virtual card API for AI shopping. Reads LITHIC_API_KEY from environment.
Set up Stripe payment method for ClawPay.
Link PayPal account using Client ID and Client Secret. Reads credentials from PAYPAL_CLIENT_ID and PAYPAL_CLIENT_SECRET environment variables or config file.
No output schemas documented. Tools return unstructured JSON (e.g., 'Get Stripe account balance', what fields?). LLMs cannot plan downstream calls or extract data without knowing result structure. Document every tool's return type.
API keys and secrets exposed in environment variable names within tool descriptions (LITHIC_API_KEY, STRIPE_SECRET_KEY, PAYPAL_CLIENT_ID/SECRET). While server-side injection is used, descriptions should NOT mention credential names, this signals where secrets are stored.
No error recovery guidance. Descriptions do not explain what to do on failure. E.g., 'pay' lacks guidance: 'If payment fails due to insufficient balance, try…'. LLMs need actionable error handling.
'list_transactions' has optional 'limit' parameter with no description, range, or default documented. What is the max? Does negative mean 'all'? Baseline requires constraints on numeric params (1-100, 1-365).
'send_paypal' requires either 'recipientEmail' or 'recipientPhone' but schema only marks 'amount' as required. Mutually-exclusive-or-both dependency is undocumented and will cause silent failures when LLM passes both or neither.
No idempotency guarantees or dry-run support for destructive operations. LLMs retry on ambiguous failures, 'pay' called twice could charge twice. No mention of idempotent keys, confirmation steps, or estimates.