Loading ToolBench
MCP server for ClawVault - AI agent payment security layer
ClawVault has 6 tools with clear verb-prefixed names and basic descriptions. However, several critical issues reduce overall quality: (1) Most parameter descriptions lack constraint information (format, ranges, enum values). (2) Output schemas are not formally documented in tool definitions, responses are free-text JSON stringified rather than structured. (3) Error handling is present but provides minimal recovery guidance. (4) No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels (WRITE, READ_ONLY, IRREVERSIBLE). (5) freeze_agent lacks any input schema or detailed description, scoring at 0. (6) request_payment's 'reason' parameter could be optional but its business purpose is unclear. Strengths: naming is consistent and action-verb-prefixed; descriptions exist and range 115 - 212 characters; parameters have basic type information (Zod strings/numbers). The server handles sensitive data (API_KEY) correctly via environment injection.
Check if a payment would be allowed under current security rules without actually creating it. Use this to preview whether a payment will be auto-approved, require manual approval, or be blocked.
Freeze an agent to immediately block all its payments
Get the current status of a payment by its ID. Use this to check if a pending payment has been approved, denied, or expired.
Get the current vault status including wallet address, balances, active rules, and usage statistics.
List recent payment transactions with pagination.
Request a payment through ClawVault. The payment will be evaluated against security rules and may be auto-approved, pending manual approval, or blocked.
freeze_agent tool has no input schema and no description in source code. Only stub declaration visible.
Parameter descriptions lack constraint information. 'amount' is described as 'Payment amount (e.g. '10.00')' but no mention of format (decimal places?), range (min/max?), or currency context. LLMs may pass invalid values like '10', '10.0000', or negative amounts.
Output schemas are not formally documented. Tools return responses as JSON-stringified text (type: 'text') rather than structured objects. This forces LLMs to parse unstructured text, wasting tokens and increasing hallucination risk. Expected pattern: return structured tool.Tool response with typed fields.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 40 | - | v1 |
No tool annotations despite clear risk levels declared. freeze_agent is IRREVERSIBLE but has no destructiveHint. request_payment is WRITE but has no annotation. get_vault is READ_ONLY but has no readOnlyHint. MCP spec supports tool annotations to guide agent behavior.
Error handling provides minimal recovery guidance. Example: 'Payment request failed: <message>' tells LLM nothing about whether to retry, call a different tool, or ask the user. Missing context like 'Try check_limits() first to validate the payment would succeed.'
Token symbol and chain parameters accept free-form strings with no validation. No enum of valid tokens (USDC, USDT, etc.) or chains (base, ethereum, etc.) despite examples in descriptions. LLMs may pass 'FAKE_COIN' or 'moonchain' and fail at API call time.
No idempotency guidance. request_payment does not document whether retrying with the same parameters produces duplicate payments or returns the existing payment. Critical for agents that retry on timeout.
list_transactions does not document maximum result limits or explain what happens if total > 1000. No guidance on whether to paginate or if results are capped server-side. Unbounded listings risk context window exhaustion.