Loading ToolBench
MCP server for Privy.io API - user statistics, wallet management, and authentication
Server has 12 tools with complete naming and descriptions. Tool names follow verb_noun conventions (get_, list_, create_, delete_, search_, update_). Descriptions are present for all tools (range 94-226 chars, median ~160 chars). Input schemas are explicitly defined with parameter types. However, output schemas are NOT documented, the code returns dict[str, Any] with no structured response documentation. Error handling is absent: no recovery guidance, no input validation, no categorization of errors as retryable/user-fixable/fatal. Parameters lack constraints (enums, ranges, patterns). Critical parameters like 'linked_accounts' accept raw dicts with no format validation. The search_users tool documents mutual exclusivity in descriptions but relies on client-side validation. Missing baseline outputs for downstream tool chaining (e.g., create_user should return user_id for chaining to create_wallet).
Create a new user with linked accounts. Supports email, phone, wallet, and 15+ OAuth providers (Apple, Discord, GitHub, Google, Farcaster, Telegram, Twitter, etc.). Optionally creates an embedded wallet.
Create or pregenerate a wallet for a user. Supports Ethereum, Solana, Bitcoin, and all EVM-compatible chains.
Delete a user by Privy DID. This action is permanent and removes all associated data.
Get user details by Privy DID (decentralized identifier). Returns user data including linked accounts (email, phone, wallet, OAuth providers), MFA methods, creation timestamps, terms acceptance, and custom metadata.
Query and retrieve user information by blockchain wallet address. Supports Ethereum, Solana, Bitcoin, and all EVM-compatible chains.
Output schemas NOT documented. All tools return dict[str, Any] with no field documentation. Agents cannot plan downstream calls or extract required IDs. Violates pattern:tool and pattern:response-shaper. Example: create_user should document it returns {user_id, linked_accounts, wallets, created_at, ...}, not opaque dict.
No error handling or recovery guidance. Code does not validate inputs, catch exceptions, or provide actionable error messages. Example: create_user accepts arbitrary list[dict] for linked_accounts with no schema or validation. If invalid, users get raw HTTP error with no guidance on what went wrong or how to fix it. Violates pattern:recovery-guide and pattern:error-classification.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
Get wallet details by wallet ID. Returns wallet address, chain type, creation/export/import timestamps, policy IDs, and owner information.
Get the current balance of a wallet.
Get transaction history for a wallet with pagination support.
List all users with pagination support. Returns paginated list of users with their linked accounts, wallets, and metadata.
List all wallets with pagination. Returns wallet addresses, chain types, and associated metadata.
Search for users by search term OR specific identifiers (emails, phone numbers, wallet addresses). Note: Use either search_term OR specific identifiers, not both. Returns user data matching the search criteria including linked accounts, MFA methods, and metadata.
Update wallet metadata and configuration.
Parameters lack constraints (enums, ranges, formats). Example: chain_type in create_wallet is a bare string with no enum of valid values (ethereum, solana, bitcoin, etc.). LLMs will hallucinate invalid values. Similarly, limit parameters in list_users and list_wallets have no min/max bounds, agents could pass limit=999999. Violates pattern:constrained-input.
Destructive operations (delete_user) lack confirmation/dry-run pattern. delete_user accepts a user_id and immediately deletes all associated data with no confirmation step, no dry-run, and no recovery tool offered. Agents make mistakes. Violates pattern:confirmation-request.
Tool chaining references missing. Example: create_user returns dict but does not document that it returns user_id; downstream create_wallet expects user_id but agents have no way to map the output of create_user to the input of create_wallet without seeing unstructured dict. Violates pattern:tool-chain and mxe:include-chaining-ids.
Pagination return structure undefined. list_users and list_wallets accept cursor parameter but code does not document what the return includes (next_cursor, total_count, has_more, etc.). Agents cannot iterate correctly without this info. Violates pattern:paginated-result.
Parameter description generic for linked_accounts in create_user. Description says 'List of account objects to link (e.g., [{"type": "email", "address": "[email protected]"}])' but includes an example value in the description. Should be: 'Array of account objects, each with required fields: type (one of: email, phone, wallet, apple, discord, github, google, farcaster, telegram, twitter, ...) and address or value.' Violates pattern:tool-description.
Mutual exclusivity in search_users documented in description but no validation in schema. Description states 'Use either search_term OR specific identifiers, not both' but the function signature accepts all params as optional. LLM could pass both search_term and emails, causing ambiguous or failing API calls. Should use mutually exclusive groups in OpenAPI/JSON Schema or validate and return clear error. Violates pattern:constrained-input and review:param-relationships.
Missing documentation of sensitive data handling. Tools like get_user and search_users return user linked accounts (email, phone, wallet addresses). Code does not document what sensitive fields are included in responses or whether they should be filtered. Violates pattern:secret-injection (response must not include secrets).
No audit logging or permission checks. Tools like delete_user and update_wallet modify state but code does not log who called what, when, or with what parameters. No permission verification before executing. Violates pattern:audit-trail and pattern:permission-gate.