Loading ToolBench
MCP server for interacting with the Solana blockchain, providing tools to check account balances, send SOL tokens, and request airdrops on the Solana devnet
SolanaBot demonstrates basic MCP registration with 3 tools, but has significant quality gaps. All tools are explicitly registered in index.js with Zod schemas and descriptions. However, descriptions are extremely brief (17-65 chars, well below the 50-200 baseline for LLM-optimized tooling), parameters lack comprehensive documentation, and error handling relies on generic string responses without recovery guidance. No output schemas are documented. The tool interface uses Solana-specific domain terms (publicKey, LAMPORTS_PER_SOL conversion) without explaining the chat-data-model mismatch, users think 'send 1 SOL', but tools expose raw lamports internally. Risk classification is present (READ_ONLY, WRITE) but not wired into responses or confirmations.
Request an airdrop of SOL tokens to a public key on devnet
Get SOL balance for a public key
Send SOL tokens from the configured sender account to a recipient
Tool descriptions are critically underdeveloped (17 - 65 characters, 3 - 30× below 50 - 200 baseline). 'Get SOL balance for a public key' lacks WHEN to use this vs alternatives, prerequisites, or return value structure. LLMs cannot reliably distinguish between tools on this basis.
No parameter descriptions beyond a single phrase. 'amount' in sendSol says 'Amount of SOL to send (must be positive)' but does not explain units (is this SOL or lamports? what precision? what happens if fractional?), minimum/maximum bounds, or failure modes if amount exceeds sender balance.
No output schema documentation. Error responses are plain text strings in a 'content' array; success responses vary (airdropSOL returns tx signature, getBalance returns formatted SOL amount). LLMs cannot parse structured output or chain tools because response schema is opaque.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 33 | - | v1 |
Error handling is generic and non-actionable. 'Error: Invalid public key format' does not guide recovery. No categorization of errors as retryable (network) vs user-fixable (bad input) vs fatal (unauthorized). Missing recovery suggestions like 'Use search_accounts to find the public key' or 'Check that amount exceeds transaction fee.'
Destructive operations (sendSol, airdropSOL) lack confirmation or dry-run capability. An LLM error could send real funds on devnet or mainnet. No idempotency guarantees documented (can retrying a sendSol with the same params cause double spend?). No destructive/idempotent hints in tool annotations.
Tool composition is incomplete. getBalance returns formatted text ('💰 Balance of 0xABC is 5.5 SOL'), not a structured object {publicKey, balanceLamports, balanceSol}. This forces downstream tools to parse text, breaking chaining. If an agent needs to route funds based on balance, it cannot extract the value programmatically.
Credentials (PRIVATE_KEY) are passed via environment variable and loaded in solana.js, which is correct. However, no audit trail, rate limiting, or permission scoping is visible. The server does not document required permissions (e.g. 'only accounts with 'send-sol' permission can invoke sendSol'). If this MCP server is compromised, an agent can drain the sender account.
Solana domain logic bleeds into tool interface. Parameter 'publicKey' expects a base58-encoded Solana public key string, but does not accept human-readable identifiers (usernames, email, wallet labels). Users think 'send to Alice', not 'send to 5E5mG7nQZXN8xZ...'. Tool must resolve natural identifiers internally.
No input validation logic visible in tool implementations. e.g., sendSol constructs a Transaction and calls connection.sendTransaction() but does not validate that 'to' is a valid base58 public key until calling 'new PublicKey(to)' inside the try block. Invalid input like 'to: 'not-a-key' is caught as a generic error string instead of a clear, early 'Invalid public key format: not-a-key. Expected base58 string (40-50 chars).'
airdropSOL uses a default parameter 'amount: 1' but does not explain this default in the parameter description. Users invoking airdropSOL with only a publicKey may be surprised to receive exactly 1 SOL instead of requesting a custom amount first. Default should be documented and justified (e.g., 'default: 1 SOL, useful for basic account activation').