Loading ToolBench
Local Mangrove-powered AI trading agent — FastAPI + MCP + APScheduler
The server has 6 tools with reasonable naming (verb_resource pattern) and descriptions present for all. However, there are significant gaps in schema documentation, parameter constraints, and error handling guidance. Descriptions are adequate but lack the actionable structure expected for A-grade tools (WHEN to use, WHAT happens, WHAT to return). Output schemas are not documented. The tools are well-intentioned but fall short of production-grade quality, typical of a C-grade (60-69) implementation with noticeable gaps.
Create + encrypt a wallet. Response carries only vault_token + reveal_cmd — plaintext never enters the Claude Code transcript. EVM only in v1.
Get token balances for an address across EVM chains (Base, Ethereum, Arbitrum, Optimism, Polygon). Input is a wallet address + chain_id; output is a dict of token symbol -> balance.
Import an existing wallet from a stashed vault_token. The user must obtain the id by running scripts/stash-secret.sh in a terminal FIRST — this tool refuses raw keys by design.
List all registered MCP tools with their access tier, parameters, and pricing. Free, no auth.
List stored wallets (secrets never returned).
Return agent status: version, wallets count, strategies by status, active cron jobs, db path, uptime. Free, no auth required.
Output schemas not documented. No description of what fields each tool returns, structure of response objects, or pagination metadata. LLMs cannot plan downstream tool chains without knowing what data to extract.
Parameter constraints missing or vague. 'chain' accepts 'evm' (default) or 'xrpl stubbed 501', but no enum definition in schema. 'network' lacks enumeration for 'mainnet|testnet'. 'chain_id' is integer with no minimum/maximum bounds documented. Free-form strings invite hallucinated values.
Error handling guidance insufficient. Tools return AgentError JSON with code/message/suggestion, but no indication of retryability, user-fixable vs fatal classification, or recovery next steps. Example: 'API key required or invalid' could guide the user to provide/check the key, but 'Invalid chain_id' lacks actionable suggestion.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 67 | 2026-07-28+ | v2 |
Descriptions lack 'WHEN to use' context. 'create_wallet' says WHAT (Create + encrypt) but not WHEN (only call if no wallet exists?) or expected response structure. 'list_tools' returns 'parameters and pricing' but no description of output format or field meanings.
list_wallets and list_tools lack pagination parameters (limit, offset, page_size) and response metadata (total_count, next_cursor). Unbounded list results risk context window exhaustion.
API key as parameter creates security friction. While api_key parameter includes guidance to omit when supplied via X-API-Key header, the parameter's presence in schemas makes it visible in traces. Documented recommendation is unclear (is header auth optional or required?).
create_wallet and import_wallet return vault_token + reveal_cmd, but response schema not documented. Does it return JSON with {vault_token, reveal_cmd}? Are there additional fields (wallet_id, address, chain, network)? LLMs need this to chain downstream calls.
list_wallets description says 'secrets never returned' but does not specify what IS returned. Are wallet addresses returned? Labels? Chain/network info? Unclear, forcing LLM to guess or attempt trial-and-error invocations.