Loading ToolBench
Mixed quality across 11 tools. Naming follows verb_noun convention well (get_ticker, open_trade, close_trade). All tools have basic descriptions (10-100 chars), but most are minimal and lack context about when to use them or what they return. Input schemas are present for all tools with proper type definitions and enums for constrained parameters (e.g., side=[buy|sell], type=[market|limit]). However, descriptions lack depth, most do not explain consequences of actions (especially critical for financial/irreversible operations like withdraw_lightning), do not mention prerequisites, and do not guide LLM decision-making. Error handling is not evident in tool definitions or handler code. No output schemas are documented. Parameter descriptions are present but sparse (averaging 30-50 chars, below the 72-char baseline). The codebase shows proper schema registration via tools array with inputSchema fields, but lacks documentation of return types or error cases.
Add margin to a running position
Cancel a pending order
Close a running trade
Create a Lightning invoice to deposit funds
Get account info including balance
Get current BTC ticker with bid, ask, index price and funding rate
List futures trades by status
No output schemas documented. Tools return API responses directly without documentation of returned fields, types, or structure. LLM cannot reason about what data is available after calling these tools.
Descriptions are minimal and do not explain consequences. Financial/irreversible operations (open_trade, withdraw_lightning, close_trade) lack warnings about state changes. Descriptions average 40-50 chars; baseline is 194 chars. No guidance on when to use vs. similar tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Open a new futures position
Update stop loss for a trade
Update take profit for a trade
Withdraw via Lightning by paying an invoice
No error handling guidance in code. Handler catches generic API errors and throws raw messages ('API error (status): response'). No recovery guidance, categorization (retryable vs. fatal), or actionable next steps for LLM.
Irreversible operation (withdraw_lightning) lacks confirmation/dry-run pattern. Direct withdrawal via paid invoice has financial consequences but no confirmation step to prevent agent mistakes.
Parameter descriptions inconsistent and sparse. 'Trade ID' is vague (what format? what is it?). 'amount' in add_margin lacks range/validation hints. Parameter annotations average 30-50 chars vs. baseline 72 chars. Some parameters have no guidance on constraints.
No validation or constraint documentation for numeric fields. leverage (1-100) has constraint in description but others do not. quantity, amount, price, stoploss, takeprofit lack min/max guidance. LLM can pass invalid values (negative amounts, leverage > 100) with no validation feedback.
Credentials passed via environment variables (correct), but no documentation of required env vars in tool descriptions. Users cannot self-serve configuration; missing credentials cause cryptic 'Missing credentials' error without guidance on how to set them.