Loading ToolBench
MCP server for Alpaca trading platform integration, providing tools and resources for account management, order placement, position tracking, market data, and asset information.
The Alpaca Trading MCP has 13 tools with significant quality gaps. While tool names follow verb_noun convention (get_*, place_*, list_*, close_*), descriptions are inconsistent, some lack detail on when/why to use them, parameter descriptions are sparse, and most critically, input/output schemas are underdocumented. Tools are registered as resources (via @mcp.resource decorator) rather than explicit tool definitions, which makes schema validation harder to assess. Reviewed source shows that tool definitions are inferred from resource handlers, not explicit schema registration. Parameter types are visible for some tools (e.g., place_market_order has symbol:string, quantity:number, side:string) but many tools lack any schema documentation. No output schemas are documented. Error handling is minimal, most tools return free-form strings without structured guidance for recovery. The server lacks security-critical features like credential injection patterns, permission gates, and audit logging for financial operations.
Cancel an existing order by its order ID.
Close a position by selling all shares of a given symbol.
Get current account information. Returns: Account summary with balance and status
Get all current open positions in the account.
Get detailed asset information by symbol.
Get historical price bars for a symbol with specified timeframe.
Input schemas missing for 5+ tools (get_account_info_tool, get_all_positions, list_tradable_assets, and resource-based tools lack explicit schema declarations)
No output schemas documented for any tool. Resource handlers return plain strings without structured field documentation, forcing LLMs to parse unstructured text and making tool chaining impossible.
Descriptions are too brief and lack context on WHEN/WHY to use each tool. E.g., 'Place a stop order to sell a stock if price drops below a specified level' (11 words) is ambiguous about order lifecycle, fills, and cancellation.
Tools are registered as resources (@mcp.resource) not explicit tool definitions. Schema validation is indirect and tool definitions may be inferred rather than explicit, capping assessability.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
Get current market quote (bid/ask) for a specific symbol.
Get details of a specific position by symbol.
List all tradable assets available on Alpaca.
Place a limit order to buy or sell a stock at a specified price.
Place a market order to buy or sell a stock. Args: symbol: Stock symbol (e.g., 'AAPL') quantity: Number of shares to buy or sell (can be fractional) side: Either 'buy' or 'sell' Returns: Order confirmation details
Place a stop-limit order combining stop and limit prices.
Place a stop order to sell a stock if price drops below a specified level.
Parameter 'side' in place_market_order, place_limit_order, etc. is described as 'Either buy or sell' but no enum constraint is declared. LLMs may pass invalid values like 'BUY' or 'long'.
No error handling guidance. Tool descriptions do not explain what happens if an order fails, a symbol is invalid, or the account is locked. Responses return raw API errors without recovery hints.
Financial operations (place_market_order, cancel_order, close_position) lack confirmation/dry-run support. An agent could accidentally execute trades without user approval.
No credential injection or secret management visible. Alpaca API key is passed to AlpacaClient() but no documentation on how it's injected securely; risk of leaking to logs or LLM context.
No permission gates or audit logging. Any agent with access can place unlimited orders. No traceability for compliance or incident response.
Parameter 'timeframe' in get_historical_bars accepts string but no enum. Accepted values (Min, Hour, Day, Week, Month) are mentioned in description but not formalized, inviting hallucinated values like 'minute' or '1H'.
Parameter 'limit' in get_recent_orders (resource) is manually validated in code (must be 1 - 100) but constraint is not declared in schema or parameter description for LLM visibility.