Loading ToolBench
A complete algorithmic trading system that uses a Large Language Model to make stock trading decisions
Server has 8 tools with basic naming and descriptions, but significant gaps in schema completeness, parameter documentation, and error handling. Tool names follow verb_noun convention (get_*, add_*, submit_*), which is good. However, most tools lack detailed parameter descriptions, output schemas are not documented, and error recovery guidance is absent. The submit_order tool is particularly concerning, it's irreversible but has no confirmation pattern or dry-run capability. Parameter types are present but descriptions are minimal (1-2 sentences). No evidence of input validation, rate limiting, or permission gates. This is a typical community-grade server with functional tools but production-readiness gaps.
Store a new situation-reflection pair.
Fetch MACD (Moving Average Convergence Divergence) for a ticker, which can be useful for decision-making. MACD > Signal means bullish crossover, MACD < Signal means bearish crossover.
Retrieve the most similar past trades and reflections done.
Fetch financial news for the given tickers on a specific date or past historical date. Call this to generate new news data.
Fetch RSI (Relative Strength Index) for a ticker, which can be useful for decision-making. RSI > 70 means overbought (sell signal), RSI < 30 means oversold (buy signal).
Get the current trading state such as cash, equity, buying power, etc.
submit_order lacks confirmation/dry-run pattern for irreversible operation. No guidance on what happens on retry, no dry-run capability, no permission checks documented.
Output schemas not documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., what does get_stock_histories return? field names? data types?).
Parameter descriptions are minimal (1-2 words). 'limit' described as 'max articles per API batch' lacks range constraints (min/max). 'period' accepts '5d, 10d, 15d' but no enum constraint or validation guidance.
No error handling or recovery guidance. Tools return raw exceptions (e.g., 'Error: Tool X error: {e}') with no actionable next steps for the LLM.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Get OHLCV price history for multiple stocks over a period of time until today's date.
Submit a dictionary of trade orders to the Alpaca API to make trades.
get_state has empty input schema ({}). No documentation of what fields are returned (cash, equity, buying_power, positions). LLM cannot reason about portfolio state without knowing the response structure.