Loading ToolBench
Stock trading skill with risk management, order placement, and outcome tracking for autonomous trading agents
This server has 8 tools with explicit schemas and descriptions, but suffers from critical naming and composition issues. Tool names lack action verbs (logger, outcomes_reducer_current_state, risk_snapshot, run_summary) making intent unclear. Many descriptions are verbose and lack WHEN/WHY guidance. Parameters are typed but descriptions are sparse. No error handling guidance, no output schema documentation, and no idempotency markers despite stateful file operations. The risk_* tools form a tightly coupled state machine (validate→commit→release) that should be better documented. Overall: definitions exist but fall short of production quality.
Append-only JSONL run log writer for the stock-trading skill. Reads one run object as JSON on stdin, appends it as a single line to logs/trading-log.jsonl, and prints {"ok": true, "run_id": ..., "path": ...} to stdout. Uses fcntl.flock for crash safety so concurrent writers cannot interleave partial lines.
Appends each line to logs/outcomes.jsonl atomically (fcntl.flock). Each entry must have run_id, ticker, outcome_state. Returns ok, appended count, and path.
Walks logs/trading-log.jsonl to find every placed order, walks logs/outcomes.jsonl to get the latest state per (run_id, ticker), and emits the work queue of decisions whose state can still advance. Terminal states (t20, unfilled_cancelled) are omitted. Returns decisions array with run_id, ticker, order_id, experiment_id, confidence, min_trade_override, limit_price, current_state, next_state, fill_price, filled_at.
Promotes a pending reservation to submitted (permanent idempotency marker). For BUYs, also advances session_deployed_confirmed. Call this AFTER the order placement succeeds on the broker side.
Clears a pending reservation and, for BUYs, refunds the reserved notional back out of session_deployed. Call this if order placement fails, or at the end of a dry run to unwind the reservation.
Tool names lack action verbs. 'logger', 'outcomes_reducer_current_state', 'risk_snapshot', 'run_summary' do not start with verbs (append_, get_, list_, etc.). LLMs cannot infer intent from these names alone.
No output schema documentation. Tools return JSON but the response structure is not formally documented. LLMs cannot plan downstream calls or extract fields reliably.
No error handling guidance. Tools do not document what errors can occur, whether they are retryable, or what the LLM should do next. E.g., risk_validate may reject an order but does not explain how to adjust parameters.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
Reads account JSON and returns risk context including pdt_headroom, max_session_allocation, max_per_position, session_deployed_so_far, blocked_tickers, and blocked_reasons. Checks for trading_blocked and account_blocked flags from Alpaca.
Validates an order against risk rules. If approved, writes a pending reservation for (date, ticker, side) to logs/state.json and (for BUYs) increments session_deployed by the reserved notional. Checks qty, limit_price, bid/ask spread, notional limits, PDT rules, and session allocation caps.
Human-readable summary of stock-trading runs. Reads logs/trading-log.jsonl (and logs/outcomes.jsonl if present) and renders a plain-text summary. Filters by date range or experiment id. Can show most recent run, aggregate runs since a date, or aggregate runs by experiment_id.
Tight coupling of risk_validate → risk_commit → risk_release. These three tools form a state machine but lack explicit documentation of the required call sequence, idempotency guarantees, or what happens if steps are skipped or retried.
Parameter descriptions are sparse. E.g., 'date' in risk_snapshot is described as 'Trading date (YYYY-MM-DD)' but does not explain what happens if the date is in the future, past, or a non-trading day. 'account' parameter is a bare object with no field-level guidance.