Transaction-cost sensitivity analysis and look-ahead-bias auditing for backtested trading strategies
Three well-defined tools with strong domain-specific descriptions (194 - 280 chars, above baseline 194 avg). All tools have explicit Zod schemas with type constraints and parameter descriptions. Tool names are action-oriented (cost_sensitivity_sweep, breakeven_cost_finder, lookahead_bias_checklist). Schemas are complete with minItems/min constraints. However, output schemas are not formally documented, responses are JSON strings without declared field types. Error handling is minimal (no recovery guidance, no categorization). No tool annotations (readOnlyHint, idempotentHint) despite all being read-only operations. Parameter descriptions are strong but could include format/range hints (e.g., 'Dollar amounts, non-negative').
Compute the exact per-trade cost (commission + slippage combined, in dollars) at which this strategy's net P&L hits exactly zero. A single concrete number: "this edge survives up to $X of combined friction per trade" — compare it directly against real expected costs for the instrument/broker rather than eyeballing a sweep table.
Recompute net P&L, win rate, and profit factor across a range of assumed per-trade transaction costs (commission + slippage combined, in dollars per round-turn trade). Backtests commonly use unrealistically cheap or zero-cost fills — this shows exactly how fast the reported edge degrades as realistic friction is added, and at what cost level it flips negative.
Structured look-ahead-bias audit for a backtest — answer what your actual backtest does for each question, get back which specific, well-known leakage patterns you're exposed to. Covers the sources that pass a compile/run cleanly and still quietly inflate results: same-bar high/low entry logic, indicators computed from an unclosed bar, position sizing off future volatility, survivorship-filtered universes, and un-timed price adjustments.
Output schemas not formally documented. Tools return JSON strings without declared field types, making it impossible for LLMs to plan downstream operations or extract specific fields reliably.
No tool annotations despite all three tools being read-only, stateless operations. Missing readOnlyHint and idempotentHint annotations prevent LLMs from optimizing retry logic and caching.
Error handling lacks recovery guidance. No actionable error messages, no categorization of retryable vs. fatal errors, no suggestions for user-fixable issues (e.g., invalid input ranges).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | 2026-07-28+ | v2 |
Parameter descriptions lack format/range hints. 'gross_trade_pnl' accepts any number array but should clarify units (dollars), sign convention (positive/negative), and realistic ranges to guide LLM input validation.
lookahead_bias_checklist parameter descriptions are complete but very long (200+ chars each). Could be condensed to 50 - 100 chars with format constraints moved to enum/pattern declarations.