Loading ToolBench
Claude Backtest Framework - A structured backtesting workflow for trading strategies with Claude Code
CBT Framework exposes 23 tools with minimal MCP structure. Most tools lack proper input schemas, parameters are described in docstrings but not registered as JSON Schema. Descriptions exist but are terse (avg ~40 chars, well below the 194-char baseline). No output schemas documented. Tools accept complex types (pd.Series, np.ndarray, List[Trade]) that cannot be serialized over MCP without custom marshalling. No error handling guidance, no parameter validation rules, no enum constraints. The server is a Python library wrapped minimally for MCP exposure, not a production-grade agent tool.
Calculate Calmar ratio (return / max drawdown)
Calculate max consecutive wins and losses
Calculate expected value per trade: (win_rate * avg_win) - (loss_rate * avg_loss)
Calculate maximum drawdown from equity curve
Calculate profit factor (gross profit / gross loss)
Calculate prop firm challenge compliance metrics including max drawdown from initial capital, daily loss tracking, phase target reached, and breach detection
Calculate period returns from equity curve
No input schemas visible. All 23 tools lack JSON Schema definitions. Parameters are documented in docstrings but not registered as structured schemas. MCP clients cannot validate inputs or guide LLM parameter selection.
Descriptions are terse (avg ~40 chars, baseline 194 chars). Most lack context for when/why to use the tool. E.g., 'Calculate Sharpe ratio from period returns' does not explain when to call it vs sortino_ratio, or what the output means for strategy evaluation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 37 | 2026-07-28+ | v2 |
Calculate Sharpe ratio from period returns
Calculate Sortino ratio (downside risk-adjusted return)
Calculate comprehensive trade statistics including total trades, win rate, profit factor, average trade metrics, and expectancy
Calculate win rate percentage from trades
Compute exponential moving average
Generate all features from NumPy arrays with strict lookahead prevention. Every feature is shifted by 1 to prevent lookahead bias. At bar t, only data from bars 0..t-1 is used.
Collect and merge all lazy frames into a single aligned DataFrame. Applies date filters before collection for memory efficiency.
Load all required datasets as lazy frames from data directory. Supports parquet and CSV files.
Compute rolling mean using cumsum trick with O(n) complexity
Compute rolling standard deviation
Numba-compiled backtest loop with prop firm challenge enforcement. Returns equity curve, trade PnLs, entry/exit bars, directions, prices, and prop firm breach information.
Shift array by N periods (positive = shift right/lag)
Print data summary including shape, columns, memory usage, and data types
Convert collected DataFrame to NumPy arrays for Numba processing. Returns dict with column names as keys and contiguous float64 arrays as values.
Validate data integrity on a sample. Checks for required OHLCV columns, null values, and duplicate timestamps.
Verify no future data leakage by checking first values are NaN
Complex parameter types (pd.Series, np.ndarray, List[Trade]) cannot be serialized over MCP without custom marshalling. LLMs cannot construct these types from chat context. Tools are designed for Python-to-Python calls, not agent invocation.
No output schemas documented. Tools return complex objects (equity curves, trade lists, metric dicts) but LLMs have no schema to understand structure. Downstream tool chaining is impossible.
No error handling guidance. Tools will fail silently or with stack traces when given invalid inputs (e.g., empty trade list, mismatched array lengths). No recovery hints for LLMs.