Loading ToolBench
Agentic time series analysis, forecasting, backtesting, deployment, and monitoring tools, exposed as FastMCP servers
Strong foundation with 14 well-scoped tools, each doing one thing. All tools have descriptions (avg 180 chars, within 10-1024 baseline). Input schemas present with types and descriptions for all parameters. However, output schemas are not explicitly documented in the visible code, only inferred from docstrings. Tool names follow verb_noun pattern (plot_*, forecast_*, check_*, basic_*). No security issues detected (no secrets in params). Error handling is implicit (relies on exceptions) rather than explicit recovery guidance. Missing tool annotations (readOnlyHint, destructiveHint) despite all tools being READ_ONLY.
Summary stats: length, date range, missing values, basic distribution. Also reports a confidence interval for the mean (Student's t, since sample sizes here can be small enough that the normal approximation isn't a safe substitute) -- a bare mean with no uncertainty band invites reading small differences (across series, or across cuts of the same series) as more meaningful than the sample size supports.
Augmented Dickey-Fuller AND KPSS tests for stationarity, each with an effect size, combined into one joint verdict. ADF's null hypothesis is that the series has a unit root (non-stationary); a small p-value (< 0.05) rejects that, suggesting stationarity. KPSS's null hypothesis is the OPPOSITE -- that the series IS stationary; a small p-value there rejects stationarity. Running both and reading them together is standard practice, because each test's blind spot is roughly the other's strength.
Fit two trivial baseline models (flat-naive and seasonal-naive) on a train split, backtest against a held-out window, and return error metrics plus residual diagnostics.
Extend the series with a trivial baseline forecast: either flat (repeat the last value) or seasonal (repeat the last full seasonal cycle). Useful as a sanity-check floor alongside a real model's forecast, same role naive baselines play in forecasting best practice.
Output schemas not explicitly documented. Tool descriptions mention return structure (e.g., 'returns error metrics plus residual diagnostics') but no formal JSON Schema for responses. LLMs cannot reliably parse or chain results without documented output types.
Tool annotations missing. All 14 tools are READ_ONLY (no state modification), but readOnlyHint is not declared in tool definitions. This forces LLMs to infer safety from descriptions rather than machine-readable metadata.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
Plot ACF and PACF as two stem plots side by side, with the SAME per-lag Bartlett significance bands acf_pacf_summary reports (shared computation via _acf_pacf_raw, not reimplemented) -- lags whose bar crosses outside the shaded band are the ones acf_pacf_summary would flag as significant.
Plot the series with points flagged by the ROBUST (median+MAD) anomaly detector marked -- reuses detect_anomalies_robust_zscore's own output (this project's recommended default over the plain z-score version, since it doesn't self-dilute on the very anomaly it's trying to flag) rather than recomputing flags independently.
Plot actual vs. predicted values over a backtest holdout, with a shaded prediction-interval band if lower/upper bounds are supplied.
Plot the series with detected changepoints as vertical lines, segments shaded alternately -- reuses detect_changepoints' own output (the same CUSUM/permutation-test results) rather than recomputing changepoints independently.
Plot a series' history plus a deployed forecast_* result's own forecast trajectory, with a shaded interval band wherever the forecast points carry lower/upper bounds.
Plot periodogram power vs. period, marking both the single globally strongest frequency (which can be a trend/edge artifact -- the finding detect_seasonality_period's dominant_period_in_reported_range flag exists to catch) and the top in-range candidate distinctly, so the two are never confused visually.
Plot per-origin MAPE across a rolling_origin_backtest's walk-forward origins, with a shaded band showing the mean +/- one std across origins -- makes cross-origin instability (a model whose accuracy swings a lot depending on which stretch of the series it's tested against) visually obvious rather than requiring a reader to compare a table of numbers.
Plot AICc by candidate SARIMA order as a bar chart, so a razor-thin margin between the top candidates (which a table of decimals can hide) is visually obvious.
Plot the additive trend/seasonal/residual decomposition as a vertically stacked subplot -- the same seasonal_decompose call seasonal_decomposition_summary's strength scores are computed from.
Plot the raw value-vs-date series. Missing values show as visible GAPS in the line (not interpolated over) -- basic_stats already treats missing data as a first-class finding in this project, so the plot shouldn't paper over it either.
Error handling is implicit (exceptions only). No explicit error recovery guidance. E.g., if a DataFrame lacks 'date' or 'value' columns, the tool will raise KeyError, but the description does not explain what to do or how to fix it. Agents cannot self-correct.
Parameter 'df' type is 'object' (generic). Should be more specific (e.g., 'DataFrame' or a structured schema describing required columns 'date' and 'value'). LLMs cannot validate input shape before calling.
plot_* tools accept optional 'out_path' parameter but do not document what happens if the path is invalid, unwritable, or the directory does not exist. No guidance on fallback behavior (e.g., return plot data in-memory vs. fail).