Loading ToolBench
ART — A Real-Time Time-Series Analysis toolkit (Box-Jenkins-Treadway methodology) + MCP server for ARIMA model building, interventions, and forecasting
14 tools with generally clear names and reasonable descriptions (avg 120 chars), but significant gaps in schema completeness and parameter documentation. Most tools lack explicit input validation constraints, enum declarations, and output schema documentation. Error handling is minimal, no recovery guidance or actionable error messages visible. Parameter descriptions exist but are often terse (avg 45 chars vs. baseline 72 chars). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk profiles (WRITE vs. READ_ONLY). Composition is sound, tools chain logically (e.g., create_inp → guided_identification → confirm_and_estimate → formal_tests), but output schemas are not documented, forcing LLMs to guess what fields to extract for downstream calls.
Build ARIMA models for multiple series in sequence (not in parallel)
Automatic ARIMA model building using heuristic rules (auto-ARIMA without full guided analysis)
Estimate ARIMA model with specified parameters and return diagnostics
Create a .inp file from user-provided data (Excel, CSV, list of numbers)
Analyze Box-Cox transformation suitability (lambda selection diagnostics)
Summarize all formal test results (unit root, seasonality, residual diagnostics) in narrative form
Run formal statistical tests on estimated model (unit root, overdifferencing, seasonality, residual diagnostics)
Output schemas not documented. Tools return results (diagnostics, reports, forecasts) but LLMs cannot see what fields to extract. Forces guessing and mid-chain failures.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Risk profile is clear from metadata (WRITE vs READ_ONLY) but not exposed to LLM. Agents cannot reason about retry safety or side effects.
Parameter descriptions are terse (avg 45 chars vs. baseline 72). Missing format constraints, ranges, and dependency hints. E.g., 'frequency' lacks valid values (12=monthly, 4=quarterly); 'start_date' format is stated but not validated.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Generate comprehensive HTML report of model identification, estimation, and diagnostics
Generate point and interval forecasts from estimated ARIMA model
Extract and format estimation results from .out file (parameters, standard errors, diagnostics)
Guided identification of ARIMA model components (lambda, differencing, seasonality, AR/MA orders) with analyst decision at each node
Reformulate deterministic seasonality to stochastic at specified frequencies using MEG test
Detect and suggest intervention (outlier, level shift, temporary change) locations and forms
Update forecast with new observations and re-forecast
No error handling guidance. Tools fail silently or return raw errors. No recovery hints (e.g., 'If .inp file not found, call create_inp first'). Agents cannot self-correct.
Enum constraints missing. 'objetivo' parameter in guided_identification and build_model declares enum values, but other parameters (e.g., 'frequency' in create_inp) accept free-form strings, inviting hallucinated values.