Loading ToolBench
10 statistical analysis skills producing Word (.docx) reports. Each tool wraps one skill's end-to-end pipeline: fetch → R stats → Word rendering. Pre-flight guards return rc=1 with a clear message on bad input; R execution failures return rc=2.
Server exposes 10 statistical analysis tools with complete input schemas and descriptions. All tools follow verb_noun naming (run_*). Descriptions are detailed (100-300 chars) and explain what each tool does. However, parameter descriptions lack actionable constraints (enums, ranges, formats). Output schemas are not documented, LLMs cannot predict what fields to expect from r_output.json or the docx path. Error handling is basic: tools return status codes (success/pre_flight_error/r_error) but lack recovery guidance. No tool annotations (readOnlyHint/destructiveHint). All tools are WRITE operations but unmarked. Security: tools accept absolute file paths and shell out to orchestrator.py, path traversal and command injection risks are present but mitigated by subprocess isolation and no user-controlled command construction.
Demonstration tool for literature review and meta-analysis workflows. Produces a sample report with placeholder data.
Group comparison (t-test / ANOVA / Wilcoxon / Kruskal-Wallis), paired tests, chi-square, or correlation — with pre-checks (Shapiro / Levene / Grubbs), effect sizes, post-hoc tests, bootstrap CI, and post-hoc power. See skills/advanced-data-analytics/USAGE.md for the input schema per mode.
Data cleaning and preparation: missing value imputation, outlier handling, normalization, and feature engineering. Produces a cleaned dataset and a detailed report.
Exploratory Factor Analysis (EFA) with parallel analysis, scree plot, factor loadings, communalities, and Cronbach's α. Supports rotation (varimax, promax, none).
N-way ANOVA / ANCOVA with main effects, interactions, post-hoc pairwise comparisons, and simple-effects analysis when the interaction is significant. Auto-recommends Type III SS for unbalanced designs.
Output schemas not documented. Tools return dict with 'status', 'rc', 'stdout', 'stderr', 'docx', 'r_output' but LLMs cannot predict structure or extract statistical results. No guidance on what fields r_output.json contains.
Parameter constraints missing. 'analysis', 'posthoc', 'p_adjust', 'correlation_method', 'chi2_variant', 'rotation', 'method' accept enums but descriptions do not list valid values. LLMs must infer from context or guess.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 67 | 2025-06-18+ | v2 |
Bayesian inference and decision analysis: posterior distributions, credible intervals, Bayes factors, and decision trees. Supports prior specification and sensitivity analysis.
Multivariate analysis: PCA, MANOVA, discriminant analysis, or clustering (k-means / hierarchical). Includes biplots, scree plots, and group separation diagnostics.
Linear (OLS) or logistic regression with full diagnostics — coefficient CI, VIF for multicollinearity, residual / Q-Q plots, Cook's D. Logistic adds ROC/AUC + Hosmer-Lemeshow.
Kaplan-Meier survival curves, log-rank test, Cox proportional hazards regression, and cumulative hazard plots. Handles censoring and time-to-event data.
Time series analysis: ARIMA, exponential smoothing, seasonal decomposition, and forecasting. Includes ACF/PACF plots and stationarity tests.
No tool annotations. All 10 tools are destructive (WRITE, create files, modify state) but lack destructiveHint=true. LLMs cannot distinguish safe read-only tools from risky write operations.
Error handling lacks recovery guidance. Status codes (pre_flight_error, r_error) are returned but no actionable next steps. 'pre_flight_error' tells LLM nothing about what went wrong or how to fix it.
Path traversal risk. Tools accept 'file' and 'output_dir' as absolute paths with no validation. Malicious input like '../../../etc/passwd' could be passed to subprocess. Mitigated by subprocess isolation but not hardened.