Loading ToolBench
An agentic AI data-science assistant that exposes the AutoDS pipeline as MCP tools. Point these tools at a tabular dataset (CSV/Excel path) to profile it, run EDA, recommend cleaning and models, train, evaluate, and get insights.
AutoDS Copilot exposes 9 tools via STDIO MCP with basic schemas and descriptions. Most tools have descriptions (8/9), but they are generic and lack actionable guidance for LLM selection. Parameter descriptions are minimal or absent. No output schemas are documented. Tools like 'full_run' and 'train_and_evaluate' combine multiple responsibilities. Error handling is absent, no recovery guidance, no categorization of retryable vs fatal errors. Security: file paths are user-supplied with minimal validation (only _resolve() path traversal check). No pagination, no rate limiting, no audit logging. Naming is mostly clear (verb_noun pattern), but some tools are overloaded (e.g., 'full_run' does 7 things in sequence).
Ask a data-science question and get an answer grounded only in the knowledge base and past runs, with the sources it used.
Detect the ML task (classification/regression/clustering/timeseries/text), target, and special columns.
Run the full AutoDS lifecycle end-to-end (profile → task → insights → cleaning → leaderboard → train → evaluate) and return a complete report.
Retrieve relevant data-science best-practice guidance for a free-text query (RAG over the knowledge base + past runs).
Cross-validate all candidate models and return a ranked leaderboard for the detected task.
Load a dataset and return its profile (shape, dtypes, missing values, duplicates).
No output schemas documented. LLMs cannot predict response structure, forcing them to parse unstructured results and plan downstream calls blindly.
Parameter descriptions are minimal or absent. 'target' param appears in 6 tools with identical 1-line description; no guidance on when it's required vs optional, or how it affects task detection.
'full_run' combines 7 sequential operations (profile → task → insights → cleaning → leaderboard → train → evaluate). Violates single-responsibility principle; LLM cannot compose or retry individual steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Recommend a data-cleaning plan (columns to drop, imputation strategies, duplicates).
Run EDA and return each chart's title, the code behind it, an insight, and the saved image path.
Train a model (auto-picks the recommended one if unspecified), evaluate it, and return metrics + insights.
No error handling or recovery guidance. Tools fail silently (e.g., 'try: record_run() except: pass' in train_and_evaluate). LLM receives no indication of partial failures or what to do next.
File path parameter 'path' accepts arbitrary strings with minimal validation. _resolve() checks existence but does not prevent directory traversal or access to sensitive files outside data_dir.