Loading ToolBench
Natural language interface for creating Apache Superset dashboards
SupersetAI provides 11 tools with consistent naming (verb_noun pattern) and reasonable descriptions (avg ~180 chars). All tools have input schemas with type definitions. However, parameter descriptions are sparse or missing for many inputs, output schemas are undocumented, and error handling guidance is absent. No tool annotations (readOnlyHint/destructiveHint) despite clear risk distinctions. The server follows basic structure but lacks LLM-optimization depth expected of production tools.
Create a bar chart (dist_bar) for comparing categories. Best for: comparing values across categories (e.g., revenue by region). Not ideal for: time-series data (use create_line_chart or create_timeseries_bar_chart instead), or part-of-whole with few categories (use create_pie_chart). Args: title: Chart title dataset_id: ID of the dataset to use metrics: Aggregation expressions. Use SQL syntax: "COUNT(*)", "SUM(column)", "AVG(column)", "MAX(column)", "MIN(column)", "COUNT(DISTINCT column)". Or a pre-defined metric name from the dataset. dimensions: Columns to group by (x-axis categories). Keep to 1-2 dimensions; too many creates unreadable charts. time_range: Time filter (e.g., "Last 7 days", "No filter") Returns the created chart information with ID and URL.
Create a line chart for trends over time. Best for: showing how a metric changes over a continuous time axis. Requires: a time/date column in the dataset (is_dttm=True). Not ideal for: categorical comparisons (use create_bar_chart). Args: title: Chart title dataset_id: ID of the dataset to use metrics: Aggregation expressions (e.g., ["AVG(price)", "SUM(sales)"]) time_column: Name of the datetime column for the x-axis. Must be a column with is_dttm=True from the dataset schema. dimensions: Optional grouping columns to create multiple lines time_grain: Temporal granularity (e.g., "P1D" for daily, "P1M" for monthly) time_range: Time filter (e.g., "Last 30 days") Returns the created chart information with ID and URL.
Execute a SQL query against a Superset database. Use this to explore data, validate column names, check cardinality, or run ad-hoc analysis before creating charts. Args: database_id: The database ID to run the query against sql: The SQL query string limit: Maximum number of rows to return (default 100) Returns columns, data rows, row count, and whether the result was truncated.
No tool annotations (readOnlyHint/destructiveHint/idempotentHint) despite clear risk distinctions. READ_ONLY tools (list_*, get_*, execute_sql, profile_*, suggest_*) and WRITE tools (find_or_create_dataset, create_*) are unmarked, forcing LLMs to infer safety from descriptions alone.
Output schemas are not documented. Tools return dict[str, Any] or list[dict[str, Any]] with no field descriptions. LLMs cannot plan downstream calls or extract required fields (e.g., dataset_id from create_bar_chart response) without trial-and-error.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | <=2025-11-25 | v2 |
Find an existing dataset or create a new one for a table. Args: database_id: The database ID containing the table table_name: Name of the table schema_name: Optional database schema name Returns the dataset information including ID and columns.
Get column information for a dataset. Args: dataset_id: The ID of the dataset Returns column names, types, and which are suitable for time/metrics.
List all available database connections in Superset. Returns a list of databases with their IDs and names. Use this first to discover what data sources are available.
List existing datasets in Superset. Args: database_id: Optional database ID to filter by Returns list of datasets that can be reused for charts.
List all schemas in a specific database. Args: database_id: The ID of the database to list schemas from Returns a list of schema names. For SQLite, this is typically ["main"]. For PostgreSQL/MySQL, this may include schemas like "public", "information_schema", etc.
List all tables in a specific database schema. Args: database_id: The ID of the database to list tables from schema_name: Optional schema name. If not provided, uses the first available schema. Returns a list of table names available in the database.
Profile a dataset to understand its data shape before creating charts. Runs exploratory SQL queries to gather row count, per-column cardinality, null counts, and sample values. Use this to make informed decisions about which chart type and columns to use. Args: dataset_id: The dataset to profile sample_size: Number of sample values to retrieve per column (default 5) Returns dataset metadata, row count, and per-column statistics.
Suggest appropriate chart types for a dataset based on its column profiles. Profiles the dataset (row count, column types, cardinality, nulls, sample values) and then recommends chart types ordered by relevance, along with example parameter hints for each. Use this BEFORE creating charts to pick the best visualization. Args: dataset_id: The dataset to analyze sample_size: Number of sample values to retrieve per column (default 5) Returns a dict with: - ``dataset_id``, ``table_name``, ``row_count``, ``columns`` — the profile - ``recommendations`` — ordered list of chart type suggestions with ``chart_type``, ``reason``, and ``suggested_params``
Parameter descriptions are missing or minimal for optional/complex params. 'schema_name' in list_tables says 'Optional schema name...' but does not explain what happens if omitted or how to discover valid schema names. 'sample_size' in profile_dataset lacks guidance on performance impact.
No error handling guidance. Tools return dict[str, Any] with no documented error cases, recovery steps, or actionable messages. If execute_sql fails (invalid SQL, timeout, permission denied), the LLM has no guidance on what to do next.
Metrics and dimensions parameters in create_bar_chart and create_line_chart accept free-form strings with no enum constraints or validation examples. LLMs may hallucinate invalid SQL expressions like 'TOTAL(sales)' instead of 'SUM(sales)'.