Loading ToolBench
An MCP server that gives AI assistants the ability to connect to, query, profile, and monitor data sources — turning any LLM into an interactive data detective.
Data Detective has 16 tools with mostly complete schemas and reasonable descriptions. All tools are explicitly registered via @mcp.tool() decorator with non-empty descriptions. However, several critical issues limit the overall score: (1) parameter descriptions are inconsistent and sometimes minimal, (2) output schemas are not documented, (3) error handling lacks recovery guidance, (4) security considerations around file paths are not addressed, (5) tool composition could be improved (e.g., create_chart and save_recipe do overlapping work). The server demonstrates solid foundational quality but lacks production-grade polish. Average per-tool score: 62.
Compare schemas of two tables and report added, removed, or type-changed columns.
Connect a data source (SQLite DB, Parquet file, or CSV).
Run a SQL query and render the results as a chart image.
Detect statistical anomalies in a numeric column using z-scores.
Scan a table for data quality issues: duplicates, high null rates, constant columns, unexpected negatives.
Disconnect a previously registered data source.
Export SQL query results to a Parquet or CSV file.
Output schemas are not documented for any tool. LLMs cannot predict response structure, field names, or types. This forces them to guess what data is available after each call and risks extraction errors.
Error handling provides no recovery guidance. Errors are caught and wrapped in JSON with type and message, but lack actionable next steps. Example: if 'sql' is invalid, the error should suggest 'Check the query syntax' or 'Call list_tables() to see available tables'.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 57 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 10 | - | v1 |
Get a random sample of rows from a table.
Get column names, types, and nullability for a table.
List all connected data sources and their tables.
List all tables and views across all connected sources.
Generate a detailed profile of a table: row count, null rates, distributions, and per-column stats.
Regenerate a chart from a previously saved recipe file.
Execute a SQL query against any connected data source.
Persist the most recent chart recipe to disk as a JSON file.
High-level summary of all connected data: source count, table count, total rows, and column names.
File path parameters (path in connect_source, output_path in export_data/create_chart/save_recipe, recipe_path in replay_chart) lack validation and security guidance. No mention of path traversal prevention, absolute vs relative paths, or sandboxing. This is a critical security gap.
Parameter descriptions are inconsistent in quality. Some are minimal (e.g., 'The table to profile.'), others provide good context. Descriptions under 10 words lack sufficient context for LLM decision-making.
Tools create_chart and save_recipe have overlapping responsibilities. create_chart renders a chart and optionally saves it; save_recipe persists a previous recipe to disk. This composition lacks clear separation of concerns. Consider consolidating into chart management (render, save, load).
Several parameters accept free-form strings where enums would be safer. Example: source_type in connect_source is documented as "One of 'sqlite', 'parquet', 'csv'" but lacks an enum constraint; chart_type in create_chart is documented as "One of 'bar', 'line', 'scatter', 'histogram'" but lacks constraint. Enums prevent hallucinated values.
No pagination support for tools that return lists (list_sources, list_tables, run_query results). Without limit/offset and total counts, large result sets could blow the context window. run_query has a limit parameter (good), but list_sources and list_tables do not.
Tool descriptions lack WHEN/WHY context. For example, 'profile_table' describes WHAT it does but not WHEN to use it vs 'get_sample' or 'run_query'. LLM decision-making is hindered when context is missing.
No idempotency guarantees documented. Tools like export_data, create_chart, save_recipe are write operations. Agents retrying on ambiguous failures could produce duplicate artifacts if these tools are not idempotent.
Output field naming and chaining IDs are not documented. For example, what does list_sources() return? Does it include table names, record counts? Without knowing the response structure, downstream tools cannot chain calls effectively.