Loading ToolBench
Local, governed analytics tools for MCP clients
Server has 20 tools with explicit schemas and descriptions, but quality is inconsistent. Most tool names follow verb_noun pattern (begin_analysis_task, complete_analysis_task, get_schema, list_sources). Descriptions are present but brief (avg ~60 chars, below the 194-char baseline). Parameter descriptions exist but lack detail on constraints, formats, and valid ranges. Output schemas are not documented, callers cannot predict response structure. Error handling is minimal; most tools return generic error dicts without recovery guidance. No tool annotations (readOnlyHint/destructiveHint) despite clear risk classifications in metadata. SQL-related tools (explain_sql, validate_sql, validate_and_execute) accept JSON-encoded parameters as strings, forcing LLMs to construct JSON manually, error-prone and non-idiomatic.
Begin an analysis task with a title and objective
Cancel an analysis task
Compare metrics across two time periods
Complete an analysis task with findings and optional next steps
Create a seeded retail demo only after the user explicitly approves it
Detect changes between baseline and current metrics
Evaluate an analysis task
Output schemas not documented. Tools return dicts but callers cannot predict field names, types, or structure. Breaks downstream tool chaining and forces LLMs to guess response format.
SQL parameter handling via JSON-encoded strings (parameters_json) is non-idiomatic and error-prone. LLMs must manually construct JSON; any syntax error fails silently. Should accept structured objects or provide a parameter builder tool.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Explain a SQL query with optional parameters
Get the schema for a data source
List available analysis recipes
List all available data sources
Validate this project's source configuration without revealing its URL
Run a named metric with optional task association
Get the schema state for a data source
Suggest table joins for a data source
Get the timeline of events for an analysis task
Validate and execute a SQL query with optional parameters, limit, and offset
Validate a SQL query with optional parameters
Verify observability evidence exists
Explain the project source-file configuration requirement
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk classifications in metadata. Agents cannot distinguish safe reads from destructive writes without parsing descriptions.
Parameter descriptions lack constraint details. E.g., 'task_id' has no format hint; 'limit' and 'offset' in validate_and_execute lack min/max bounds. LLMs cannot validate inputs before calling.
Error handling returns generic dicts without recovery guidance. E.g., 'error: TASK_UNKNOWN' tells LLM nothing about next steps. Should include actionable messages like 'Task not found. Call list_tasks() to see available tasks.'