The server provides 19 tools with mostly consistent naming following verb_noun patterns (list_, get_, create_, update_, archive_, delete_, publish_, execute_, add_). Tool descriptions are present and generally adequate (median ~80 chars), but lack action-oriented context about WHEN to use tools vs alternatives. Input schemas are properly typed with JSON Schema, but parameter descriptions are sparse and generic. No documented output schemas. Error handling is present but minimal, no recovery guidance or actionable error messages visible. The tool set is well-composed with clear one-to-one responsibilities (e.g., separate create_visualization, update_visualization, delete_visualization), and naming disambiguates effectively (e.g., list_queries vs get_query vs get_query_result). However, output responses lack field documentation, LLMs must infer what fields to expect from downstream tool calls. Security and secrets handling not visible in provided code. Per-tool scores average 62, primarily due to missing output schema documentation and sparse parameter descriptions.
Output schemas not documented. Tool responses (e.g., list_data_sources, get_query) lack field-level documentation. LLMs cannot infer which fields are returned or what downstream tools expect. This forces agent guesswork and risks broken tool chains.
Parameter descriptions are generic and lack actionable constraints. E.g., 'max_age' in execute_query has no explanation of units (seconds vs milliseconds), valid range, or behavior. 'options' in visualization tools has no structure guidance. Type constraints (enums, ranges, formats) are missing from descriptions.
Document output schemas for all tools. For each tool, specify returned fields, types, and ranges. E.g., list_data_sources returns {id: number, name: string, type: string, paused: boolean}. This enables tool chaining and downstream planning.
Add constraint descriptions to all parameters. For max_age in execute_query, state: 'Maximum age of cached results in seconds (0 = disable cache, default = 3600)'. For visualization.options, describe the expected object structure (keys, value types).
Rewrite tool descriptions to include WHEN and WHY. E.g., archive_query: 'Soft-delete a Redash query. Use this when retiring a query but keeping history. Note: linked visualizations are orphaned but not deleted. This operation is idempotent.' For comparison, update_query: 'Modify query definition, name, or data source. Use when the query logic or scope has changed.'
Add confirmation/dry-run support for destructive operations. For archive_query, archive_dashboard, delete_visualization, delete_widget, add an optional 'dry_run' boolean parameter that validates but doesn't commit. Or add a separate 'confirm_delete' tool that accepts a confirmation token from a prior deletion request.
Document error scenarios with recovery hints in tool descriptions. E.g., execute_query: 'Returns job status. If status is error, check data_source_id is valid and SQL syntax matches the source's dialect (PostgreSQL, MySQL, etc.). Call list_data_sources() to verify available sources.'
Clarify pagination behavior. For list_queries, document: 'Returns paginated results with total count. Use page_size (1 - 100, default 25) and implement offset-based iteration. If more than 100 results, loop through pages until count is exhausted.'
Score history
Overall score trend
↑ 1 points across a rubric change (v1 → v2)
60/100
Scored
Grade
Overall
Spec posture
Rubric
2026-09-22
C
60
2026-07-28+
v2
2026-03-09
D
59
-
v1
delete_widgetdestructiveauthsource verified75/100
Delete a widget from a Redash dashboard
execute_queryread onlyauthsource verified82/100
Execute a SQL query against a Redash data source and return results
Update a widget on a Redash dashboard (text or options only)
Tool descriptions lack WHEN-to-use context and dependencies. E.g., archive_query says 'Delete a Redash query' but doesn't explain if it's idempotent, what happens to linked visualizations/dashboards, or when to call it vs update_query. Descriptions don't guide tool selection when multiple similar tools exist.
No error handling guidance visible in tool descriptions. Errors like 'query_id not found', 'invalid visualization type', or 'data source syntax error' are not documented with recovery hints. LLMs lack guidance on how to self-correct or what to try next.
Pagination support unclear. list_queries and list_data_sources accept page_size but response structure and total count handling not documented. Large result sets could exceed context windows without proper pagination guidance.
Tool composition gaps: create_dashboard returns a dashboard_id (presumably), but subsequent tools (add_widget, update_dashboard) require dashboard_id. Response chaining not documented, unclear which fields are returned by which tools to feed into downstream calls.
create_dashboardcreate_querycreate_visualization
Document required response fields for chaining. E.g., create_query must return query_id; create_dashboard must return dashboard_id; create_visualization must return visualization_id. Specify these in tool descriptions so agents know what to extract.
Add idempotency statements. E.g., create_query with identical params: does it create a duplicate or return existing? Archive operations: can they be called multiple times safely? Idempotency is critical for agent retry logic.
Separate mutually exclusive parameters in tool descriptions. E.g., add_widget accepts visualization_id OR text, not both. State: 'Provide either visualization_id (to embed a saved visualization) or text (for a text widget), not both.'
Add natural-language identifier support. If tools currently require data_source_id, consider accepting data_source_name as an alternative and resolving it via list_data_sources. This reduces agent friction.