Loading ToolBench
A Model Context Protocol (MCP) server for AI solutions to communicate with Databricks
The server provides 10 tools with complete input schemas and descriptions. Tool names follow verb_noun conventions appropriately (list_*, get_*, execute_*). However, several critical quality gaps reduce the score: (1) Output schemas are not documented in the visible code, no structured return type specifications provided, only brief descriptions mentioning 'results as a markdown table' or generic listings. (2) Parameter descriptions lack depth and specificity, most default to 'Target workspace name (optional, uses default if omitted)' with minimal guidance on what each tool returns or when to use it. (3) Error handling is minimal in the visible code; while error_handler.py exists, the tool definitions themselves do not include recovery guidance or categorization. (4) Description lengths vary wildly, some tools have adequate descriptions (databricks_execute_sql_query ~120 chars) while others are terse (databricks_list_catalogs ~55 chars). (5) No documented limitations, constraints, or side effects for destructive operations (execute_sql_query marked WRITE risk has no confirmation/dry-run pattern visible). The codebase shows good structure (connection pooling, caching, config management) but the tool interface itself lacks LLM-optimized depth.
Execute a SQL query against Databricks and return results as a markdown table. Use when: running ad-hoc SQL analysis, exploring data, or answering ques
Get activity events for a specific cluster, including scaling, errors, and user actions.
Get detailed information about a specific Unity Catalog table, including columns and metadata.
Get detailed status and health information for a specific SQL warehouse.
List all Unity Catalog catalogs in the Databricks workspace.
List all Databricks clusters with their current state. Use when: checking compute availability, diagnosing job failures, or auditing cluster configurations.
Output schemas completely undocumented. Tools describe return values in prose ('results as markdown table', 'list of schemas') but no structured schema or field definitions provided. LLMs cannot reliably parse unstructured text responses or plan downstream tool calls without knowing return field names and types.
Parameter descriptions are shallow and generic. Most optional 'workspace' params repeat 'Target workspace name (optional, uses default if omitted)' with no guidance on workspace resolution, format, or valid values. Many params lack range constraints (e.g., 'limit' and 'offset' have no stated min/max, though code shows DEFAULT_PAGE_LIMIT and clamping logic).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 56 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 41 | - | v1 |
List all schemas in a Unity Catalog catalog.
List all tables in a Unity Catalog schema.
List all volumes in a Unity Catalog schema.
List all SQL warehouses with their current state. Use when: diagnosing SQL connectivity issues — shows if warehouses are RUNNING, STOPPED, or STARTING.
databricks_execute_sql_query marked WRITE risk but no confirmation/dry-run pattern visible. Tool accepts arbitrary SQL including INSERT, UPDATE, DELETE, but no dry-run capability, parameter validation hints, or rollback guidance. Description does not warn of irreversible consequences or suggest read-only checking.
Pagination model partially documented. 'limit' and 'offset' params present and typed as integers, but no stated range constraints, default values, or maximum fetch limits visible in tool definitions. Code hints at DEFAULT_PAGE_LIMIT and clamp_pagination() but this is not surfaced in the tool schema.
Parameter format and enum guidance missing. 'event_types' in databricks_get_cluster_events accepts comma-separated strings with no documented values. 'full_table_name' format ('catalog.schema.table') is inferred, not stated. 'warehouse_id' and 'cluster_id' formats undocumented.
Error recovery guidance absent. No tool definitions include hints for recovery paths (e.g., 'If cluster not found, call list_clusters() first'). Error formatting logic exists in code (get_error_handler, _format_sql_error) but is not documented in tool descriptions.
Minimal tool composition guidance. Dependency chains exist (catalog → schema → table) but are not documented. No clear narrative of when to call discovery tools (list_*) vs detail tools (get_*), or how tool outputs chain into downstream parameters.