Loading ToolBench
MCP server providing tools for Databricks operations including SQL execution, warehouse management, and DBFS file operations
This server has 4 tools with basic schemas and descriptions, but critical gaps prevent higher scores. All tools have names starting with action verbs and input schemas are visible in server/tools.py. However, descriptions are generic (10-50 chars for most), parameter descriptions are minimal or absent, output schemas are not documented, and error handling provides no recovery guidance. The execute_dbsql tool is particularly concerning: it accepts free-form SQL strings without validation, lacks constraint documentation on the limit parameter, and returns unstructured result objects. Most tools lack parameter-level descriptions. This is typical D-to-low-C territory, functional but well below production quality.
Execute a SQL query on Databricks SQL warehouse.
Check the health of the MCP server and Databricks connection.
List files and directories in DBFS (Databricks File System).
List all SQL warehouses in the Databricks workspace.
execute_dbsql accepts free-form SQL without validation, injection protection, or input sanitization. No parameter descriptions for 'query', 'warehouse_id', 'catalog', 'schema', or 'limit'. LLM has no guidance on valid SQL syntax, warehouse ID format, or limit constraints.
No output schemas documented for any tool. Callers cannot predict response structure, making chaining impossible and forcing the LLM to parse free-form text. execute_dbsql returns {success, data, row_count} but format of 'data' is never specified.
Parameter descriptions are missing or trivial. 'warehouse_id' says '(optional, uses env var if not provided)' but does not describe format (alphanumeric?, length bounds?). 'limit' lacks range constraints (is 1000 acceptable? 1M?). 'query' has no description of expected SQL dialect or safety constraints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
Error responses in execute_dbsql provide no recovery guidance. Returning {success: False, error: '...'} tells the LLM nothing, should specify whether error is retryable, user-fixable, or fatal, and suggest next steps like 'Try list_warehouses() first.'
Tool descriptions are 10 - 50 characters, far below the 50 - 200 baseline for LLM optimization. 'Check the health...' and 'List all SQL warehouses...' lack context on WHEN to call these tools, what they return, and how they chain to other tools.
No pagination support or result limits enforced in list_warehouses and list_dbfs_files. If a workspace has hundreds of warehouses or deep directory trees, returning all results will blow context windows. Response should include page/offset, limit, and total_count.
execute_dbsql silently constructs SQL (e.g. 'USE CATALOG ...; USE SCHEMA ...;') without exposing this transformation in the description. LLM may expect a different query behavior than what actually executes. Should document that catalog and schema params are prepended to the query.
Databricks credentials (DATABRICKS_HOST, DATABRICKS_TOKEN) are injected via environment variables, which is correct. However, the tool accepts optional warehouse_id parameter, if an LLM accidentally logs this param, it could leak warehouse identifiers. Consider accepting only user-friendly names and resolving to IDs server-side.