Loading ToolBench
Single tool with incomplete schema documentation and minimal error handling. Tool name is action-oriented (execute_statement) but lacks depth in parameter constraints and output schema specification. Description is present but generic. Parameter descriptions exist but lack validation guidance, constraints, and format specifications. Output schema is mentioned in the docstring but not formally documented. No error handling patterns implemented. Security concern: hardcoded placeholder credentials in source code.
Execute a SQL statement against a Databricks SQL warehouse.
Missing input validation and constraint documentation. Parameters like 'warehouse_id', 'row_limit', and 'statement' lack explicit format specifications, valid ranges, and error guidance. No enums for optional fields.
Output schema not formally documented. Docstring mentions 'manifest', 'result_set', 'status' but provides no structured specification of field types, array structures, or field meanings. LLM cannot reliably parse response to extract needed data.
No error handling or recovery guidance. Tool directly returns raw API response (response.json()) without validating HTTP status codes, catching exceptions, or providing actionable error messages. LLM receives uncategorized errors with no guidance on retry eligibility or user-fixable steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 41 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 42 | - | v1 |
Hardcoded placeholder credentials in source code: DATABRICKS_HOST, DATABRICKS_TOKEN, WAREHOUSE_ID loaded from environment but WAREHOUSE_ID defaults to '<YOUR_WAREHOUSE_ID>' string literal. Credentials exposed as tool parameter (warehouse_id) defaults to placeholder, risking accidental exposure in logs.
Parameter 'parameters' (optional dict for SQL injection points) not validated or sanitized. No guidance on SQL injection prevention. LLM could be tricked into passing malicious payloads.
No pagination support. row_limit defaults to 10 but tool lacks next_cursor, offset, or page parameters. If result exceeds row_limit, agent cannot retrieve remaining rows without making blind subsequent calls.
Response contains raw API metadata (e.g. HTTP wrapper, status codes, query IDs) that wastes tokens. Should strip verbose fields and return only result rows and essential metadata (total_rows, has_more).