Loading ToolBench
A VS Code extension that gives dbt the same kind of language support that most programming languages have had for years. Now it's here, for dbt Core.
dbt Anvil exposes 14 tools with basic naming and descriptions, but critical gaps undermine production readiness. Tool names follow verb_noun convention (get_*, list_*, run_*), which is good. However, descriptions are inconsistent: some are adequate (40-80 chars), others are minimal (20-30 chars). Most critically, NO input schemas are visible in the provided source code, only parameter names and brief descriptions appear in the tool list. Without explicit JSON Schema definitions showing types, constraints, and required fields, LLMs cannot validate inputs or understand parameter relationships. Output schemas are entirely undocumented. Error handling is absent, no guidance on retryability, user-fixable vs fatal errors, or recovery paths. The run_model tool (WRITE risk) lacks confirmation/dry-run patterns. Parameter descriptions lack format constraints (e.g., unique_id format, filter syntax). No evidence of pagination support for list_* tools despite dbt projects potentially having hundreds of models.
Compile a dbt model to SQL
Get detailed information about a user-defined function
Get the upstream and downstream lineage for a dbt resource
Get detailed information about a specific dbt macro
Get the columns and their descriptions for a dbt model
Retrieve metadata about a dbt resource (model, source, macro, or function) by unique ID or name
List all user-defined functions in the dbt project (dbt 1.11+)
List all dbt macros in the project
NO input schemas visible in source code. Tool definitions show parameter names and brief descriptions only; no JSON Schema with types, constraints, or required field declarations. This violates the foundational pattern:tool requirement and prevents LLM input validation.
Output schemas completely undocumented. No specification of return types, field names, or structure for any tool. LLMs cannot plan downstream calls or extract required data without knowing what fields to expect.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | <=2025-11-25 | v2 |
List all dbt models in the project with their metadata
List all dbt sources in the project with their metadata
Parse the dbt project and rebuild the manifest
Execute a dbt model run command
Search for dbt resources by name or description
Execute dbt tests for a model
run_model (WRITE operation) lacks confirmation/dry-run pattern. Destructive operations should support a dry-run or confirmation step to prevent accidental execution. No error guidance on retryability or recovery.
list_* tools (list_models, list_sources, list_macros, list_functions) lack pagination parameters (limit, offset, page_size) and return structure documentation. dbt projects can have hundreds of models; unbounded results will exhaust context windows.
Parameter descriptions lack format constraints and validation rules. E.g., unique_id format (e.g., 'model.project.name'), filter syntax, select selector syntax not documented. LLMs cannot self-correct invalid inputs without explicit constraints.