Loading ToolBench
A MCP (Model Context Protocol) server for interacting with dbt resources.
41 tools with significant quality gaps. Most tools (35/41) lack parameter descriptions entirely, parameters have null descriptions in schema. Tool descriptions are present but generic (avg ~40 chars, well below 194-char baseline). No output schemas documented. Placeholder tools (dbt_compile, dbt_run, dbt_test) are duplicates of existing tools. No tool annotations (readOnlyHint/destructiveHint). Error handling not visible in tool definitions. Naming is generally good (verb_noun pattern), but parameter quality is critically weak.
Build dbt models and tests.
Cancel a job run.
Compile dbt models.
Compile dbt models (placeholder)
Run dbt models (placeholder)
Run dbt tests (placeholder)
Generate dbt documentation.
Execute SQL using dbt SQL tools.
35 of 41 tools have parameters with null descriptions. LLMs cannot infer parameter meaning from names alone (e.g., 'status' could mean HTTP status, job status, or boolean flag). This violates the critical pattern that every parameter must have a non-empty description.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | <=2025-11-25 | v2 |
Generate dbt model YAML.
Generate dbt source YAML from database schema.
Generate dbt staging model.
Get all models from dbt Discovery.
Get dimensions for metrics.
Get entities for metrics.
Get details for a specific exposure.
Get exposures from dbt Discovery.
Get details for a specific job.
Get a specific job run artifact.
Get details for a specific job run.
Get focused error information for a failed job run.
Get mart models from dbt Discovery.
Get compiled SQL for metrics.
Get child models.
Get details for a specific model.
Get model health information.
Get parent models.
List dbt resources.
List artifacts for a job run.
List jobs in an account.
List runs in an account.
List metrics from dbt Semantic Layer.
Parse dbt project.
Query metrics from dbt Semantic Layer.
Retry a failed job run.
Run dbt models.
Get server status and capabilities
Show dbt query results.
Run dbt tests.
Test that the server is running
Convert text to SQL using dbt SQL tools.
Trigger a job run.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required data without knowing what fields to expect. This is a critical gap for tool composition.
Three placeholder tools (dbt_compile, dbt_run, dbt_test) duplicate existing tools (compile, run, test) with identical functionality but marked as placeholders. This creates ambiguity for LLMs and violates the single-responsibility principle.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present. Tools marked as DESTRUCTIVE (build, run, test, trigger_job_run) should have destructiveHint=true to signal to agents that these operations have irreversible consequences.
Tool descriptions are generic and very short (avg ~40 chars vs 194-char baseline). Descriptions like 'List jobs in an account' lack context for when to use the tool vs similar tools, and do not explain what the tool returns or any prerequisites.