Loading ToolBench
Governed text-to-SQL MCP server: the LLM writes the SQL, QueryGate validates, tenant-scopes, cost-checks, and executes it read-only.
QueryGate demonstrates solid tool design with clear naming (verb_noun pattern), comprehensive descriptions (100-300 chars), and proper JSON Schema with type constraints. All 6 tools have input schemas with typed parameters and descriptions. Tool annotations (readOnlyHint) are correctly applied. However, output schemas are not explicitly documented in the source code, responses are described narratively rather than as formal JSON Schema. Error handling is present but recovery guidance is minimal. The server follows good composition patterns (each tool has one responsibility) and includes rate limiting and cost-checking logic.
Return one model's schema: columns with types, declared joins, and metrics. Use these names verbatim when writing SQL for qg_run_query. `joins` lists the join keys this project actually declares. Use them rather than guessing which columns relate two tables. If a metric is listed, compute it exactly as its `expr` + `filter` say; that is the agreed business definition, not a suggestion.
List the real distinct values of a column so you filter on facts, not guesses. Runs a governed SELECT DISTINCT (the tenant filter is injected automatically).
Compute a defined metric. Prefer this over writing the aggregation yourself. A metric is a business definition a human owns ("completed_revenue" counts only completed orders). Ask for it by name and the server builds the SQL from that definition, so your number matches what everyone else gets. Write your own SQL with qg_run_query only when no metric covers the question.
Profile a table before querying it: row count, nulls, cardinality, ranges. Use this to understand the shape of the data (is this column mostly NULL? how many distinct values would a GROUP BY produce? what date range exists?) without pulling thousands of rows into context. Cheaper and more reliable than sampling by hand. Runs governed, so the numbers describe only the rows you may see.
Output schemas not formally documented. Tool descriptions mention response structure (e.g., 'models', 'metrics', 'count' for search_catalog) but no JSON Schema is visible in source. LLMs cannot reliably parse unstructured responses or plan downstream calls without explicit output schemas.
Error handling lacks recovery guidance. Code shows error_json() and with_tool_errors() decorators, but error messages in source do not include actionable next steps (e.g., 'Try qg_search_catalog() first' or 'Invalid column, call qg_get_table_stats() to see available columns'). Errors should guide LLM recovery.
Parameter 'limit' in qg_search_catalog and qg_get_filter_values lacks rationale in description. Why 50 max for search but 10000 for filter values? Descriptions should explain the constraint (e.g., 'Limited to 50 to avoid overwhelming the LLM context window').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | B | 70 | 2026-07-28+ | v2 |
Execute a SQL query against the warehouse, with governance. The LLM writes the SQL; QueryGate validates it (no DDL, no writes, no probe-able columns), tenant-scopes it (adds WHERE clauses), cost-checks it (EXPLAIN and refuse above the ceiling), and runs it read-only. Always call qg_search_catalog and qg_describe_model first to ground your query in real schema. Use real filter values from qg_get_filter_values, not guesses. If a metric exists, use qg_get_metric instead of writing the aggregation yourself. The response includes the result rows and a `provenance` footer (tables, plan cost, tenant scopes) for debugging.
Find warehouse models relevant to a question (server-side hybrid retrieval). Call this FIRST to discover which models to query, then qg_describe_model for exact columns. Search in English and expand with synonyms; the server fuses keyword and semantic matching, so intent works even when the exact word is absent. One or two searches is enough; pick the best match and move on. The response also lists any defined METRICS on the matched models. A metric is a human-owned definition of a business term (e.g. "completed_revenue"); prefer it and compute its `expr` (+ `filter`) rather than inventing your own.
qg_run_query description mentions 'no PII in the result' but does not explain how the server enforces this or what happens if PII is detected. Is it a warning, a block, or a filter? Ambiguity invites misuse.
qg_get_metric parameter 'dimensions' is an array but lacks description of expected element type (string column names?). Parameter descriptions should specify format and constraints for complex types.