Loading ToolBench
A deterministic, fail-closed write gate for LLM-generated SQL, exposed over MCP.
sqlgate defines 5 tools with clear, domain-specific names (sqlgate_check_sql, sqlgate_submit_sql, etc.) that follow verb_noun convention. All tools have descriptions (194 - 250 chars, within baseline 34 - 392 range). Input schemas are present with typed parameters (db: string, sql: string, title: string, ticket_id: string). However, parameter descriptions are minimal (5 - 20 chars: 'database name', 'SQL statement(s)'), falling short of the 72-char baseline. No output schemas are documented, LLMs cannot predict response structure. Error handling is implicit (NotImplementedError caught in get_status/list_pending) but lacks recovery guidance. No enums, constraints, or validation rules visible. Tool composition is sound (each does one thing), but descriptions lack WHEN/WHY context and dependency hints.
Rule-only verdict, no ticket: whitelist-template match each statement and return decision auto / needs_human_review / rejected plus per-statement reasons.
Look up a single ticket's status and review/execution result. Only backends with an audit trail support this.
List tickets currently parked for human review. Only backends with an audit trail support this.
Submit a batch through the full pipeline: only an entirely on-template batch that clears the row-count + precheck gates auto-executes (auto_executed); anything off-template parks for a human (needs_human_review); dangerous SQL is rejected with no ticket (rejected). The everyday backfill lane.
Human-authorized bypass — call ONLY after the statement's real impact has been measured and the user has explicitly authorized it. Lets an off-template UPDATE/INSERT execute (approved_executed). Hard guards approval cannot lift: rejected (dangerous) SQL, engine-broken SQL, impact that could not be measured, and any impact above approved_max_rows.
No output schemas documented. LLMs cannot predict response structure (fields, types, pagination). Responses return dicts with implicit 'error' or status fields, undocumented.
Parameter descriptions are minimal (5 - 20 chars: 'database name', 'SQL statement(s)'). Baseline is 72 chars. Missing format constraints, examples of valid input, and dependency hints (e.g., 'ticket_id from a prior submit call').
Error handling lacks recovery guidance. get_status/list_pending catch NotImplementedError and return {'error': '...'}, but do not tell the LLM what to do next or which executors support these operations.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
No input validation or constraint documentation. 'db' and 'sql' parameters accept free-form strings with no enum, regex, or length limits. LLMs may pass invalid database names or malformed SQL without guidance.
Tool descriptions lack WHEN/WHY context and decision criteria. E.g., sqlgate_submit_sql says 'everyday backfill lane' but does not explain when to use it vs sqlgate_check_sql or sqlgate_submit_sql_approved.