Loading ToolBench
Node.js/Python wrapper for converting natural language to SQL using AI. Exposes database connection, schema analysis, and SQL generation tools via MCP protocol.
This server has 17 tools across two distinct implementations (nlsql_mcp_server and analytics_bird), creating significant inconsistency. Tools 1-10 are from the nlsql implementation with reasonable schemas and descriptions; tools 11-17 are from the analytics_bird BIRD benchmark suite with better-structured schemas. However, critical gaps undermine quality: (1) No output schemas documented for any tool, LLMs cannot predict what fields will be returned; (2) Tool names lack clear action verbs in some cases (e.g., 'connect' vs 'connect_database' duplication, 'analyze_schema' is vague); (3) Many parameter descriptions are generic ('Name of the table to sample' could be more specific about format/encoding); (4) No error handling guidance visible in source; (5) Security concerns with password parameters exposed directly. The average tool description length (~120 chars for nlsql tools, ~80 chars for analytics_bird tools) falls below the 194-char baseline for A+ tools. Only 4 of 17 tools have parameter descriptions that explain context beyond the parameter name itself.
Analyze database schema and structure
Open a database connection. Returns a `connection_id` to use in subsequent tool calls. For BIRD this will be a `sqlite:///...` DSN.
Connect to a database (SQLite, PostgreSQL, or MySQL)
Connect to the sample NBA database for testing
Return columns, types, primary/foreign keys, and row counts for the given tables. Call before writing SQL — the schema is rarely what the question text implies.
Close a connection opened with `connect`.
Disconnect from current database
NO OUTPUT SCHEMAS DOCUMENTED. LLMs cannot predict what fields tools return, forcing them to guess about downstream field availability. This violates the fundamental pattern that tools must document return types for proper chaining.
DUPLICATE/CONFLICTING TOOL NAMES. 'connect_database' (nlsql) vs 'connect' (analytics_bird), 'disconnect_database' vs 'disconnect'. LLMs will be confused about which tool to select. Names must be unambiguous within the same server.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 63 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 27 | 2024-11-05+ | v1 |
Run a SELECT/WITH/EXPLAIN/PRAGMA query. Read-only is enforced. Returns rows (up to `max_rows`, default 100) and elapsed time.
Execute SQL query on connected database
Get current database connection status
Get detailed database information including tables, columns, and relationships
Get sample data from a specific table
List tables on a connection. Optional regex `pattern` filters names.
Convert natural language question to SQL query using AI
Return up to `n` rows from `table` for inspection (default n=5).
Parse-only / EXPLAIN dry run. Returns {valid: bool, error?: str}. Use this to catch typos before `execute_sql`.
Validate SQL query syntax and structure
PASSWORD EXPOSED AS PARAMETER. 'connect_database' accepts a 'password' field as plain text input. Agent traces log all parameters, credentials must use server-side secret injection via environment variables or vault, never tool parameters.
VAGUE TOOL NAMES. 'analyze_schema' does not clearly indicate whether it inspects structure or performs optimization/validation. 'connect' is generic and undifferentiated from 'connect_database'. Action verbs should be specific: 'inspect_schema', 'analyze_schema_structure', 'describe_schema'.
GENERIC PARAMETER DESCRIPTIONS. Many parameters have minimal context. E.g., 'table_name' (get_table_sample) should clarify: 'Name of the table to sample (case-sensitive, must exist in connected database)'. Parameter descriptions average ~40 chars vs. 72-char baseline for A+ tools.
NO ERROR HANDLING GUIDANCE. No visible documentation of error classification, retry eligibility, or recovery paths. Tools like 'execute_sql_query' and 'natural_language_to_sql' can fail (invalid query, AI service down) but source shows no error response structures or guidance for agents.
INCONSISTENT TOOL INTERFACE ACROSS IMPLEMENTATIONS. Tools 1-10 (nlsql_mcp_server) and 11-17 (analytics_bird) serve the same domain (SQL databases) but have different naming conventions, parameter styles, and descriptions. E.g., 'execute_sql_query' vs 'execute_sql', 'get_connection_status' vs no equivalent. Agents will struggle with redundant, overlapping tools.
NO PAGINATION OR RESULT LIMIT DOCUMENTATION. 'get_database_info' and 'describe_schema' could return large result sets (many tables, many columns), but no pagination parameters or max-result guidance visible. LLMs may exhaust context with verbose responses.