Loading ToolBench
Single tool 'query' with visible schema and basic descriptions, but significant quality gaps. The tool description is generic (18 chars, well below 34 char p10 baseline). No parameter descriptions at all in the visible registration, LLMs cannot infer what individual params control. Input schema is present (JSON Schema via Zod) but parameters lack actionable constraints and format guidance. No output schema documentation. Error handling returns generic 'Database error' messages without recovery guidance. Tool does not follow verb_noun naming (bare 'query' is ambiguous, 'execute_sql_query' would be clearer). No distinction between read-only and write operations despite the risk annotation claiming READ_ONLY. Connection parameters are exposed as optional inputs rather than injected server-side, creating potential for secrets leakage if defaults are overridden. Tool registration visible in both examples/simple-server.ts and src/index.ts, but source code is incomplete in the provided snippet (src/index.ts cuts off mid-method).
Execute SQL queries against an MSSQL database
Tool description is only 18 characters ('Execute SQL queries against an MSSQL database'), far below the 34-char p10 baseline. Too short to guide LLM tool selection; lacks WHEN to use, prerequisites, or state-changing behavior notification.
No parameter descriptions visible in the tool registration. The Zod schema in examples/simple-server.ts defines types but no descriptions, LLMs cannot infer what 'encrypt', 'trustServerCertificate', or 'database' parameters actually control. Schema comments are missing entirely.
Tool name 'query' is ambiguous and generic. Does it SELECT, INSERT, UPDATE, DELETE, or execute stored procedures? Does it support transactions? 'execute_sql_query' or 'execute_mssql' would be clearer. Current name violates verb_noun convention, should start with 'execute'.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 32 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 42 | - | v1 |
No output schema documented. LLMs cannot predict what fields the tool returns or how to chain results into downstream calls. The implementation returns JSON.stringify(result.recordset), but recordset structure is undocumented.
Error handling returns only 'Database error: {message}' with isError=true. No recovery guidance, LLM does not know if error is retryable, user-fixable, or fatal. No suggestions for next steps (e.g., 'check connection parameters', 'verify query syntax').
Risk annotation claims READ_ONLY, but tool accepts any SQL string including INSERT, UPDATE, DELETE, and DROP. No validation to enforce read-only queries. Tool description does not warn that write operations are possible, violating the risk declaration.
Connection credentials (username, password) are exposed as optional tool parameters. If an agent invokes without explicit params, defaults are used, but this design invites secrets in traces. Best practice: inject via environment variables or vault, not as tool parameters. See pattern:secret-injection.
No constraints on query parameter: no length limit, no regex pattern, no format documentation. LLMs could pass 100MB of SQL or malicious payloads. Should validate for SQL injection risks and enforce reasonable bounds (e.g., 64KB max).
Parameter 'port' has no range constraint, LLM could pass port 999999 or negative values. Should declare 1 - 65535 range with description.
Mutual dependency between connectionString and (host, username, password) is documented only in Zod refine() validator logic, not in any parameter description. LLMs cannot read schema constraints, only text descriptions. Must document in each param's description that these are mutually exclusive.
No pagination support. Tool executes arbitrary SQL including SELECT * FROM huge_table, results could be tens of thousands of rows, blowing context window. Should accept LIMIT and OFFSET parameters or cap result set at a reasonable default (e.g., 1000 rows) with guidance on pagination.