Loading ToolBench
Two tools with complete schemas and reasonable descriptions, but significant gaps in error guidance, parameter validation, and output documentation. execute_sql lacks dry-run/timeout error recovery hints; get_table_info returns unstructured JSON without field-level documentation. Both tools accept free-form strings (sql, table) without injection safeguards documented. Descriptions are adequate (100-150 chars) but lack WHEN/WHY context. No tool annotations (readOnlyHint, destructiveHint). Error handling returns JSON strings rather than structured responses, forcing LLM parsing. Missing pagination for large result sets.
Execute a SQL statement against the SQLite database. Args: sql: The SQL statement to execute. dry_run: If true, returns the query plan (EXPLAIN QUERY PLAN) without executing. max_rows: Maximum number of rows to return. Defaults to 100.
Get complete schema and metadata for a SQLite table. Returns columns (name, type, nullable, default, primary key), DDL, indexes, foreign keys, and row count in a single call. Args: table: The table name to inspect.
SQL injection risk: execute_sql and get_table_info accept unescaped table/sql parameters. No input sanitization documented or visible. Descriptions do not warn LLM about injection hazards.
Error responses are JSON strings, not structured objects. LLM must parse JSON within JSON. No error classification (retryable vs fatal). Timeout errors lack retry guidance.
Output schema undocumented. execute_sql returns {results, row_count, truncated, total_rows} but LLM has no schema reference. get_table_info returns {table, columns, ddl, indexes, foreign_keys, row_count}, column structure not specified.
No pagination support. execute_sql caps at max_rows=100 but does not return total_rows consistently (only when truncated). Large queries may silently truncate without clear indication.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Missing tool annotations. execute_sql is destructive (INSERT/UPDATE/DELETE) but lacks destructiveHint. get_table_info is read-only but lacks readOnlyHint. Agents cannot infer safety without annotations.