Loading ToolBench
The server defines 6 tools with reasonable naming (verb_noun pattern) and basic descriptions, but has critical gaps in schema completeness, parameter validation, and error handling. Tools use Zod for schema validation in source code, but descriptions are generic (10-100 chars), and critical context about query execution risks, write-mode restrictions, and recovery paths is missing. Parameter descriptions exist but lack constraints (e.g., no mention of SQL injection risk, query complexity limits, or result size bounds). Output structure is undocumented, tools return plain text JSON without explaining field names or pagination. The 'execute' tool lacks confirmation or dry-run patterns despite being destructive. Error handling is minimal (only 'Storage not found').
Get schema/structure information for a collection/table
Execute write operations (INSERT, UPDATE, DELETE for SQL; insert, update for NoSQL)
Get detailed information about a specific storage connection
List collections/tables/datasets in a storage (tables for RDB, collections for NoSQL, datasets for Athena)
List all configured storage connections (RDB, NoSQL, Athena, etc.)
Execute a query on any storage (SQL for RDB/Athena, NoSQL query for MongoDB, etc.)
execute tool (WRITE) lacks confirmation/dry-run pattern and destructive operation warning. No description states that calls are irreversible (DELETE/UPDATE/INSERT). LLMs cannot distinguish between safe-to-retry and destructive operations.
query and execute tools accept arbitrary SQL/NoSQL as free-form strings with no constraints, format description, or injection guards documented. Parameter descriptions do not warn about SQL injection risk or query complexity limits.
No output schema documentation. Tools return text JSON responses, but field names (e.g., 'storages', 'result', pagination fields) are undocumented. LLMs cannot reliably extract data for downstream calls.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 56 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 56 | - | v1 |
Result pagination and limits are not addressed. query and execute tools may return unbounded result sets, risking context window exhaustion. No limit parameter or cursor documented.
Error handling is minimal. Only 'Storage not found' is handled. No guidance for query failures (syntax errors, timeout, permission denied), no retryability classification, no actionable recovery steps.
Parameters 'parameters' (in query/execute) are typed as 'any' with minimal documentation. LLMs cannot infer what structure is expected or what data types are valid.
Descriptions are generic and lack context for LLM selection. E.g., 'Execute a query on any storage' does not explain when to use query vs execute, what 'query' means for MongoDB, or what happens on failure.
No documentation of storage_id enum or discovery mechanism. LLMs cannot learn which storage IDs are valid without first calling list_storages. Descriptions should hint: 'Call list_storages() first to get valid IDs.'