Loading ToolBench
MCP-style HTTP server that multiplexes N database/REST backends behind one auth-protected endpoint, with LRU cache, SQL allow-list, rate limiting, and per-target health checks.
Three tools with minimal descriptions and incomplete schemas. Tool names lack action verbs (health, list_targets, query are generic). Descriptions exist but are brief (34-89 chars, below the 50-200 char LLM-optimized baseline). The 'query' tool has a partial schema with three parameters, but 'health' and 'list_targets' have empty input schemas ({}), violating the requirement that every parameter be described. No output schemas documented. Error handling exists in code (classifyError, audit logging) but is not surfaced in tool definitions. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite all three being READ_ONLY. Security is handled server-side (AUTH_TOKEN, rate limiting, SQL validation) but not declared in tool scopes.
Check server health and per-target status. Returns service info, port, target statuses, and cache statistics.
List all available database targets and cache statistics. Requires authentication.
Execute a read-only SQL query against a specified target database. Supports caching, result summarization for large result sets, and query timeout.
Tool names lack action verbs. 'health' and 'list_targets' are generic; 'query' is vague. LLMs infer intent from names, 'check_health', 'list_database_targets', 'execute_sql_query' would be clearer.
No output schemas documented. LLMs cannot plan downstream calls or extract fields without knowing response structure. health returns {ok, service, port, targets, cache}; list_targets returns {ok, targets, cache}; query returns {ok, target, rowCount, truncated, elapsedMs, cache, [rows|summary]}.
No tool annotations despite all three being READ_ONLY. Missing readOnlyHint annotation prevents LLMs from reasoning about safety and retry semantics.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 35 | 2026-07-28+ | v2 |
query parameter 'sql' lacks format/pattern constraints. Description says 'SELECT/SHOW/DESCRIBE/EXPLAIN/WITH query' but no enum or regex enforces this. LLMs may pass INSERT/UPDATE/DELETE despite validation.