Loading ToolBench
MCP server for querying DuckDB datasets from the dank-data catalog
Single tool 'query' with basic definition. Tool has a clear description and proper input schema with type and required flag, but lacks output schema documentation, error guidance, and proper parameter descriptions. No parameter description for the 'sql' parameter beyond generic text. No return value structure documented. No error handling guidance for LLMs (e.g., SQL errors, what to do on timeout). The tool lacks idempotency documentation and security context (read-only hint). Definition quality is below average for production use.
Execute a read-only SQL query against the DuckDB database and return CSV results
Missing output schema documentation. Tool returns CSV results but LLMs have no specification of what fields or structure to expect. This forces LLMs to guess at parsing and risks misunderstanding results.
Parameter 'sql' lacks actionable description. Current text 'The SQL query to execute' is generic and under 30 chars. Should specify format constraints, example valid queries, whether prepared statements are supported, size limits, and timeout behavior.
No error recovery guidance for LLM. SQL execution can fail (syntax errors, permission errors, timeouts). Tool provides no recovery instructions. LLM cannot self-correct or adapt strategy when query fails.
Missing tool annotations (readOnlyHint). While tool is read-only and marked Risk=READ_ONLY in metadata, this is not exposed in the schema. Tool should include readOnlyHint=true annotation so LLMs understand it is safe to retry.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 15 | - | v1 |
No documentation of output format or fields. Description says 'return CSV results' but does not specify column order, header inclusion, escaping rules, or maximum row count. LLMs cannot reliably parse CSV without explicit structure.
Tool name 'query' is too generic. Does not follow verb_noun pattern (e.g., 'execute_sql' or 'query_duckdb'). Generic name conflicts with LLM's internal reasoning patterns and provides no specificity about what resource or service is being queried.
No input validation constraints. SQL parameter accepts any string; no regex pattern, length limit, or character restrictions documented. LLMs may pass commands that cause timeouts or resource exhaustion.