Loading ToolBench
Doris CLI is a STDIO-only tool with 11 commands exposed as tools. Tool naming follows verb_noun convention (use, auth_add, auth_list, sql, tablet, profile_list, profile_get, profile_diff, profile_history). Descriptions are present but brief (10-50 chars for most tools). Input schemas are visible in the source with type definitions and parameter descriptions. However, output schemas are not documented anywhere in the provided code. Error handling is minimal, no recovery guidance, no categorization of retryable vs fatal errors. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels (READ_ONLY, WRITE, DESTRUCTIVE). Parameter descriptions are sparse; many lack format/constraint details. No pagination support visible for list tools (profile_list accepts limit but no offset/cursor). Composition is reasonable, tools are single-purpose, but chaining is not optimized (e.g., profile_get requires query_id but profile_list does not return query_id in visible output).
Add a new environment
List all configured environments
Remove an environment
Test connection and show status
Compare two query profiles (slow vs fast run)
Get a specific query profile (default: summary + plan + operators + table context)
Show execution history for a query pattern (from audit_log)
List recent query profiles
No output schemas documented. LLMs cannot plan downstream tool calls or extract required fields (e.g., does profile_list return query_id for use with profile_get?). Every tool response is opaque.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite explicit risk levels in metadata (READ_ONLY, WRITE, DESTRUCTIVE). Agents cannot distinguish safe from dangerous operations.
Descriptions are too brief (40-60 chars average). 'List all configured environments' and 'Test connection and show status' lack context for when/why to call them. Rubric baseline: 194 chars average for A+ tools.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | <=2025-11-25 | v2 |
Execute SQL queries
Tablet and bucket analysis
Switch active environment
No error handling guidance. Tools like auth_remove (DESTRUCTIVE) and sql (WRITE) have no recovery hints, retryability classification, or confirmation patterns. Agents cannot self-correct on failure.
Parameter descriptions lack format/constraint details. 'SQL query to execute' does not specify max length, allowed syntax, or timeout. 'Environment name' does not specify character restrictions or naming rules.