Loading ToolBench
The server defines 4 tools with explicit names, descriptions, and input schemas. Tool naming follows verb_noun convention (list_*, get_*, execute_*) and is appropriate. However, descriptions vary significantly in quality and completeness. Parameter descriptions are present but inconsistent, some are detailed (with constraints and guidance like the execute_query description explaining result limits), while others are minimal. Output schemas are partially documented (only through type response examples in code, not formal JSON Schema declarations). Error handling is minimal, no guidance on recovery, categorization, or actionable error messages. Security-wise, credentials are handled via Azure SDK (not exposed in parameters), but audit trails and permission checks are not evident. The execute_query tool has extensive guidance in its description about result truncation and optimization strategies, setting it apart. Overall, this is a borderline C/D server with one strong description that lifts it slightly.
Execute a read-only query. Ask the user for permission before executing the query. It has to be a valid KQL query. Write queries are not allowed. Result truncation is a limit set by default on the result set returned by the query. Kusto limits the number of records returned to the client to 500,000, and the overall data size for those records to 64 MB. When either of these limits is exceeded, the query fails with a partial query failure. Exceeding these limits will generate an exception. Reduce the result set size by modifying the query to only return interesting data. There are several strategies to avoid this. 1/ Use the summarize operator group and aggregate over similar records in the query output. 2/ Potentially sample some columns by using the take_any aggregation function. 3/ Use a take operator to sample the query output. 4/Use the substring function to trim wide free-text columns. 5/ Use the project operator to drop any uninteresting column from the result set.
Get the schema of a specific table in an Azure Data Explorer database
List all databases in a specific Azure Data Explorer cluster
List all tables in a specific Azure Data Explorer database
Missing output schema documentation. Tools return structured JSON (e.g., ListDatabasesResponse with 'databases' array), but no formal schema definitions are visible in the source. LLMs cannot infer return types without explicit documentation.
Parameter descriptions lack constraint details. 'cluster' parameter says 'Name of the cluster' but does not specify format (e.g., 'short name like myCluster or FQDN like myCluster.kusto.windows.net'). 'query' parameter in execute_query does not specify KQL syntax or length limits.
Minimal error handling and recovery guidance. No error categorization (retryable vs user-fixable vs fatal). list_databases returns 'cluster name missing' but does not guide the LLM on how to recover. execute_query warns about result limits but does not return structured error codes that let the agent self-correct.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 50 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 44 | - | v1 |
No result pagination or limits enforced. list_tables and list_databases could return thousands of items without pagination support. execute_query relies on Kusto's 500K row/64MB limits but does not offer offset/limit parameters to the agent.
Parameter 'cluster' description instructs users to ask for the name if not available, but this guidance is too vague. No format clarification (is it 'mycluster' or 'mycluster.kusto.windows.net'?) and no examples provided. This forces LLMs to guess or ask follow-up questions.
No permission gates or scope declarations. Tools execute Kusto queries against the authenticated user's clusters/databases, but no explicit documentation of required permissions (e.g., 'read:kusto', 'admin:cluster'). Audit trails are not evident.