Loading ToolBench
Secure data access gateway for AI agents with policy enforcement, masking, and audit logging.
Server provides 7 well-named, read-only discovery and query tools with clear verb-noun naming (list_*, describe_*, preview_*, execute_*). All tools have descriptions (avg ~80 chars) and input schemas with typed parameters. However, descriptions are terse (under 100 chars for most tools), lack WHEN/WHY guidance, and omit dependency hints. Output schemas are not documented in the visible code. Error handling is absent from tool definitions. Parameter descriptions exist but are minimal (e.g., 'The datasource ID to list resources from'). No tool annotations (readOnlyHint, idempotentHint) despite all being read-only. Missing guidance on pagination limits, result caps, and chaining IDs across tools.
Describe one resource and its readable columns.
Run one guarded read-only SQL query.
List the datasources visible to the current runtime identity.
List readable resources underneath one datasource.
Find readable resources by one or more governance tag names.
List the governance tags visible to the current runtime identity.
Output schemas not documented. Tool definitions show input schemas but no return type documentation. LLMs cannot plan downstream calls or extract chaining IDs (e.g., resource_id from list_resources to pass to describe_resource).
Descriptions are terse (avg 70 chars, baseline 194 chars). Missing WHEN/WHY context. E.g., 'List the datasources visible to the current runtime identity' lacks guidance on when to call this vs list_resources, or what to do with the result.
No tool annotations despite all tools being read-only and idempotent. Missing readOnlyHint and idempotentHint annotations that would help LLMs reason about safety and retry behavior.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 67 | 2026-07-28+ | v2 |
Preview rows from one resource with policy and masking enforcement.
No pagination or result limit guidance. preview_resource and execute_query have 'limit' params with defaults (20, 100) but descriptions do not explain max bounds, when pagination is needed, or how to fetch next page.
No error handling guidance in tool definitions. No recovery hints (e.g., 'If resource not found, call list_resources() first'). No categorization of errors as retryable vs user-fixable.