Loading ToolBench
Command-line tool for testing Scalekit tool execution via the Connect SDK. Supports generating authorization links, executing tools, and fetching tool metadata.
Three tools with complete input schemas and descriptions, but significant quality gaps. Tool names lack clear action verbs (generate_link, execute_tool, proxy_request are vague). Descriptions are lengthy (200+ chars) and lack WHEN/WHY guidance. Parameters have types and descriptions, but no enums for constrained values (e.g., 'method' accepts GET/POST/etc but no enum). No output schemas documented. Error handling is present in code but not surfaced in tool definitions. Missing idempotency hints and permission scopes. No tool annotations (readOnlyHint, destructiveHint). Composition is weak: execute_tool and proxy_request overlap significantly, both execute actions on behalf of a user.
Verify the connected account is active, then execute the specified tool. For OAuth connections: uses get_or_create and prompts for authorization if not active. For non-OAuth connections: only checks if the account exists and is active — directs the user to the Scalekit Dashboard if not found or not active.
Check or create a connected account and generate an authorization link if not active. For OAuth connections: uses get_or_create and generates a magic link if not active. For non-OAuth connections: only checks if the account exists and is active — directs the user to the Scalekit Dashboard if not found or not active.
Make a proxied HTTP request through Scalekit on behalf of a connected account. Handles binary responses (files, exports) that gRPC-based execute_tool cannot.
Tool names lack action verbs. 'generate_link' is vague (generate what? authorization link? magic link?). 'execute_tool' is generic and does not convey the OAuth/non-OAuth branching logic. 'proxy_request' is too low-level. LLMs cannot infer intent from these names alone.
No output schemas documented. Tool definitions do not specify what fields are returned (e.g., does generate_link return {link, expires_at, status}?). LLMs cannot plan downstream calls or extract required data without knowing the response structure.
Missing tool annotations. No readOnlyHint, destructiveHint, or idempotentHint. execute_tool and proxy_request are marked WRITE but lack destructiveHint annotation. generate_link is marked READ_ONLY but lacks annotation. Agents cannot determine retry safety or side-effect scope.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
Parameter 'method' in proxy_request accepts free-form string with default 'GET', but no enum constraint. Should declare enum: [GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS]. LLMs may hallucinate invalid HTTP methods.
Overlapping tool responsibilities. execute_tool and proxy_request both execute actions on behalf of a user via a connection. execute_tool uses gRPC; proxy_request uses HTTP. This composition violates single-responsibility principle and forces LLMs to reason about which to use.