Loading ToolBench
Verifies a WorkOS secret API key, then creates an Organization via the WorkOS Management API and prints its id + handle.
Four tools with clear verb-noun naming (verify, create, find, get) and reasonable descriptions (80-180 chars). All have input schemas with typed parameters and descriptions. However, output schemas are not documented, LLMs cannot predict return structure. Parameter descriptions lack format constraints (e.g., 'apiKey must start with sk_test_ or sk_live_'). Error handling is present in code but not surfaced in tool definitions. No tool annotations (readOnlyHint, destructiveHint). Missing pagination guidance for list operations.
Creates an Organization via POST /organizations or returns an existing one if found by name. Handles the already-registered case by looking first and returning the existing record instead of erroring out.
Searches for an existing organization by name using the WorkOS search endpoint. Returns null if not found or if the search request fails.
Retrieves an organization by id from the WorkOS API.
Verifies identity by listing one organization. A 200 proves the sk_ key is valid and tells us the key works in its environment (test vs production is encoded by the dashboard env).
Output schemas not documented. LLMs cannot predict what fields (id, name, domains, created_at, etc.) are returned or their types. Forces agents to guess structure.
Parameter descriptions lack format constraints. 'apiKey' should state 'Must be a WorkOS secret key prefixed sk_test_ (test env) or sk_live_ (production)'. Current description is vague.
No tool annotations. readOnlyHint missing on read-only tools (verifyApiKey, findExistingOrganizationByName, getOrganization). destructiveHint missing on createOrFetchOrganization (WRITE risk).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
findExistingOrganizationByName returns null on search failure but does not document this behavior. LLMs cannot distinguish 'not found' from 'search failed'.
createOrFetchOrganization accepts optional domainArg (comma-separated string) but does not document format or validation rules. LLMs may pass invalid domain lists.