Loading ToolBench
Single tool with well-structured schema and clear descriptions. Tool follows verb-noun naming (refresh_aws_sso_token) and includes proper parameter validation with regex patterns and length limits. Input schema is explicitly visible with type definitions and descriptions for all parameters. However, output schema is not documented, the tool returns JSON text without a formal schema definition. Error handling lacks actionable recovery guidance (e.g., suggests retry strategy or alternative tools). The tool description is adequate but could be more prescriptive about when to use it vs. alternative authentication paths.
Initiates AWS SSO login flow to refresh expired authentication tokens. This will open a browser window for the user to complete authentication. Use this when AWS operations fail due to expired SSO tokens.
Output schema not documented. Tool returns JSON text via text content type without a formal schema definition. LLM cannot predict response structure for downstream planning.
Error responses lack actionable recovery guidance. When profile resolution fails, error messages state the problem but do not guide the LLM on next steps (e.g., 'Try calling X with Y' or 'Retry with parameter Z').
No parameter indicates what the tool returns on success. Description states 'open a browser window' but does not document the actual JSON response structure (success bool, profile string, profileSource enum, message string).
Tool requires both 'profile' and 'server' parameters to be optional, but at least one must be provided. This dependency is documented in the tool description but not enforced as a schema constraint. LLMs may omit both and trigger an error.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 60 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 57 | - | v1 |