Loading ToolBench
Fast, modular AWS SSO authentication tool with MCP server support, credential caching, and EC2/EKS integration
Server has 8 tools with consistent naming (verb_noun pattern: aws_list_*, aws_get_*, aws_switch_*, aws_ensure_*, aws_update_*). All tools have descriptions (avg ~100 chars, within baseline 34-392). However, parameter descriptions are sparse or missing context. Input schemas are present but lack detail: most parameters are bare strings without enums, format constraints, or validation guidance. Output schemas are not documented, LLMs cannot predict response structure. Error handling returns success/error flags but lacks recovery guidance. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels (READ_ONLY vs WRITE). Missing pagination for list tools. Composition is reasonable (single responsibility per tool), but aws_list_ec2_instances and aws_list_eks_clusters lack limit/offset parameters.
Check if AWS credentials are valid, and automatically refresh using cached SSO refresh token if expired.
Get current AWS caller identity (Account ID, IAM Role/User ARN, User ID) using STS.
Get raw AWS session environment variables (AccessKey, SecretKey, SessionToken, Region) for isolated subprocess execution.
List EC2 instances in a specific AWS region using the specified profile.
List EKS clusters in a specific AWS region and optionally configure local ~/.kube/config.
List all available local AWS profiles, current active default profile, and configured regions.
No output schemas documented. LLMs cannot predict response structure (fields, types, nesting). Responses return success/error flags and data, but structure is implicit.
List tools (aws_list_profiles, aws_list_ec2_instances, aws_list_eks_clusters) lack pagination parameters (limit, offset, next_cursor). No indication of result count or truncation. Large result sets will blow context window.
Parameter descriptions lack validation constraints. 'profile_name' and 'region' are bare strings with no enum, format, or length guidance. LLMs will hallucinate invalid values (e.g. 'us-middle-1' for region).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | <=2025-11-25 | v2 |
Switch the default AWS credentials profile in ~/.aws/credentials to a specific profile name.
Configure local kubeconfig to connect to a specific EKS cluster.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. aws_switch_profile, aws_ensure_credentials, aws_list_eks_clusters (with update_kubeconfig=true), and aws_update_kubeconfig are WRITE operations but unmarked.
Error responses lack recovery guidance. 'Could not retrieve caller identity' and 'Failed to set profile' do not tell LLM what to try next (e.g. 'Check credentials with aws_ensure_credentials' or 'List available profiles with aws_list_profiles').