Loading ToolBench
A Model Context Protocol (MCP) server providing tools to interact with AWS services including S3, EC2, RDS, CloudWatch, Cost Explorer, and IAM operations.
The server provides 3 tools with varying quality. Two cost management tools (ce-get_cost_and_usage, ce-get_cost_and_usage_with_resources) have excellent, detailed descriptions with comprehensive parameter documentation and use cases. However, the generic aws_sdk_wrapper tool is problematic: it lacks proper parameter validation guidance, accepts a free-form 'operation_kwargs' dict that invites hallucinated AWS API calls, and conflates multiple concerns (any service, any operation) into a single tool. Parameter descriptions are present but inconsistent in depth. Output schemas are not documented, callers don't know the structure of responses. Error handling guidance is absent; the tools will surface raw AWS SDK errors without recovery suggestions. Naming is clear (verb_noun pattern), but the generic wrapper violates the single-responsibility principle. Overall Definition Quality is fair-to-poor due to lack of output documentation, weak error handling, and the generic tool's design issues.
A generic AWS SDK wrapper to call any AWS service and operation. Args: service_name (str): The name of the AWS service to call (e.g. 's3', 'ec2', 'rds', etc.). operation_name (str): The name of the operation to call (e.g. 'list_buckets', 'describe_instances', etc.). region_name (str): The AWS region to use. profile_name (str): The name of the AWS profile to use. operation_kwargs (dict): The arguments to pass to the operation. Returns: Any: The response from the AWS service. Example: aws_sdk_wrapper('ce', 'get_cost_and_usage_with_resources', region_name='us-east-1', profile_name='my_profile', operation_kwargs={'TimePeriod': {'Start': '2023-01-01', 'End': '2023-01-31'}, 'Granularity': 'MONTHLY', 'GroupBy': [{'Type': 'DIMENSION', 'Key': 'SERVICE'}], 'Metrics': ['BlendedCost']})
Retrieve comprehensive AWS cost and usage data with advanced filtering, grouping, and time-based analysis. This tool provides detailed cost analysis across AWS services, accounts, and custom dimensions. Essential for cost optimization, budgeting, chargeback reporting, and financial governance. Supports multiple granularities, cost metrics, and complex filtering for precise analysis. **Required Parameters:** - profile_name (str): AWS profile name from ~/.aws/credentials - region (str): AWS region (e.g., 'us-east-1') - Cost Explorer is global but requires region - start (str): Start date in YYYY-MM-DD format (e.g., '2024-01-01') - end (str): End date in YYYY-MM-DD format (e.g., '2024-01-31') * Maximum 12 months range for daily data * Maximum 12 months range for monthly data * Maximum 1 week range for hourly data **Optional Parameters:** - granularity (str): Time granularity for cost breakdown. Default: 'MONTHLY' * 'DAILY': Day-by-day cost breakdown (max 12 months) * 'MONTHLY': Month-by-month cost breakdown (max 12 months) * 'HOURLY': Hour-by-hour cost breakdown (max 1 week) - group_by (List[Dict[str, str]]): Grouping dimensions for cost analysis **Service Grouping:** [{'Type': 'DIMENSION', 'Key': 'SERVICE'}] - Group by AWS service **Account Grouping:** [{'Type': 'DIMENSION', 'Key': 'LINKED_ACCOUNT'}] - Group by AWS account **Geographic Grouping:** [{'Type': 'DIMENSION', 'Key': 'REGION'}] - Group by AWS region [{'Type': 'DIMENSION', 'Key': 'AVAILABILITY_ZONE'}] - Group by AZ **Resource Grouping:** [{'Type': 'DIMENSION', 'Key': 'INSTANCE_TYPE'}] - Group by EC2 instance type [{'Type': 'DIMENSION', 'Key': 'USAGE_TYPE'}] - Group by usage type [{'Type': 'DIMENSION', 'Key': 'OPERATION'}] - Group by operation **Tag-based Grouping:** [{'Type': 'TAG', 'Key': 'Environment'}] - Group by Environment tag [{'Type': 'TAG', 'Key': 'Project'}] - Group by Project tag [{'Type': 'TAG', 'Key': 'Owner'}] - Group by Owner tag **Multiple Grouping:** [{'Type': 'DIMENSION', 'Key': 'SERVICE'}, {'Type': 'TAG', 'Key': 'Environment'}] - metrics (List[str]): Cost metrics to retrieve. Default: ['BlendedCost'] * 'BlendedCost': Cost after applying Reserved Instance and Savings Plans discounts * 'UnblendedCost': On-demand cost without discounts * 'NetBlendedCost': BlendedCost minus credits and refunds * 'NetUnblendedCost': UnblendedCost minus credits and refunds * 'UsageQuantity': Usage amount (hours, GB, requests, etc.) * 'NormalizedUsageAmount': Usage normalized to equivalent units - filter_expression (Dict[str, Any]): Advanced filtering for targeted analysis **Service Filters:** {'Dimensions': {'Key': 'SERVICE', 'Values': ['Amazon Elastic Compute Cloud - Compute']}} **Account Filters:** {'Dimensions': {'Key': 'LINKED_ACCOUNT', 'Values': ['123456789012']}} **Region Filters:** {'Dimensions': {'Key': 'REGION', 'Values': ['us-east-1', 'us-west-2']}} **Tag Filters:** {'Tags': {'Key': 'Environment', 'Values': ['production', 'staging']}} **Complex Filters (AND/OR/NOT):** {'And': [{'Dimensions': {'Key': 'SERVICE', 'Values': ['EC2']}}, {'Tags': {'Key': 'Environment', 'Values': ['production']}}]} - next_page_token (str): Pagination token from previous response * Use NextPageToken from previous call for large datasets **Common Use Cases:** 1. **Monthly service breakdown:** granularity='MONTHLY', group_by=[{'Type': 'DIMENSION', 'Key': 'SERVICE'}] 2. **Daily cost trends:** granularity='DAILY', metrics=['BlendedCost'] 3. **Environment cost analysis:** group_by=[{'Type': 'TAG', 'Key': 'Environment'}] 4. **EC2 cost optimization:** filter_expression={'Dimensions': {'Key': 'SERVICE', 'Values': ['Amazon Elastic Compute Cloud - Compute']}} 5. **Multi-account analysis:** group_by=[{'Type': 'DIMENSION', 'Key': 'LINKED_ACCOUNT'}] 6. **Regional cost comparison:** group_by=[{'Type': 'DIMENSION', 'Key': 'REGION'}] **Response includes:** Time periods, cost amounts by specified metrics, grouping dimensions, and metadata for comprehensive cost analysis and reporting. **Best Practices:** - Use monthly granularity for trend analysis - Apply service filters to focus on specific cost areas - Combine dimension and tag grouping for detailed insights - Use pagination for large datasets - Consider NetBlendedCost for accurate cost reporting
aws_sdk_wrapper accepts arbitrary operation_kwargs (dict) without validation, schema constraints, or guidance. This pattern invites the LLM to pass malformed or dangerous AWS API calls (e.g., delete operations, overly permissive policies). No enum constraints, no examples of valid kwargs structures, no validation hints.
aws_sdk_wrapper violates single-responsibility principle. It conflates multiple concerns: service selection, operation selection, credential management, and parameter passing. This forces the LLM to reason about AWS API semantics and should be split into domain-specific tools (e.g., list_s3_buckets, describe_ec2_instances) or at least narrowed to a specific service family.
No output schema documented for any tool. Callers don't know what fields to expect in responses, AWS API responses are verbose and include pagination internals, audit metadata, and raw IDs. LLMs cannot plan downstream calls or extract key data without explicit schema documentation.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 64 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 47 | - | v1 |
Retrieve detailed AWS cost and usage data with individual resource-level granularity and identification. This tool provides the most granular cost analysis available, showing costs for individual AWS resources such as specific EC2 instances, RDS databases, and S3 buckets. Essential for detailed cost allocation, rightsizing analysis, and identifying cost optimization opportunities at the resource level. **Required Parameters:** - profile_name (str): AWS profile name from ~/.aws/credentials - region (str): AWS region (e.g., 'us-east-1') - Cost Explorer is global but requires region - start (str): Start date in YYYY-MM-DD format (e.g., '2024-01-01') - end (str): End date in YYYY-MM-DD format (e.g., '2024-01-31') * **Note:** Maximum 14 days range for resource-level data * Hourly granularity not supported for resource-level analysis **Optional Parameters:** - granularity (str): Time granularity for cost breakdown. Default: 'MONTHLY' * 'DAILY': Day-by-day resource cost breakdown (max 14 days) * 'MONTHLY': Month-by-month resource cost breakdown (max 14 days) * **Note:** HOURLY not supported for resource-level data - group_by (List[Dict[str, str]]): Grouping dimensions for resource analysis **Resource Grouping:** [{'Type': 'DIMENSION', 'Key': 'RESOURCE_ID'}] - Group by individual resources **Service + Resource:** [{'Type': 'DIMENSION', 'Key': 'SERVICE'}, {'Type': 'DIMENSION', 'Key': 'RESOURCE_ID'}] **Tag-based Resource Analysis:** [{'Type': 'TAG', 'Key': 'Name'}, {'Type': 'DIMENSION', 'Key': 'RESOURCE_ID'}] [{'Type': 'TAG', 'Key': 'Environment'}, {'Type': 'DIMENSION', 'Key': 'RESOURCE_ID'}] - metrics (List[str]): Cost metrics to retrieve. Default: ['BlendedCost'] * 'BlendedCost': Cost after applying Reserved Instance and Savings Plans discounts * 'UnblendedCost': On-demand cost without discounts * 'NetBlendedCost': BlendedCost minus credits and refunds * 'NetUnblendedCost': UnblendedCost minus credits and refunds * 'UsageQuantity': Resource usage amount (hours, GB, requests, etc.) * 'NormalizedUsageAmount': Usage normalized to equivalent units - filter_expression (Dict[str, Any]): Advanced filtering for targeted resource analysis **Service Filters:** {'Dimensions': {'Key': 'SERVICE', 'Values': ['Amazon Elastic Compute Cloud - Compute']}} **Resource Type Filters:** {'Dimensions': {'Key': 'INSTANCE_TYPE', 'Values': ['t3.micro', 'm5.large']}} **Tag-based Resource Filters:** {'Tags': {'Key': 'Environment', 'Values': ['production']}} {'Tags': {'Key': 'Owner', 'Values': ['team-a', 'team-b']}} **Complex Resource Filters:** {'And': [{'Dimensions': {'Key': 'SERVICE', 'Values': ['EC2']}}, {'Tags': {'Key': 'Environment', 'Values': ['production']}}]} - next_page_token (str): Pagination token from previous response * Essential for resource-level data due to large result sets **Common Use Cases:** 1. **EC2 instance cost analysis:** filter_expression={'Dimensions': {'Key': 'SERVICE', 'Values': ['Amazon Elastic Compute Cloud - Compute']}} group_by=[{'Type': 'DIMENSION', 'Key': 'RESOURCE_ID'}] 2. **RDS database cost breakdown:** filter_expression={'Dimensions': {'Key': 'SERVICE', 'Values': ['Amazon Relational Database Service']}} group_by=[{'Type': 'DIMENSION', 'Key': 'RESOURCE_ID'}] 3. **S3 bucket cost analysis:** filter_expression={'Dimensions': {'Key': 'SERVICE', 'Values': ['Amazon Simple Storage Service']}} group_by=[{'Type': 'DIMENSION', 'Key': 'RESOURCE_ID'}] 4. **Team-based resource costs:** filter_expression={'Tags': {'Key': 'Team', 'Values': ['backend', 'frontend']}} group_by=[{'Type': 'TAG', 'Key': 'Team'}, {'Type': 'DIMENSION', 'Key': 'RESOURCE_ID'}] 5. **Environment resource breakdown:** filter_expression={'Tags': {'Key': 'Environment', 'Values': ['production']}} group_by=[{'Type': 'DIMENSION', 'Key': 'RESOURCE_ID'}] **Response includes:** Time periods, individual resource identifiers, cost amounts, resource metadata, and grouping dimensions for granular cost analysis.
Error handling is absent. Tools will surface raw AWS SDK exceptions (botocore errors, access denied, invalid parameters, timeouts) without recovery guidance. LLMs cannot determine whether to retry, ask the user, or abort. No error categorization, no actionable messages.
The ce tools accept 'group_by' parameter with type=['array','object','string'], this is overly permissive and confusing. The parameter description provides examples, but formal schema constraints (enum, minItems, required structure) are missing. LLMs may pass invalid structures.
aws_sdk_wrapper description includes a literal example with hardcoded operation_kwargs. Replace with formal constraint declarations.