Loading ToolBench
A comprehensive AWS security inspection and remediation tool providing tools for security analysis across IAM Access Analyzer, CloudFront, EC2, ECR, Athena, and credentials management
AWS Security MCP demonstrates solid foundational structure with 21 tools across access analyzer, Athena, CloudFront, EC2, and ECR domains. All tools have descriptions (10-200+ chars) and properly typed input schemas with parameter descriptions. Strengths: consistent verb-noun naming (list_, get_, search_), comprehensive parameter documentation with types, clear cross-account session context support throughout. Weaknesses: output schemas are not documented (only input schemas visible), error handling lacks actionable recovery guidance, no tool annotations (readOnlyHint/destructiveHint), return structures inferred rather than explicitly defined in source. Pagination support is present but inconsistent (some tools have limit/next_token, others have max_items/next_token). Per-tool scores range 55-75; median ~65.
Get list of connected AWS accounts with valid sessions. This tool returns information about AWS accounts that the MCP server currently has valid credential sessions for, including session keys that can be used in other operations.
Count EC2 instances, optionally filtering by state and security group rules.
Execute an Athena SQL query asynchronously and return execution ID immediately. This follows the proper MCP pattern - the query is submitted and you get back a query_execution_id. Use get_athena_query_status and get_athena_query_results to check progress and retrieve results.
Get detailed information about a specific Access Analyzer.
Get detailed schema information for a specific Athena table.
Output schemas not documented. Only input schemas are visible in source code. LLMs cannot plan downstream tool calls without knowing what fields to expect from responses (e.g., what fields does list_analyzers return? Does get_analyzer return arn, name, type, createdAt?). This violates pattern:tool-description and pattern:response-shaper.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 65 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 58 | - | v1 |
Get detailed information about a specific CloudFront distribution.
Get vulnerability scan findings for a container image. This tool retrieves scan findings for the specified container image.
Get information about all images in an ECR repository. This tool retrieves details about all container images in the specified repository.
Get the IAM policy for an ECR repository. This tool retrieves the repository policy for the specified ECR repository.
Get detailed information about a specific Access Analyzer finding.
List all IAM Access Analyzers in the account.
List all available data catalogs in AWS Athena. This is essential for discovering CloudTrail and other security datasets that might be in different catalogs (not just AwsDataCatalog).
List all databases in the specified AWS Athena data catalog.
List all available cross-account sessions with their keys and metadata. This tool provides detailed information about all available sessions that can be used with other MCP tools by specifying the session_key.
List CloudFront distributions in the AWS account.
List EC2 instances with details.
List all ECR repositories in the AWS account.
List findings from a specific IAM Access Analyzer.
Get findings filtered by resource type category.
Refresh AWS cross-account sessions for multi-account access. This tool automatically discovers organization accounts and establishes fresh credential sessions for cross-account security monitoring.
Search for ECR repositories and get detailed information. This tool allows searching for repositories by exact name match and returns detailed information about the matched repositories, including policy information and recent images.
No tool annotations present. Tools lack readOnlyHint, destructiveHint, or idempotentHint metadata. Per MCP spec (2026-07-28), every tool should declare its risk profile via annotations. Current spec requires this for proper agent safety reasoning and audit trails. refresh_aws_session is marked WRITE but has no annotation in the tool definition.
Inconsistent pagination pattern. Some tools (list_findings, list_ec2_instances, list_athena_catalogs) use max_items + next_token, others use limit + next_token. This cognitive load forces LLMs to reason about parameter naming variance. Baseline from production tools shows canonical approach is (limit, offset/next_token). None return a total_count or total_available, which aids planning.
Error handling lacks actionable recovery guidance. Source code shows generic error JSON {status: error, message: str(e)}, e.g., in access_analyzer_tools.py lines 40-47 and 88-96. When an Access Analyzer ARN is invalid, LLMs receive only the exception message, not guidance like 'Use list_analyzers() to find valid ARNs' or categorization of whether the error is retryable. Violates pattern:recovery-guide.
execute_athena_query description states 'returns execution ID immediately' but implementation is not visible in source. If true, tool correctly implements async pattern (MCP Multi Round-Trip Requests / polling), but follow-up tools get_athena_query_status and get_athena_query_results are not listed in the 21-tool inventory. This risks agent confusion about incomplete tool chains.
Parameter constraints not enforced in descriptions. E.g., status param in list_findings is described as 'Optional filter for finding status (ACTIVE, ARCHIVED, RESOLVED)' but no explicit enum constraint or validation hint. execute_athena_query accepts output_location and workgroup with no format guidance. Baseline production tools specify constraints inline: '(one of: ACTIVE, ARCHIVED, RESOLVED)' or '(S3 path format: s3://bucket/prefix)'. Violates pattern:constrained-input.