Securing Your Data Lake Using S3 Access Points

IAM policies, Access Control Lists, bucket policies, KMS policies, and just when you thought S3 security couldn’t get any harder AWS introduces a new way to manage access control for your buckets called “access points”.  

What’s their use case?

Released at re:Invent 2019, access points are the newest way of managing access to multi-tenant S3 buckets at scale and make it easier to implement fine-grained access control for each application accessing the S3 buckets. 

Before access points, S3 bucket policies needed to be updated with each change in the scope of the permissions of a particular application. These made bucket policies big and cumbersome to manage. 

Access points give you the ability to implement policies specific to your application without having to update your bucket policy with each change. They can also be restricted to only allow traffic from a specific VPC which helps protect your objects from accidental exposure. 

And in contrast to S3’s global namespace, access points require their name to be unique only to your account and region. This makes it easier to maintain consistent names across your environment without having to change your application’s code. 

How do we use S3 Access Points?

Let’s explore how access points work and any gotchas with an example.

In this example, we’ll create an S3 bucket and roles with different access requirements accessing the bucket through access points. Here’s what the architecture looks like:

As the picture shows, we will have 3 AWS IAM roles accessing the yummy-food S3 bucket. The first will be a “chef” role who will only have access to create objects in the bucket. The keto-dieter role will have access to any objects prefixed with “keto/”. The omnivore role will have access to get any objects in the bucket. The 3 roles will be limited to only be able to access the S3 bucket through their corresponding access point. 

At the time of this writing terraform doesn’t support S3 access points, so we will use CloudFormation instead. If you want to view the full template it’s available in this repository. 

Let’s walk through the important parts of the template. 

The bucket policy

When using access points, the S3 bucket policy must allow at least the same level of access you intend to grant through the access point. 

YummyFoodS3BucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties: 
      Bucket: !Ref YummyFoodS3Bucket
      PolicyDocument: 
        Version: 2012-10-17
        Statement:
        - Effect: Allow
          Action: 's3:*'
          Principal: ‘*’
          Resource: !Sub 'arn:aws:s3:::${YummyFoodS3Bucket}/*'
          Condition: 
            StringEquals: 
              s3:DataAccessPointAccount: !Sub '${AWS::AccountId}'

As you can see, we’re granting any principal access to perform any action on the bucket (s3:*) as long as the request comes through an S3 access point in the same AWS account we’re provisioning the CloudFormation template. Since Access Points are bucket specific, we’re keeping this policy broad in order to avoid touching it with each new principal (e.g. IAM role) we want to grant access or change in permissions.

The IAM Roles

Something that might surprise you is that we don’t need to attach any IAM policies to the roles themselves. As long as the access is not being explicitly denied, AWS will grant access to the objects in S3 based on the allow statements in the bucket policy and S3 access point. 

Here’s what the CloudFormation template to create the roles looks like:

ChefRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              AWS: !Sub 'arn:aws:iam::${AWS::AccountId}:user/${IamUser}'
            Action:
              - 'sts:AssumeRole'
      Description: IAM role for the Chef

  OmnivoreRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              AWS: !Sub 'arn:aws:iam::${AWS::AccountId}:user/${IamUser}'
            Action:
              - 'sts:AssumeRole'
      Description: IAM role for the omnivore

  KetoDieterRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              AWS: !Sub 'arn:aws:iam::${AWS::AccountId}:user/${IamUser}'
            Action:
              - 'sts:AssumeRole'
      Description: IAM role for the Keto Dieter

As you can see, we’re not attaching any policies and have a trust policy where we’re using an IAM user to assume these roles. I plan on testing accessing the S3 buckets from inside an EC2 instance in a VPC that has an S3 VPC endpoint and also from my laptop.

The Access Points

The access points will have IAM policies that allow the minimum access needed for each of the roles. We’ll create an access point per role.

The chef-access-point will allow s3:PutObject on the bucket.

ChefAccessPoint:
    Type: 'AWS::S3::AccessPoint'
    Properties: 
      Bucket: !Ref YummyFoodS3Bucket
      Name: !Ref ChefAccessPointName
      Policy: 
        Version: 2012-10-17
        Statement:
        - Effect: Allow
          Action: 's3:PutObject'
          Principal:
            AWS: !GetAtt ChefRole.Arn
          Resource: !Sub 'arn:aws:s3:${AWS::Region}:${AWS::AccountId}:accesspoint/${ChefAccessPointName}/object/*'
      PublicAccessBlockConfiguration: 
        BlockPublicAcls: True
        BlockPublicPolicy: True
        IgnorePublicAcls: True
        RestrictPublicBuckets: True
      VpcConfiguration:
          VpcId: !Ref VpcId

The omnivore-access-point allows the omnivore role the s3:GetObject action on any objects through the access point.

OmnivoreAccessPoint:
    Type: 'AWS::S3::AccessPoint'
    Properties: 
      Bucket: !Ref YummyFoodS3Bucket
      Name: !Ref OmnivoreAccessPointName
      Policy: 
        Version: 2012-10-17
        Statement:
        - Effect: Allow
          Action: 's3:GetObject'
          Principal:
            AWS: !GetAtt OmnivoreRole.Arn
          Resource: !Sub 'arn:aws:s3:${AWS::Region}:${AWS::AccountId}:accesspoint/${OmnivoreAccessPointName}/object/*'
      PublicAccessBlockConfiguration: 
        BlockPublicAcls: True
        BlockPublicPolicy: True
        IgnorePublicAcls: True
        RestrictPublicBuckets: True
      VpcConfiguration:
          VpcId: !Ref VpcId

Finally, the keto-dieter-access-point allows the keto dieter access to s3:GetObject, but only objects prefixed “keto/*”. 

KetoDieterAccessPoint:
    Type: 'AWS::S3::AccessPoint'
    Properties: 
      Bucket: !Ref YummyFoodS3Bucket
      Name: !Ref KetoDieterAccessPointName
      Policy: 
        Version: 2012-10-17
        Statement:
        - Effect: Allow
          Action: 's3:GetObject'
          Principal:
            AWS: !GetAtt KetoDieterRole.Arn
          Resource: !Sub 'arn:aws:s3:${AWS::Region}:${AWS::AccountId}:accesspoint/${KetoDieterAccessPointName}/object/keto/*'
      PublicAccessBlockConfiguration: 
        BlockPublicAcls: True
        BlockPublicPolicy: True
        IgnorePublicAcls: True
        RestrictPublicBuckets: True
      VpcConfiguration:
          VpcId: !Ref VpcId

Testing the stack

Let’s test if everything is working as expected. We can start by creating a CloudFormation stack based on our template. You can use the command below on the AWS CLI changing any parameters as appropriate for your environment. 

$ aws cloudformation create-stack  \
  --stack-name s3-access-points-experimentation \
  --template-body file:///home/ec2-user/environment/s3-access-points/cftemplate.yaml \
  --capabilities CAPABILITY_NAMED_IAM \
  --on-failure DO_NOTHING \
  --parameters \
    ParameterKey=BucketName,ParameterValue=yummy-food \
    ParameterKey=IamUser,ParameterValue=myuser \
    ParameterKey=VpcId,ParameterValue=vpc-12345678

You can view the stack’s status by describing it. The creation will be finished when "StackStatus" shows "CREATE_COMPLETE".

$ aws cloudformation describe-stacks --stack-name s3-access-points-experimentation

Once the creation has completed, let’s capture the outputs as variables so we can use them in subsequent steps.

$ CF_OUTPUTS=$(aws cloudformation describe-stacks \
  --stack-name s3-access-points-experimentation \
  --query "Stacks[0].Outputs")

$ CHEF_ROLE_ARN=$(echo $ | \
  jq -r '.[] | select(.OutputKey=="ChefRoleArn").OutputValue')

$ CHEF_ACCESS_POINT_ARN=$(echo $ | \
  jq -r '.[] | select(.OutputKey=="ChefAccessPointArn").OutputValue')

$ KETO_DIETER_ROLE_ARN=$(echo $ | \
  jq -r '.[] | select(.OutputKey=="KetoDieterRoleArn").OutputValue')

$ KETO_DIETER_ACCESS_POINT_ARN=$(echo $ | \
  jq -r '.[] | select(.OutputKey=="KetoDieterAccessPointArn").OutputValue')

$ OMNIVORE_ROLE_ARN=$(echo $ | \
  jq -r '.[] | select(.OutputKey=="OmnivoreRoleArn").OutputValue')

$ OMNIVORE_ACCESS_POINT_ARN=$(echo $ | \
  jq -r '.[] | select(.OutputKey=="OmnivoreAccessPointArn").OutputValue')

Testing the Chef Role

Now let’s assume the chef role to start testing: 

$ ASSUME_ROLE_OUTPUT=$(aws sts assume-role \
  --role-arn $ \
  --role-session-name chef)     

$ export AWS_ACCESS_KEY_ID=$(echo $ASSUME_ROLE_OUTPUT \
  | jq -r '.Credentials.AccessKeyId')
$ export AWS_SECRET_ACCESS_KEY=$(echo $ASSUME_ROLE_OUTPUT \
  | jq -r '.Credentials.SecretAccessKey')
$export AWS_SESSION_TOKEN=$(echo $ASSUME_ROLE_OUTPUT \
  | jq -r '.Credentials.SessionToken')

To confirm your shell is currently configured to use the chef role you can use the following command: 

$ aws sts get-caller-identity

{
    "Account": "1234567890", 
    "UserId": "ASDFGHJKLZXCVBNMQWERT:chef", 
    "Arn": "arn:aws:sts::1234567890:assumed-role/s3-access-points-experimentation-ChefRole-ASDFASDF/chef"
}

Now, let’s put a few objects into the bucket. This is the functionality you’ve allowed on this role and you shouldn’t get any permission errors.

$ mkdir keto && \
  echo "steak" > keto/steak.txt && \
  echo "bacon" > keto/bacon.txt
$ mkdir standard && \
  echo "french fries" > standard/fries.txt && \
  echo "potato salad" > standard/potatosalad.txt

$ aws s3api put-object --bucket $ --key keto/steak.txt \
  --body keto/steak.txt
$ aws s3api put-object --bucket $ --key keto/bacon.txt \
  --body keto/bacon.txt
$ aws s3api put-object --bucket $ --key standard/fries.txt \
  --body standard/fries.txt
$ aws s3api put-object --bucket $ \ 
  --key standard/potatosalad.txt --body standard/potatosalad.txt

If you try retrieving an object, you will get an error message.

$ aws s3api get-object --key standard/potatosalad.txt \
  --bucket $ standard/potatosalad.txt

An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

Testing the Omnivore Role

The first step to start testing the omnivore role is to reconfigure your shell to use it. Let’s unset the environment variables and assume the role.

$ unset AWS_ACCESS_KEY_ID
$ unset AWS_SECRET_ACCESS_KEY
$ unset AWS_SESSION_TOKEN

$ ASSUME_ROLE_OUTPUT=$(aws sts assume-role \
  --role-arn $ \
  --role-session-name omnivore)     

$ export AWS_ACCESS_KEY_ID=$(echo $ASSUME_ROLE_OUTPUT \
  | jq -r '.Credentials.AccessKeyId')
$ export AWS_SECRET_ACCESS_KEY=$(echo $ASSUME_ROLE_OUTPUT \
  | jq -r '.Credentials.SecretAccessKey')
$ export AWS_SESSION_TOKEN=$(echo $ASSUME_ROLE_OUTPUT \
  | jq -r '.Credentials.SessionToken')

$ aws sts get-caller-identity

{
    "Account": "1234567890", 
    "UserId": "ASDFGHJKLZXCVBNMQWERT:omnivore", 
    "Arn": "arn:aws:sts::1234567890:assumed-role/s3-access-points-experimentation-OmnivoreRole-ASDFASDF/omnivore"
}

Now let’s try getting objects saved into both the “keto/*” and “standard’*” directory. As configured in the CloudFormation template, the omnivore role should have access to retrieve any objects through its access point. 

$ aws s3api get-object --key standard/potatosalad.txt \
  --bucket $ standard/potatosalad.txt

{
    "AcceptRanges": "bytes", 
    "ContentType": "binary/octet-stream", 
    "LastModified": "Sat, 01 Feb 2020 03:38:16 GMT", 
    "ContentLength": 13, 
    "ETag": "\"e946cedb3a9380609c6152a8d1b54c36\"", 
    "ServerSideEncryption": "AES256", 
    "Metadata": 
}

$ aws s3api get-object --key keto/bacon.txt \
  --bucket $ keto/bacon.txt

{
    "AcceptRanges": "bytes", 
    "ContentType": "binary/octet-stream", 
    "LastModified": "Sat, 01 Feb 2020 03:38:15 GMT", 
    "ContentLength": 6, 
    "ETag": "\"009e04f96048c9b8ed9e0007127da602\"", 
    "ServerSideEncryption": "AES256", 
    "Metadata": 
}

If you try to put an object through the omnivore access point, or try to get an object from a different access point you will get an access error.

$ aws s3api put-object --bucket $ \
  --key standard/potatosalad.txt --body standard/potatosalad.txt

An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

$ aws s3api get-object --key standard/potatosalad.txt \
  --bucket $ standard/potatosalad.txt

An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

Testing the Keto Dieter Role

Let’s unset the environment variables and assume the keto dieter role.

$ unset AWS_ACCESS_KEY_ID
$ unset AWS_SECRET_ACCESS_KEY
$ unset AWS_SESSION_TOKEN

$ ASSUME_ROLE_OUTPUT=$(aws sts assume-role \
  --role-arn $ \
  --role-session-name keto-dieter)     

$ export AWS_ACCESS_KEY_ID=$(echo $ASSUME_ROLE_OUTPUT \
  | jq -r '.Credentials.AccessKeyId')
$ export AWS_SECRET_ACCESS_KEY=$(echo $ASSUME_ROLE_OUTPUT \
  | jq -r '.Credentials.SecretAccessKey')
$ export AWS_SESSION_TOKEN=$(echo $ASSUME_ROLE_OUTPUT \
  | jq -r '.Credentials.SessionToken')

$ aws sts get-caller-identity

{
    "Account": "1234567890", 
    "UserId": "ASDFGHJKLZXCVBNMQWERT:keto-dieter", 
    "Arn": "arn:aws:sts::1234567890:assumed-role/s3-access-points-experimentation-KetoDieterRole-ASDFASDF/keto-dieter"
}

As configured in the CloudFormation template, you should be able to get objects from “keto/*” without permission issues. 

$ aws s3api get-object --key keto/bacon.txt \
  --bucket $ keto/bacon.txt

{
    "AcceptRanges": "bytes", 
    "ContentType": "binary/octet-stream", 
    "LastModified": "Sat, 01 Feb 2020 03:38:15 GMT", 
    "ContentLength": 6, 
    "ETag": "\"009e04f96048c9b8ed9e0007127da602\"", 
    "ServerSideEncryption": "AES256", 
    "Metadata": 
}

If you try to get an object prefixed with “standard/*”, you would get a permissions error.

$ aws s3api get-object --key standard/potatosalad.txt \
  --bucket $ standard/potatosalad.txt

An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

Conclusion & Gotchas

Access Points allow for managing S3 bucket permissions at an application level. They facilitate managing access control of S3 buckets shared across multiple applications and can help ensure these are only being accessed through a VPC endpoint. 

At the same time this additional access control layer makes it harder to audit your permissions as now you have to consider Organizations’ SCPs, IAM policies, S3 bucket policies, ACLs, and Access Point policies when determining if a particular principal has access to objects in your S3 buckets. If you want to learn more about S3 security I suggest watching the Deep dive on Amazon S3 security and management (STG301-R3) session from re:Invent 2019.