Compliant infrastructure using infrastructure as code

When you are using compute you have a lot of options. One of these options is Amazon EC2. In a world where more and more workloads become serverless. You might still have this use-case that is better off on EC2. But, how do you combine EC2 with compliance and security? In this blog post we will explore how we can build a compliant and secure EC2 stack.

Compliance in AWS

When we talk about compliance we are actually saying AWS Config. AWS Config is the service that enables you to confirm if the infrastructure is compliant. I wrote a blog in the past on how you can write your own rules. But out of the box AWS has the AWS Foundational Security Best Practices v1.0.0. This standard comes with a few config rules for EC2:

• ec2-instance-multiple-eni-check, this control checks if Amazon EC2 instance uses multiple ENI/EFA. This control will pass if single network adapters is used.
• ec2-paravirtual-instance-check, this control checks if the EC2 virtualization type is paravirtual. The control fails for an EC2 instance if virtualizationType is set to paravirtual.
• ec2-instance-no-public-ip, this control checks whether EC2 instances have a public IP address. The control fails if the publicIp field is present in the instance configuration. This control applies to IPv4 addresses only.
• ec2-imdsv2-check, this control checks if your EC2 instances have IMDSv2 configured. The control passes if HttpTokens is set to required for IMDSv2. The control fails if HttpTokens is set to optional.
• ec2-instance-managed-by-ssm, this control checks if the EC2 instances are managed by SSM

The first 3 are easy, with a proper VPC design and using sane defaults you will be compliant. But for the last 2 you need to perform some extra actions.

Security in AWS

Security is a broad topic and can include a lot of different things. In this blog I will only focus on the EC2 instance itself. This means that patch management and hardening is in scope. The service that AWS has for this is called AWS Inspector. This tool will scan the EC2 instance for known vulnerabilities. Inspector will report the findings through the AWS Console. It also integrates with Security Hub. Here you can manage all your security related findings from a single place.

Inspector can also scan your instance against the CIS hardening baseline.

Why should I use infrastructure as code?

When you use IaC (infrastructure as code), you create repeatable and predictable infrastructure. Now why is this important? Assume you create an EC2 instance, and you will use the console to launch it. Now AWS Config will start reporting that IMDSv2 is not enforced. And that the instance is not managed by SSM (AWS Systems Manager). This is because you forgot all those extra steps. Humans are not designed to think about all the options that we need to include. IaC is designed to do this for you, you define a template. You deploy the template and the end result is what is defined in the template.

If you design your templates to be compliant and secure. The infrastructure that you deploy will be compliant and secure.

Let’s build a compliant and secure template!

Now my IaC technology of choice is AWS CloudFormation. So the examples that I will present are built in CloudFormation. But, this can also be achieved in any other IaC technology that you want to use.

Setting up an instance profile

We want the EC2 instance to be managed by SSM. For this we need to create an IAM Role and an Instance Profile. We will attach 2 managed policies:

• AmazonSSMManagedInstanceCore, enables the instance to be managed by SSM.
• AmazonInspector2ManagedCispolicy, enables the ability to perform the CIS baseline scan.

 Role:
   Type: AWS::IAM::Role
   Properties:
     AssumeRolePolicyDocument:
       Version: 2012-10-17
       Statement:
         - Effect: Allow
           Action: sts:AssumeRole
           Principal:
             Service: ec2.amazonaws.com
     ManagedPolicyArns:
       - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
       - arn:aws:iam::aws:policy/AmazonInspector2ManagedCispolicy


 InstanceProfile:
   Type: AWS::IAM::InstanceProfile
   Properties:
     Roles:
       - !Ref Role

Setting up the EC2 Instance

When you launch an EC2 instance we will be needing a security group. We also want our EC2 instance to be managed by SSM. To SSM Agent comes pre-installed in Amazon Linux 2. Needs to be able to connect outbound via HTTPS, for this reason we add 0.0.0.0/0 on port 443 on the egress traffic.

We also want to make IMDSv2 required. We can configure this in the launch template MetadataOptions. To ensure that all security updates are installed during the first boot. We include the yum update -y --security option in the user-data.

You can now use the launch template to create an EC2 instance. Or, use an auto scaling group to launch one or more EC2 instances.

 InstanceSecurityGroup:
   Type: AWS::EC2::SecurityGroup
   Properties:
     GroupDescription: Security group for the test instance
     VpcId: "{{resolve:ssm:/landingzone/vpc/vpc-id}}"
     SecurityGroupEgress:
       - Description: Allow outbound connectivity to port 443.
         IpProtocol: tcp
         FromPort: 443
         ToPort: 443
         CidrIp: 0.0.0.0/0

 LaunchTemplate:
   Type: AWS::EC2::LaunchTemplate
   Properties:
     LaunchTemplateData:
       IamInstanceProfile:
         Arn: !GetAtt InstanceProfile.Arn
       ImageId: "{{resolve:ssm:/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2}}"
       InstanceType: t3.micro
       SecurityGroupIds:
         - !Ref InstanceSecurityGroup
       MetadataOptions:
         HttpTokens: required
       UserData:
         Fn::Base64: !Sub |-
           #!/bin/bash -x
           yum update -y --security

 Instance:
   Type: AWS::EC2::Instance
   Properties:
     LaunchTemplate:
       LaunchTemplateId: !GetAtt LaunchTemplate.LaunchTemplateId
       Version: !GetAtt LaunchTemplate.LatestVersionNumber
     SubnetId: "{{resolve:ssm:/landingzone/vpc/private-subnet-1-id}}"

Spinning up the instance

If you combine the two snippets you can spin-up a compliant EC2 instance. This instance has all the latest security patches installed. But when you look into Inspector you will notice that there are still some findings reported.

The findings that popped up during the time of writing were kernel related. So for me to fix this I need to update the kernel. When you lookup your EC2 instance in the console you will notice the Connect button. This will create a shell session to your instance:

Updating the kernel

First, let’s see what kernel we have running:

s​​h-4.2$ uname -r
4.14.336-253.554.amzn2.x86_64
sh-4.2$

Now let’s see what kernels are available:

sh-4.2$ amazon-linux-extras | grep kernel
49  kernel-5.4               available    [ =stable ]
55  kernel-5.10              available    [ =stable ]
62  kernel-5.15              available    [ =stable ]
sh-4.2$

We will install the latest version of the kernel 5.15 with the following command:

sh-4.2$ sudo amazon-linux-extras install kernel-5.15

Because a kernel update requires a reboot we need to reboot the instance:

sudo reboot

All done!

Avoiding the kernel update

The problem with the kernel update is the required reboot. You only have two options to remove this step:

1. Create your own up to date base AMI.
2. Use the Amazon Linux 2023 AMI with a newer kernel version.

Maintaining a base AMI comes with a lot of extra overhead. So my recommendation is to always use the newer version that uses the newer kernel versions.

But when you need to harden your images using the CIS benchmarks. You might need to start maintaining your own images anyway. There are CIS Hardened images available through the marketplace for an extra charge.

Conclusion

IaC (Infrastructure as Code) can help you spin up compliant and secure infrastructure. It removes the human aspect of forgetting to configure certain options. Using IaC you will have a predictable outcome every time you spin up the same template. This makes it reliable and is less time consuming. As a bonus you are not triggering any config rules to be marked as non compliant.

Photo by Hitesh Choudhary