How to keep AMI up-to-date by using Packer, AWS Codebuild, Terraform

How to keep AMI up-to-date by using Packer, AWS Codebuild, Terraform

Hello World!

This time I want to share one way to create a golden AWS AMI and keeping it up-to-date. The goal is to have an automated way for the AMI to get the latest update.

I will be using Windows AMI, Packer, AWS CodeBuild, and Terraform.

1. Packer

Packer is a free and open-source tool for creating golden images for multiple platforms from a single source configuration.

With Packer, we want to create an AMI that fetches the latest Windows AMI version, does Windows Update, and baked some packages into it.

Few things that I want to bake into the AMI are:

Amazon CloudWatch Agent.

We want to have memory and disk usage metrics for our server.

Amazon CodeDeploy Agent.

We want to deploy our app using the CodeDeploy service.

Install IIS
Install dotnet-hosting-3.1.10
Windows Update

We'll use this provisioner. Please refer to its documentation for complete information.

Ok, let's build the AMI.

First, prepare variables.json for our variables containing the AWS access required. Make sure we created the iam_instance_profile we need.

{
    "region": "ap-southeast-1",
    "aws_access_key": "{{env `AWS_ACCESS_KEY_ID`}}",
    "aws_secret_key": "{{env `AWS_SECRET_ACCESS_KEY`}}",
    "iam_instance_profile": "packer",
    "winrm_username_env": "Administrator"
}

Inside we take the default AWS config of the instance, define the instance profile for the running EC2 instance, and Winrm username for Packer to perform all the actions inside the instance.

Second, create dotnet-hosting.json.

{
  "sensitive-variables": [
    "aws_access_key",
    "aws_secret_key",
    "winrm_username_env"
  ],
  "builders": [
    {
      "type": "amazon-ebs",
      "access_key": "{{ user `aws_access_key` }}",
      "secret_key": "{{ user `aws_secret_key` }}",
      "source_ami_filter": { 
        "filters": {
          "virtualization-type": "hvm",
          "name": "Windows_Server-2019-English-Full-Base-*",
          "root-device-type": "ebs"
        },
        "owners": [
          "801119661308"
        ],
        "most_recent": true
      },
      "region": "{{ user `region` }}",
      "instance_type": "t3.medium",
      "iam_instance_profile": "{{user `iam_instance_profile`}}",
      "ami_name": "dotnet-hosting-ami-packer--{{timestamp}}",
      "ebs_optimized": true,
      "launch_block_device_mappings": [
        {
          "device_name": "/dev/sda1",
          "volume_size": 100,
          "volume_type": "gp2",
          "delete_on_termination": true
        }
      ],
      "winrm_username": "{{ user `winrm_username_env`}}",
      "user_data_file": "./scripts/bootstrap_win.txt",
      "communicator": "winrm"
    }
  ],
  "provisioners": [
    {
      "type": "windows-update",
      "search_criteria": "IsInstalled=0",
      "filters": [
          "exclude:$_.Title -like '*Preview*'",
          "include:$true"
      ],
      "update_limit": 25
    },
    {
      "type": "powershell",
      "scripts": [
          "./scripts/agent.ps1",
          "./scripts/install_dotnet_hosting.ps1"
      ]
    },
    {
      "type": "windows-restart"
    },
    {
      "script": "./scripts/sysprep.ps1",
      "type": "powershell"
    }
  ]
}

I'll break down the JSON to give more explanation.

Here we query to get the latest AMI of the Windows server 2019

"source_ami_filter": {
        "filters": {
          "virtualization-type": "hvm",
          "name": "Windows_Server-2019-English-Full-Base-*",
          "root-device-type": "ebs"
        },
        "owners": [
          "801119661308"
        ],
        "most_recent": true
      },

Here we query to get the latest AMI of the Windows server 2019

"launch_block_device_mappings": [
        {
          "device_name": "/dev/sda1",
          "volume_size": 100,
          "volume_type": "gp2",
          "delete_on_termination": true
        }
      ]

Resize the root volume of the instance rather than using the size from the original AMI.

      "winrm_username": "{{ user `winrm_username_env`}}",
      "user_data_file": "./scripts/bootstrap_win.txt",
      "communicator": "winrm"

We define how the packer will communicate with the instance. Since we are using Winrm, we need to bootstrap the instance to configure winrm.

bootstrap_win.txt

<powershell>

# First, make sure WinRM can't be connected to
netsh advfirewall firewall set rule name="Windows Remote Management (HTTP-In)" new enable=yes action=block

# Delete any existing WinRM listeners
winrm delete winrm/config/listener?Address=*+Transport=HTTP  2>$Null
winrm delete winrm/config/listener?Address=*+Transport=HTTPS 2>$Null

# Disable group policies that block basic authentication and unencrypted login

Set-ItemProperty -Path HKLM:\Software\Policies\Microsoft\Windows\WinRM\Client -Name AllowBasic -Value 1
Set-ItemProperty -Path HKLM:\Software\Policies\Microsoft\Windows\WinRM\Client -Name AllowUnencryptedTraffic -Value 1
Set-ItemProperty -Path HKLM:\Software\Policies\Microsoft\Windows\WinRM\Service -Name AllowBasic -Value 1
Set-ItemProperty -Path HKLM:\Software\Policies\Microsoft\Windows\WinRM\Service -Name AllowUnencryptedTraffic -Value 1


# Create a new WinRM listener and configure
winrm create winrm/config/listener?Address=*+Transport=HTTP
winrm set winrm/config/winrs '@{MaxMemoryPerShellMB="0"}'
winrm set winrm/config '@{MaxTimeoutms="7200000"}'
winrm set winrm/config/service '@{AllowUnencrypted="true"}'
winrm set winrm/config/service '@{MaxConcurrentOperationsPerUser="12000"}'
winrm set winrm/config/service/auth '@{Basic="true"}'
winrm set winrm/config/client/auth '@{Basic="true"}'

# Configure UAC to allow privilege elevation in remote shells
$Key = 'HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System'
$Setting = 'LocalAccountTokenFilterPolicy'
Set-ItemProperty -Path $Key -Name $Setting -Value 1 -Force

# Configure and restart the WinRM Service; Enable the required firewall exception
Stop-Service -Name WinRM
Set-Service -Name WinRM -StartupType Automatic
netsh advfirewall firewall set rule name="Windows Remote Management (HTTP-In)" new action=allow localip=any remoteip=any
Start-Service -Name WinRM
</powershell>

Once the instance is reachable, we continue using the windows-update provisioner to update the Windows Server with a certain filter or none.

"provisioners": [
    {
      "type": "windows-update",
      "search_criteria": "IsInstalled=0",
      "filters": [
          "exclude:$_.Title -like '*Preview*'",
          "include:$true"
      ],
      "update_limit": 25
    },
    {
      "type": "powershell",
      "scripts": [
          "./scripts/agent.ps1",
          "./scripts/install_dotnet_hosting.ps1"
      ]
    },
    {
      "type": "windows-restart"
    },
    {
      "script": "./scripts/sysprep.ps1",
      "type": "powershell"
    }
  ]

Run additional PowerShell script to install agents, IIS, and dotnet-hosting, at the end restart the server after successful install. Finally, do a Sysprep to the system to get a fresh OS.