In this post we would be learning how we can deploy to AWS EC2 with GitHub pipeline, secrets (for sensitive credentials) and docker compose.

Prerequisite :

• Knowledge of AWS

• Linux knowledge

• Basic github action knowledge

• AWS account (and a running ec2 instance)

• A repo containing your basic app (we will be using a nest-js app in this lesson)

• Basic knowledge of docker and Yaml (You can get links in my article on How to setup docker compose for development environment)

First step is to GitHub secrets to store our important credentials. Which will be mongodb-url . You can use any secret your application depends on .

On GitHub , navigate to you repository then click settings > secrets and variables > actions > New repository secrets .

I would be storing my mongo uri as MONGO_URI on GitHub secret.

Next step is to create an Elastic container registry on AWS.

To do this follow the steps below

• Navigate to your amazon web console

• type ECR in the search box

• Click elastic container registry

• click create repository

• select private

• enter your repository name (copy this as I will refer to it as yourecrname in the instructions below. Substitute it accordingly)

• Click create repository

Now store the registry number as ECR_REGISTRY in github secrets

add AWS_SECRET_ACCESS_KEY as your IAM secret access key on AWS,

AWS_ACCESS_KEY_ID as your IAM access key id on AWS,

EC2_HOST as your EC2 ipv4 address on AWS,

EC2_USERNAME as ubuntu (if you used an Ubuntu AMI for your EC2 else check the appropriate EC2 username and input it) on AWS . Preferably use ubuntu AMI as the instruction below is written for it.

EC2_PRIVATE_KEY as your pem key ( the one you registered with the EC2 . Copy it and paste it in the secret box) on AWS.

AWS_REGION as your appropriate region.

we would be creating our docker-compose file which we would be using for deployment in our pipeline.

In the root of your project create a file name docker-compose.yourecrname.prod.yml

Paste the following code in your

services:
    api:
        image: ${ECR_REGISTRY}/yourecrname:1.0.0
        command: npm run start:prod
        environment:
            - MONGO_URI=${MONGO_URI}
        ports:
            - 8000:8000

Now let's explain what is going on here. The services block allows us to specify different services as we can use docker-compose to manage a multi-container application. In this case we are only running one application which is api.

The image tag is used to specify the image our container will be using. The line ${ECR_REGISTRY}/yourecrname:1.0.0 is referencing your ecr which we have stored in secrets and specifically your repository with the tag 1.0.0 . Once we start writing the pipeline you would see how this adds up.

The command tag is used to replace the default one define in the Dockerfile used to build the image . We are using nest js application so we are running npm run start:prod to spin up the application in production mode.

Then we are setting environmental variables for the container from the env variable that will be set after we ssh into the EC2 instance.

Then we are using ports to expose the port 8000 in the container to the port 8000 outside the container.

With that said , the next step is to create out github action pipeline to automate deployment to ec2. Carefully follow the steps below :

• On your github repository click actions > Skip this and set up a workflow yourself

• Then paste the content of the yaml file below into the input box provided.

• The comment on each step explains the step.

• Try to get an idea of what each step does for understanding sake

name: Deploy to AWS Ec2 #Name of action
    #specifying action trigger
    on:
       push:
        branches:
            - main

    jobs: # Set of Operation that will run a github virtual environment
        Build_and_push_image: # specifying one job
            runs-on: ubuntu-latest # specifying the OS the job will run on
            steps: # list or operations that will run
                - name: Checkout repository # Name of first operation
                  uses: actions/checkout@v2 # using an existing github action to checkout into the main repository (like git checkout branchname)

                - name: Install Docker #Operation to install docker listed
                  run: | # Run helps to run commands in the vitual environment (In this case we are running steps to install docker so we can build and push docker images to our ECR)
                    sudo apt-get update
                    sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-common
                    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/docker.gpg
                    echo "deb [arch=amd64 signed-by=/etc/apt/trusted.gpg.d/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
                    sudo apt-get update
                    sudo apt-get -y install docker-ce docker-ce-cli containerd.io    

                - name: Configure AWS credentials # To push images to ECR we need to configure AWS credential so we can login
                  uses: aws-actions/configure-aws-credentials@v1 # uses an existing github action  and provides credentials from our secrets
                  with:
                    aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
                    aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
                    aws-region: ${{ secrets.AWS_REGION }}

                - name: Login to AWS ECR # This step logs into ECR with an existing github action after setting configurations
                  id: login-ecr
                  uses: aws-actions/amazon-ecr-login@v1

                - name: Build and push Docker image # This step pushes to ecr by using the docker command under run
                  env:
                    ECR_REGISTRY: ${{ secrets.ECR_REGISTRY }}
                    IMAGE_TAG: 1.0.0
                  run: |
                    docker build -t $ECR_REGISTRY/yourecrname:$IMAGE_TAG .
                    docker push $ECR_REGISTRY/yourecrname:$IMAGE_TAG

                - name: copy file via ssh key #We would need our docker compose file in the ec2 instance so we are copying it there ahead of time using an existing scp github action
                  uses: appleboy/scp-action@v0.1.7 # the githhub action we are using
                  with:  #passing the required credentials from our secret
                    host: ${{ secrets.EC2_HOST }}
                    username: ${{ secrets.EC2_USERNAME }}
                    key: ${{ secrets.EC2_PRIVATE_KEY }}
                    port: 22
                    source: "docker-compose.yourecrname.prod.yml"
                    target: "/home/ubuntu"
        pull_and_deploy_in_ec2:  # After pushing to ecr we need to pull from our ec2 and spin up the container. so we define a seperate job for it
            runs-on: ubuntu-latest # specifying OS for job to run in
            needs: Build_and_push_image
            steps:
            - name: Deploy to EC2
              uses: appleboy/ssh-action@master
              with:
                host: ${{ secrets.EC2_HOST }}
                username: ${{ secrets.EC2_USERNAME }}
                key: ${{ secrets.EC2_PRIVATE_KEY }}
                script: |

                # Installing aws cli

                sudo apt-get update

                sudo apt-get install -y awscli

                # Configure AWS credentials

                aws configure set aws_access_key_id ${{ secrets.AWS_ACCESS_KEY_ID }}

                aws configure set aws_secret_access_key ${{ secrets.AWS_SECRET_ACCESS_KEY }}

                aws configure set default.region ${{ secrets.EC2_HOST }}

                # Install Docker inside the remote EC2 instance

                sudo apt-get update

                sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-common

                curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/docker.gpg

                echo "deb [arch=amd64 signed-by=/etc/apt/trusted.gpg.d/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

                sudo apt-get update

                sudo apt-get -y install docker-ce docker-ce-cli containerd.io

                # Authenticate to ECR

                aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin ${{ secrets.ECR_REGISTRY }}

                # Add environmental variable
                export ECR_REGISTRY=${{ secrets.ECR_REGISTRY }}
                export MONGO_URI="${{ secrets.MONGO_URI }}" # Added quote incase of characters that have other function, this could disrupt the process if not added

                # Add SSH user to the docker group
                sudo usermod -aG docker $USER

                # Install Docker Compose inside the remote EC2 instance
                sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
                sudo chmod +x /usr/local/bin/docker-compose

                # Clear existing container and images with added project name, This will prevent the command from clearing containers of other docker compose file
                docker-compose -f ~/docker-compose.yourecrname.prod.yml -p yourecrname down --rmi all

                # Run the Docker commands for deployment with added project name , future down command will clear containers and images with this project name
                docker-compose -f ~/docker-compose.yourecrname.prod.yml -p yourecrname up -d

Afterwards merge the changes to your main branch . Click actions to check the progress. Once deployed you can ssh into your EC2 via your terminal . Then run docker ps to view your running container . You can also run ls to view your docker-compose.yourecrname.prod.yml file.

I hope this helps . Thanks