Exploring AWS EKS Auto-mode | Sample app deployment (Part-2)

Exploring AWS EKS Auto-mode | Sample app deployment (Part-2)

Hein Htet Win's photo
Hein Htet Win
·

15 min read

Table of contents

Hello Everyone, welcome back to the part-2 of our exploring AWS EKS Auto Mode series. In this part, we will explore into the features and benefits EKS Auto Mode offers one by one by testing with practical examples. You can also look the full documentation by AWS to learn more about the automated components and features of EKS Auto Mode.

Upgrading the cluster

Before we dive into deploying our application into the cluster, let’s try to upgrade our cluster to Kubernetes version 1.31. It is the latest Kubernetes version EKS provided as of now (17.Jan.2025). As you can see in the screenshots below, our kubectl client has version 1.31.0 and our EKS cluster has version of v1.30.8.

Also, we have one worker node available for our workloads and the node itself has version v1.30.5 Normally, when we upgrade a Kubernetes cluster, we could have one minor version different between the worker nodes and control plane. It means if we will upgrade our cluster to 1.31.x, our worker nodes can have 1.30.x version and after our upgrade for control plane is successful, we would need to upgrade the worker nodes one by one.

Let’s see how the EKS Auto Mode upgrades our cluster and how easy is to upgrade our K8s cluster using Auto Mode. First, we will upgrade our control plane by using EKS console itself. We can either click the Upgrade Version button on the top right or just simply click Upgrade now button that the pop up is showing.

After confirming the version to upgrade, we can now check the Update history tab of EKS console and it should show something like this:

After running the update for about 10-12 minutes, EKS has successfully upgraded our control plane version to 1.31 but it didn’t upgrade our worker node. The reason why it didn’t upgrade our worker node I assume is because we only have one worker node and we can’t terminate all the running workloads on it. If we do have multiple worker nodes and appropriate Pod Disruption Budgets (PDBs) set up, it would granularly upgrade our worker nodes too.

Important thing to note here is we also don’t need to upgrade our add-ons (if we have any) while using EKS Auto Mode compared to what we have done before, upgrading our EKS add-ons ourselves to match the version constraints with our control plane.

Looking into node pools and node classes

As you may probably know if you’re familiar with EKS, to run workloads on our EKS clusters, we first need to have worker nodes in our cluster which will interact with our control plane and host the containers. Before the Auto Mode, we have a variety of choices how to initiate our worker nodes. Those include:

  • Managed Node Groups: Managed Node Groups automate the provisioning, scaling, and lifecycle management of worker nodes (EC2 instances) for your EKS cluster. We can also include managed node group configurations in eksctl config file.

  • Self-managed Node Groups: You manually provision and manage EC2 instances as worker nodes and attach them to your EKS cluster. This method is more popular before the Managed Node Groups came and dynamic provisioner like Karpenter and Auto Scaler are out of the answers.

  • Fargate Profiles: AWS Fargate is a serverless compute engine that allows you to run containers without managing the underlying infrastructure. By using Fargate profiles with EKS, you can actually create Fargate hosts only when there is incoming workloads on your cluster. Moreover, you don’t need to manage the underlying host when you choose Fargate.

  • Karpenter: Karpenter automatically launches just the right compute resources to handle your cluster's applications. It is specifically designed to dynamically provision worker nodes based on resources requests and limits you defined on your workload and it chooses available and matching node sizes and types from your definition of NodePool and NodeClasses.

  • Cluster Autoscaler: Similar to Karpenter, Cluster Autoscaler also provision worker nodes in our cluster dynamically when there is a surge of incoming workloads. But in my opinion, Cluster Autoscaler has broader use cases as it seamlessly supports not only EKS, but also in other cloud service providers such as Azure and GCP.

But with the EKS Auto Mode, we no longer need to consider how we want to configure and provision worker nodes for our cluster. EKS Auto Mode dynamically adds or removes nodes in your EKS cluster based on the demands of your Kubernetes applications. This minimizes the need for manual capacity planning and ensures application availability and moreover, we can focus on building our business logic and optimizing application rather than having to worry about infrastructure configurations.

EKS Auto Mode still uses Karpenter in the backend to make it possible to dynamically provision worker nodes on the go as we need for our workloads.

Customizable NodePools and NodeClasses: If your workload requires changes to storage, compute, or networking configurations, you can create custom NodePools and NodeClasses using EKS Auto Mode. While default NodePools and NodeClasses can’t be edited, you can add new custom NodePools or NodeClasses alongside the default configurations to meet your specific requirements.

What’s NodePool and NodeClasses?

NodePool

  • What it is: A NodePool defines how and when worker nodes should be provisioned.

  • What it does:

    • Specifies the types of nodes (e.g., instance types, sizes) to use.

    • Defines scaling behavior (e.g., minimum and maximum number of nodes).

    • Sets rules for where pods should run (e.g., labels, taints, and tolerations).

  • Example: You can create a NodePool for "high-memory workloads" that uses r5.large instances and scales between 2 and 10 nodes.

NodeClass

  • What it is: A NodeClass defines where the nodes should be provisioned (e.g., in which AWS infrastructure).

  • What it does:

    • Specifies the AWS-specific details like:

      • Subnets (where nodes are launched).

      • Security groups (network rules).

      • AMI (operating system image).

      • IAM roles (permissions for nodes).

  • Example: A NodeClass might specify that nodes should be launched in the us-east-1a subnet with a specific security group and IAM role.

How EKS Auto Mode handles persistent storage

In this section, we will explore into how EKS Auto Mode handles persistent storage for us in the cluster. We will verify this by deploying a stateful application which need persistent storage (pvs). But before we do this, one pre-requisite for us is to have a default storage class to use in the cluster if the application manifest file isn’t defined otherwise.

Creating default storage class

So, we will create a default storage class in our cluster in order to use for upcoming persistent volume claims (pvc) and dynamically provision persistent volumes for them.

Let’s create a storageclass definition file named default-sc.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: default-ebs-sc
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: ebs.csi.eks.amazonaws.com
volumeBindingMode: WaitForFirstConsumer
parameters:
  type: gp3
  encrypted: "true"

We can deploy this storageclass to our cluster by using command: kubectl apply -f default-sc.yaml Now if we check our storage classes in our cluster, should see something like:

Deploying the sample wordpress application

Now that we’ve our default storage class, we can move into deploying the wordpress application. In order to deploy our wordpress application, wordpress uses MySQL as backend database so we first need to deploy MySQL deployment in our cluster. To do that, we first need to set a database password which we will use later to connect and authenticate to our database. Let us create a root DB password by creating a Kubernetes secret object:

kubectl create secret generic mysql-pass --from-literal password="mysecretpass"

After creating a secret, we can deploy our MySQL deployment into the cluster by creating mysql-deployment.yaml definition file and apply that using kubectl apply -f mysql-deployment.yaml:

In Production environment, you should not use this simple method of creating database and should consider more secure and reliable approach such as using MySQL Operator or using managed services like Aurora and RDS.

apiVersion: v1
kind: Service
metadata:
  name: wordpress-mysql
  labels:
    app: wordpress
spec:
  ports:
    - port: 3306
  selector:
    app: wordpress
    tier: mysql
  clusterIP: None
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
  labels:
    app: wordpress
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: wordpress-mysql
  labels:
    app: wordpress
spec:
  selector:
    matchLabels:
      app: wordpress
      tier: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: wordpress
        tier: mysql
    spec:
      containers:
      - image: mysql:8.0
        name: mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-pass
              key: password
        - name: MYSQL_DATABASE
          value: wordpress
        - name: MYSQL_USER
          value: wordpress
        - name: MYSQL_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-pass
              key: password
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pv-claim

You can also check your deployment is created successfully and check some logs of mysql pod to ensure that it is ready to accept database connections.

Now, we can deploy our wordpress definition file and its service using wordpress-deployment.yaml definition file and apply that using kubectl apply -f wordpress-deployment.yaml

apiVersion: v1
kind: Service
metadata:
  name: wordpress
  labels:
    app: wordpress
spec:
  ports:
    - port: 80
  selector:
    app: wordpress
    tier: frontend
  type: ClusterIP
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: wp-pv-claim
  labels:
    app: wordpress
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: wordpress
  labels:
    app: wordpress
spec:
  selector:
    matchLabels:
      app: wordpress
      tier: frontend
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: wordpress
        tier: frontend
    spec:
      containers:
      - image: wordpress:6.2.1-apache
        name: wordpress
        env:
        - name: WORDPRESS_DB_HOST
          value: wordpress-mysql
        - name: WORDPRESS_DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-pass
              key: password
        - name: WORDPRESS_DB_USER
          value: wordpress
        ports:
        - containerPort: 80
          name: wordpress
        volumeMounts:
        - name: wordpress-persistent-storage
          mountPath: /var/www/html
      volumes:
      - name: wordpress-persistent-storage
        persistentVolumeClaim:
          claimName: wp-pv-claim

You can also check your deployment is created successfully and check some logs of wordpress pod to ensure that it is deployed without any issues:

In this example, we only created ClusterIP type service to expose traffic into our wordpress deployment. You can check that out by using kubectl get svc command. So, you would say how we can test our service is accepting traffic or not?

Right, for that we would simply use port-forwarding in this section to see that our application is working properly and in next section, I will show you how to expose traffic into our application to the external world using ingress.

To use port-forwarding, we would simply use kubectl port-forward svc/wordpress 30000:80 to forward traffic coming into port 30000 of our localhost to port 80 of our wordpress service.

And if you simply call port 30000 of our localhost, should see something similar like this:

This means that your service can return a html file when the user request is sent and so it is a success. Let’s move into next section to fully expose our application using Ingress obejct.

How can we expose our application traffic?

In this section, we will dive deeply into exposing our application traffic using Ingress Kubernetes objects. You may probably know that if we want to expose traffic in Kubernetes clusters, we usually use Ingress for controlling traffic based on hostname, path and path prefixes etc. Before Auto Mode, if we want to use AWS ELB Load Balancers in our EKS clusters, we have to install AWS LoadBalancer Controller to manage lifecycle and creation of ALB(Application Load Balancer) or NLB(Network Load Balancer).

But with the EKS Auto Mode, we no longer need to install additional controller for that purpose and can easily use Ingress objects just with a few pre-requisites.

AWS suggests using Application Load Balancers (ALB) to serve HTTP and HTTPS traffic. So, in this example we will use ALB to expose our WordPress application.

Step 1: Create Ingress Class Parameters

Create an IngressClassParams object to specify AWS specific configuration options for the Application Load Balancer. Use the reference below to create an IngressClassParams object named ingress-class-params.yaml

apiVersion: eks.amazonaws.com/v1
kind: IngressClassParams
metadata:
  name: alb
spec:
  scheme: internet-facing

Apply this by using kubectl apply -f ingress-class-params.yaml

Step 2: Create IngressClass

Create an IngressClass that references the AWS specific configuration values set in the IngressClassParams resource. Note the name of the IngressClass . In this example, both the IngressClass and IngressClassParams are named alb.

Use the is-default-class annotation to control if Ingress resources should use this class by default.

Create a file named ingress-class.yaml with following content:

apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: alb
  annotations:
    # Use this annotation to set an IngressClass as Default
    # If an Ingress doesn't specify a class, it will use the Default
    ingressclass.kubernetes.io/is-default-class: "true"
spec:
  # Configures the IngressClass to use EKS Auto Mode
  controller: eks.amazonaws.com/alb
  parameters:
    apiGroup: eks.amazonaws.com
    kind: IngressClassParams
    # Use the name of the IngressClassParams set in the previous step
    name: alb

Apply this by using kubectl apply -f ingress-class.yaml

Step 3: Create Ingress

Create an Ingress resource. The purpose of this resource is to associate paths and ports on the Application Load Balancer with workloads in your cluster. Create a Ingress definition file named wordpress-ingress.yaml with following content:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: wordpress-ingress
spec:
  # this matches the name of IngressClass.
  # this can be omitted if you have a default ingressClass in cluster: the one with ingressclass.kubernetes.io/is-default-class: "true"  annotation
  ingressClassName: alb
  rules:
    - http:
        paths:
          - path: /*
            pathType: ImplementationSpecific
            backend:
              service:
                name: wordpress
                port:
                  number: 80

Apply this by using kubectl apply -f wordpress-ingress.yaml

Step 4: Access your application

Use kubectl get ingress to find the status of the Ingress. It can take a few minutes for the load balancer to become available. You can also get the LoadBalancer endpoint by using kubectl get ingress wordpress-ingress -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'

Now, if you call this returned URL in your browser, you should see the fresh installation of WordPress site like this:

In this example, I only used http port and no SSL certificate. In the production workloads, if you want to serve https traffic, you can set the default configurations of which ACM certificates to use in IngressClassParams object.

Pod Identity is built-in EKS Auto Mode

In this section, we will look into how EKS Auto Mode handles authentication and authorization for our pods in the cluster. So, let’s talk first about how the pods in Kubernetes communicate with different AWS services in general. To authenticate our pods to AWS services such as S3, Lambda, Step Functions or any other services, we have to give IAM roles to assume from our pods, right.

We can either use IRSA(IAM Roles for Service Accounts) or we can install Pod Identity Agent add-on and use for gaining fine-grained permissions for our application pods. With EKS Auto Mode, we no longer need to install this Pod Identity add-on anymore and it is included by default when we first created our cluster.

Without further ado, let’s actually see how the Pod Identity works in real life:

Step 1: Create an IAM role and policy

Next step is to create an IAM role with correct policy permission to allow access to different AWS services from your application pod. In this example, I want to allow my testing pod to be able to list s3 objects and buckets. So, let’s create an IAM role and attach S3ReadOnly permission to that role,

First, create a file called trust-policy.json with following content:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "pods.eks.amazonaws.com"
            },
            "Action": [
                "sts:AssumeRole",
                "sts:TagSession"
            ]
        }
    ]
}

Create the role with the command:

aws iam create-role \
    --role-name EksPodS3ReadOnlyRole \
    --assume-role-policy-document file://trust-policy.json

Finally, attach the S3 ReadOnly access that is provided by AWS default with our IAM role created:

aws iam attach-role-policy \
    --role-name EksPodS3ReadOnlyRole \
    --policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

Step 2: Testing without pod-identity permission

In this step, before connecting our Kubernetes service account with IAM role we create above, we will first test that without the correct permission, our testing pod cannot call to S3 service on our behalf.

If we run a testing aws cli pod with image provided from AWS: kubectl run --rm -it cli-pod --image=amazon/aws-cli --restart=Never --command -- aws s3 ls, we can see in the below screenshot that the pod terminated with Error because it cannot locate correct AWS credentials.

Step 3: How to create a pod identity associations

To create a pod identity association in our EKS cluster, go to EKS console and under Access section, there is a section called Pod Identity associations where we can create bindings of our IAM roles and EKS service acounts.

In this example, I will just give our default service account in the default namespace with the IAM role we created above.

Step 4: Verify everything is working

Finally, after creating required IAM roles, attached policies and bind it to our Kubernetes service account, if we re-run our test cli pod with following command: kubectl run --rm -it cli-pod --image=amazon/aws-cli --restart=Never --command -- aws s3 ls We can see that our pod can now successfully connect to s3 service and list buckets under my account.

If you want to give different workloads of different permissions to AWS services, you can also create another service account in the same namespace and attach it to different IAM roles with appropriate policies.

Summary

In this series of reviewing EKS Auto Mode features and its benefits, we’ve got to see how the AWS EKS Auto Mode handles different things for us in terms of Load Balancing, Nodes Provisioning and Pod Identity Configurations. Before the Auto Mode, it is almost impossible to run and manage your workloads on EKS without the proper knowledge of how to configure things and different aspects of different tools such as IRSA(IAM role for service account), LBC (Load Balancer Controller) for provisioning Ingress, Karpenter or Cluster Autoscaler for dynamic nodes provisioning etc..

But with the introduction of Auto Mode, it completely changes how we manage our EKS clusters and different components. In fact, we don’t even need to install many different components for our clusters to work properly. So, it is a very good feature for us and I hope to see many more enhancements and added features for Auto Mode in the near upcoming future too.

In the future, I will try to also write more about EKS Auto Mode including what is the different purposes for default node pools such as system and general-purpose, how to use customized node pools for different workloads such as your DEV, TEST and PROD workloads can have each set of NodePool configurations. Also, how to use network policies in EKS Auto Mode etc..

Thank you all for reading and hope you can follow along and test with me too. 😉

EKSAWSDevopsKubernetesautomation