Deploy your Dockerized Application Using a CI/CD pipeline on Kubernetes

Earlier this year I got the Bertelsmann scholarship. I chose the cloud track and the final capstone project was to deploy a dockerized application using Jenkins on EKS(Elastic Kubernetes Service). Just to give some context and meaning to the services and tools used, EKS is AWS managed Kubernetes Service. It allows you provision a Kubernetes Cluster on the AWS cloud environment. Jenkins is a CI/CD tool that automates processes using a Jenkinsfile(Jenkins configuration file). The Jenkinsfile states the stages and the processes to be carried out in each stage. This article details the steps taken to accomplish this task.

Prerequisites
You’ll need a VPS. You can get one on AWS or GCP. I’ll be using AWS in this article. You can refer to my article on setting up a VPS on AWS and also configuring a swap space for your VPS. While setting up the server security groups on AWS, open the following ports 22 and 8080. This will allow you connect to the Jenkins UI. We’ll be using an Ubuntu 18.04 image for this deployment.

Step 1: Install Jenkins and extra packages
We’ll be using a repository that I’ve setup with all the necessary scripts and also the Jenkins pipeline. This project was set up for the use case in this article but using the configuration in this repo you can create a CI/CD for any project. Go through the README to understand the workings in the project. You can check it out here. Open the repository and check out the contents of initial-setup.sh . This helps you install some necessary packages. Copy the contents of the file to the server.

$ vi initial-setup.sh

Save and close the file. Run the script to install the packages.

$ chmod +x initial-setup.sh
$ ./initial-setup.sh

Once the script is done, visit your servers public IP on port 8080 to view the Jenkins dashboard (http://<YOUR SERVER’S IP>:8080). You’ll see a screen like the one below.

To get your administrator password, run this command to get it.

$ sudo cat /var/lib/jenkins/secrets/initialAdminPassword

This will output your administrator password. The username associated with this password is admin incase you want to log in with this user. Once you input your administrator password you get a page asking for installation of plugins.

Select Install suggested plugins to install some standard plugins that will help during your pipeline setup. Once the installation is done you are asked to create a normal user.

After creating the user, you are asked to set your Jenkins URL. Leave it as default and now you should be logged in.

We have to do a little bit of housekeeping. On the left sidebar, click on ‘Manage Jenkins’. This opens the Management dashboard of Jenkins. Click on ‘Configure Global Security’. Scroll down and enable ‘Enable proxy compatibility’ on the ‘CSRF Protection’. Apply and Save.

We’ll be using Blue Ocean plugin on Jenkins to setup our CI/CD pipeline. Blue Ocean is just a new elegant UI to the classic Jenkins. Even though Jenkins is extensible and robust, its poor user experience is criticised a lot. Jenkins introduced Blue Ocean to improve the user experience along with some benefits. On the left sidebar click on Manage Jenkins. This opens the Management dashboard of Jenkins. Click on ‘Manage Plugins’. Click on ‘Available’ and then search for ‘Blue Ocean’.

You can select all the Blue Ocean Plugins and Click on ‘Download now and install after restart’. Once the download is complete, click on ‘Go back to the top page’. Go back to the plugins manager and search for aws on the available plugins. Install the ‘Pipeline: AWS Steps’. Click on ‘Download now and install after restart’.

Once download is complete, Click on ‘Restart Jenkins when installation is complete and no jobs are running’.

Once you click on it, you’ll be redirected to a waiting page for the update to complete. Once the update is completed, you’ll be redirected back to the Jenkins dashboard.

Sometimes you might get stuck on this page. You can refresh and see if the dashboard comes up. Also sometimes the Jenkins server might not restart for the new plugins to install successfully. You can connect to your EC2 using your Linux terminal or Putty. Once connected, you can check the status of the Jenkins server and also if the runtime matches with when the plugins were installed. If the runtime matches, you can ignore restarting the Jenkins server.

# Check status of Jenkins server
$ systemctl status jenkins

# Restart the Jenkins server
$ sudo systemctl restart jenkins

Step 2: Setup AWS CLI and provision Kubernetes Cluster
While using Jenkins, the shell commands in the pipeline stages are executed using the Jenkins user. Some of this commands need to be run using super user privileges. We’ll give the Jenkins user super privileges so that these commands will run without error in the pipeline.

$ sudo visudo -f /etc/sudoers

Add following line at the end of the visudo file.

jenkins ALL= NOPASSWD: ALL

This will also allow the Jenkins User to run su (super user) commands without the need to input a password. Once this is done, switch to the Jenkins user on the terminal.

$ sudo su - jenkins

Next we need to configure the AWS CLI with an IAM user with programmatic access. This IAM user needs to have the necessary privileges to create and manage the EKS Cluster. You can see how to do that here. I’ll advice you to grant the IAM user AdministratorAccess to make provisioning of the cluster easier for you. Once you create the IAM user, take note of the Access key ID and Secret access key. Now configure the AWS CLI with the IAM user. You should be logged in as the Jenkins user on the terminal. Use us-east-1 as the region while setting up the AWS CLI.

$ aws configure

Once the configuration is complete, run this command to make sure the IAM user was successfully configured.

$ aws s3 ls

You might get an error like this An error occurred (AccessDenied) when calling ListBuckets operation: Access Denied. This error is okay because you didn’t give the user permission to Access S3. You shouldn’t see this error if you grant the IAM user AdministratorAccess or Access to S3. Now that the AWS CLI is configured with an IAM user that has the necessary privileges we can go ahead and create the Kubernetes cluster. Run this command to create the cluster.

$ eksctl create cluster --name stage-cluster --version 1.14 --region 
us-east-1 --nodegroup-name standard-workers --node-type t2.small --
nodes 2 --nodes-min 1 --nodes-max 4 --managed

The creation of the Kubernetes cluster takes about 20–30 minutes to complete. You can also keep track of the creation on the AWS console. Once creation is complete, you can run this command to check your nodes. Note you should be logged in as the Jenkins user.

$ kubectl get nodes

Now that your cluster is up and running, we can move on to the next step.

Step 3: Setup AWS and Docker Credentials on Jenkins
Open your Jenkins dashboard (http://<YOUR SERVER’S IP>:8080). Login with the user that you created. On the left sidebar, click on ‘Manage Jenkins’. This opens the Management dashboard of Jenkins. Click on ‘Manage Credentials’.

Click on ‘global’. On the left sidebar, click on ‘Add credentials’. Now we’ll add your dockerhub credentials which is where the docker images that is built in the pipeline will be pushed to. Set the ID of the credential as ‘dockerhub’ because it is referenced by the Jenkinsfile.

Next add another credential using your AWS IAM user Access key ID and Secret access key. Set the ID of the credential as ‘aws’ because it is referenced by the Jenkinsfile.

Step 4: Setup pipeline on Blue Ocean
Open your Jenkins dashboard (http://<YOUR SERVER’S IP>:8080). On the left sidebar click on ‘Open Blue Ocean’. This will open up the Blue Ocean UI.

Setup the pipeline using the github repo that was pointed out earlier.

Immediately you’re done, the pipeline should start building immediately. You can click on the project to follow the deployment process.

This means that the application has been deployed successfully. To get the URL to visit, make sure you’re still logged in as the Jenkins user on the terminal. Then run the following commands.

# If not logged in
$ sudo su - jenkins

# To get nodes
$ kubectl get nodes

# To get pods running
$ kubectl get pods

# To get services and the URL of the application 
$ kubectl get services

Visiting the URL of the Load Balancer will show you the web application running.

You can also setup the pipeline to automatically rebuild and redeploy the application on Kubernetes when there’s any new commits in the repo. Click the gear sign on the pipeline dashboard. Go to the ‘Scan Repository Triggers’ and click the checkbox. Set it to 1 minute. Apply and Save.

Congrats!!!
We’ve been able to deploy a dockerized web application on Kubernetes Using Jenkins to manage the flow from code change to deployment. Any change you make on the source code and push to the repo will be automatically deployed without your manual intervention. Goodluck!!!

REFERENCE Links:
1) https://logz.io/blog/amazon-eks-cluster/
2) https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstances.html