Build A Django CI/CD Pipeline using Jenkins
I’ve built multiple Django projects and deployed on servers for testing on a production like environment. Now there was an admin overhead that came with that. Anytime I made a code change I had to connect to the server, use version control to update the code and restart services like nginx and gunicorn in order for my changes to reflect on the frontend. Now imagine doing that a hundred times (pulling hair and screaming in agony). Luckily for the past year I’ve been learning about DevOps and some nice tools that come with it. (DrumRoll) Introducing Jenkins, this is a CI/CD tool that removes the admin overhead that comes with repetitive tasks. Now let’s try removing that admin overhead I complained about earlier.
Prerequisites
You’ll need a VPS. You can get one on AWS or GCP. I’ll be using AWS in this article. You can refer to my article on setting up a VPS on AWS and also configuring a swap space for your VPS. While setting up the server security groups on AWS, open the following ports 22, 80 and 8080. This will allow you connect to the Jenkins UI and the sample website we’ll be using. We’ll be using a CentOS 7 image for this deployment. Make sure you have a normal user with sudo privileges to perform all the installation and setup. You can refer to this article on how to create a normal user.
Step 1: Install Jenkins and extra packages
We’ll be using a repository that I’ve setup with all the necessary scripts and also the Jenkins pipeline. This project was set up for the use case in this article but using the configuration in this repo you can create a CI/CD for any project. Go through the README to understand the workings in the project. You can check it out here. Open the repository and check out the contents of initial-setup.sh . This helps you install some necessary packages. Copy the contents of the file to the server.
$ vi initial-setup.sh
Save and close the file. Run the script to install the packages.
$ chmod +x initial-setup.sh
$ ./initial-setup.sh
Once the script is done, visit your servers public IP on port 8080 to view the Jenkins dashboard (http://<YOUR SERVER’S IP>:8080). You’ll see a screen like the one below.
To get your administrator password, run this command to get it.
$ sudo cat /var/lib/jenkins/secrets/initialAdminPassword
This will output your administrator password. The username associated with this password is admin incase you want to log in with this user. Once you input your administrator password you get a page asking for installation of plugins.
Select Install suggested plugins to install some standard plugins that will help during your pipeline setup. Once the installation is done you are asked to create a normal user.
After creating the user, you are asked to set your Jenkins URL. Leave it as default and now you should be logged in.
Step 2: Disable SELinux and extra Jenkins configuration
SELinux defines access controls for the applications, processes, and files on a system. It uses security policies, which are a set of rules that tell SELinux what can or can’t be accessed, to enforce the access allowed by a policy. But when trying to debug permission problems, it might make sense to temporarily disable SELinux. In this case, you can opt to have SELinux run in permissive mode, either for the entire system, or for a specific (set of) types or disable it. Run this command to check the status of SELinux:
$ sestatus
If it is enabled and set to enforcing mode, then we’ll change it to permissive mode. Nginx might not get access to your socket file if these instructions are not carried out. Open the SELinux configuration file:
$ sudo vi /etc/selinux/config
Change enforced to permissive
// Remove
SELINUX=enforcing
// Replace with
SELINUX=permissive
Save and close the file. We are meant to reboot the server for the changes to take effect but before that let’s do some Jenkins configuration for ease of use. Using Jenkins on CentOS is a hassle. One reason is that the Jenkins user is set up as a service account instead of a user account. This prevents the Jenkins user from having superuser-like privileges. This is different on Ubuntu where the Jenkins user is a user account and I can grant the user sudo privileges by using visudo. For your jobs to run smoothly on CentOS, you have to switch the user running the jobs from Jenkins to a user with sudo privilege. I’ll be switching to centos user. Open up the this script (using VI or other editor):
$ sudo vi /etc/sysconfig/jenkins
Find this line and change to “centos” or any user you prefer:
$JENKINS_USER="centos"
Then change the ownership of Jenkins home, webroot and logs:
$ sudo chown -R centos:centos /var/lib/jenkins
$ sudo chown -R centos:centos /var/cache/jenkins
$ sudo chown -R centos:centos /var/log/jenkins
Restart Jenkins and check the user has been changed:
$ sudo /etc/init.d/jenkins restart
$ ps -ef | grep jenkins
Now you should be able to run the Jenkins jobs as the centos user. Now you can reboot the server:
$ sudo shutdown now -r
Reconnect back to your instance and check the status of SELinux:
$ sestatus
If it has changed to permissive then you’re good to go.
Step 3: Setup Jenkins Pipeline
We’re almost ready to setup the Jenkins pipeline. We have to do a little bit of housekeeping. Log in to the Jenkins dashboard.
On the left sidebar click on Manage Jenkins. This opens the Management dashboard of Jenkins. Click on ‘Configure Global Security’. Scroll down and enable ‘Enable proxy compatibility’ on the ‘CSRF Protection’. Apply and Save.
Now we can setup the Jenkins pipeline. Go to Jenkins home and Click on New Item. Enter Name and select Pipeline:
Go on to the next page. Scroll down, switch pipeline definition to ‘Pipeline Script from SCM’, Select Git as the SCM and Input your repository URL.
Add a trigger to the pipeline to rerun when there’s a new commit. This is what updates the code changes to your application automatically without you manually running the commands. You can add multiple triggers like running at different times of the day. The triggers makes use of the linux cron job string format. You can read more about it here.
Once all this is done you can Apply and Save. Your Pipeline should start running anytime soon. If you check the README of the project used in this article there are some prerequisite steps that should be carried out to have everything running smoothly.
The pipeline ran successfully. Though this doesn’t guarantee that everything is okay. You can check the logs of the stages in the pipeline to verify that everything is working.
After checking the logs of all stages looks like everything ran fine. Now I can visit the webpage using the IP Address of the server.
Now with the way the pipeline was configured if there’s a new commit on the repository, it’ll pull the changes and redeploy your application without your manual intervention.
Congrats!!!
We’ve been able to eliminate the admin overhead that comes with testing your Django project in a production like environment. This will allow you make changes and see the effects almost immediately. Goodluck!!!
REFERENCE Links:
