In a Jenkins Master Slave Continuous Integration (CI) /Continuous Delivery (CD) approach, if the master goes down, the entire build system will be affected and the project/product cannot be released on time. In this blog, we will outline how to establish high availability of Jenkins to avoid business impact.
Jenkins is one of the leading ASCII text file automation servers for continuous integration and automation with its various plugins that support building, deploying and automating any project. Jenkins is beneficial when you need to execute a predefined list of tasks as quickly as possible due to a change in repository, trigger a pipeline that executes totally different jobs and perform the tasks.
Each part is monitored, and notifications will be triggered to the user if something breaks. It allows you the privilege of preventing tasks from being executed. This is useful for an outsized project where multiple individuals work on code without knowing the changes their colleagues are making. This leads to increased risk of errors and bugs once the codes are integrated into the branch. Jenkins detects the bug once each code is committed and advises the developers about the issue.
To solve the problem outlined above, CloudifyOps recommended implementing a multi-master Jenkins setup (active and standby) approach.
Requirements:
- Create 3 servers (one is lightweight server for nginx-proxy).
- Create an EFS file system to attach it as a backend to the Jenkins master servers.
Create EFS file system: Open your AWS account and navigate to the EFS and click on create file system. You will see the page shown in the screenshot below.
Provide a name for the EFS. Select the Virtual Private Cloud (VPC). The VPC must be in the same server where our Jenkins server is hosted.
If you have servers in 2 different availability zones, choose regional (it helps to attach servers which are in 2 different availability zones).
EFS file system creation is complete and now we need to install the nfs client in both instances.
Open both servers and run the below commands.
- sudo apt update -y
- sudo apt install nfs-common -y
Mount EFS to the Jenkins servers
Create Jenkins folder to use as a mount point
- mkdir /var/lib/Jenkins
Replace file system id and region names in the below commands with your file system id and region name.
Check if we can reach the file system using the below command
- nslookup file-system-id.efs.aws-region.amazonaws.com
You will get an output as seen here.
Use the below command to mount the filesystem.
- sudo mount -t nfs -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport file-system-id.efs.aws-region.amazonaws.com:/ /var/lib/Jenkins
You can check the mount using this command
- df -h
At the bottom of the image, you can see the file system is mounted on the path /var/lib/Jenkins
Repeat the steps for both the servers . Now we need to edit the /etc/fstab to make the mount consistent
- Open the etc/fstab and add the below line to the file and save and close it (replace region name and file system id).
file-system-id.efs.aws-region.amazonaws.com:/ /var/lib/Jenkins nfs
Jenkins Installation:
Now install Jenkins in both servers
sudo apt install openjdk-11-jdk
wget -q -O — https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add –
sudo sh -c ‘echo deb https://pkg.jenkins.io/debian-stable binary/ > \
/etc/apt/sources.list.d/jenkins.list’
sudo apt-get update
sudo apt-get install Jenkins
Configure Nginx-proxy:-
Now open the nginx light-weight server and configure the nginx proxy.
Install the nginx in the server using below command
- apt get update
- apt install nginx -y
Now navigate to the /etc/nginx
Here you need to edit a file named nginx.conf
You will also find directories with the names
Sites-enabled
Sites-available
Identify the default file in these directories, edit them and add the below content to the default files.
Add the below configuration having your Jenkins servers public IP addresses placed in it.
events {
worker_connections 1024;
}
stream {
upstream stream_backend {
server 54.160.28.123:8080;
server 3.91.102.187:8080 backup;
}
server {
listen 80;
proxy_connect_timeout 1s;
proxy_timeout 3s;
proxy_pass stream_backend;
}
}
Place this configuration in the nginx.conf and you’ll find default file inside the sites-enabled and sites-available
Replace those default files with the above content.
After adding this, restart the nginx service
- sudo service nginx restart
Now when we hit the nginx server public ip address that will redirect to the Jenkins server.
Note:
Some times nginx modules will not be downloaded automatically for that you need to run the below command
- apt install nginx-mod-stream
After installing the module also we need to restart the nginx.
To learn more about these cutting edge technologies & real time industry applied best practices, follow our LinkedIn Page. To explore our services, visit our website.
#cloudifyops #jenkins #continuousintegration #continuousdeployment, #multimaster #efs #AWS #jenkinshighavailability
