Tuesday, September 30, 2014

Getting started with Jenkins CI



1          CI (Continuous Integration)

Continuous integration is nothing but automatically taking the latest copy (Head version) of the source code from the Subversion/Git/CVS repository and deploy to the DEV/QA/PROD servers in a given time interval. Jenkins/Team city/Bamboo (Continuous integration tool) can be configured in such a way that It will take the latest version of code from SVN repository every day/at some time interval and deploy it to say “DEV/QA” environment automatically.





There are 4 phases in build process:

Build

1-      Checkout the latest code from version control repository.
2-      Build the code and prepare artifacts (Jar/WAR/ZIP etc.)
3-      Execute unit test cases (JUnit, NUnit, XQuery Unit etc.)
4-      Run code analysis tools (PMD/FindBugs/XQAnalyser etc.)
5-      Generate reports for the unit tests and code analysis and publish on build tool.

Deployment

1-      Connect to DEV environment.
2-      Push the newly build artifacts to a folder on DEV server (any name can be used).
3-      Stop the application server (wait for the app server to fully stop).
4-      Look for deployment directory.
5-      Take the backup of existing artifacts to backup folder from deployment directory.
6-      Clean the deployment directory.
7-      Copy the new artifacts to deployment directory.
8-      Start the application server.
9-      Perform the post deployment steps (if any)

 Post Deployment (if any, optional)

1-      Configure databases if there is any configuration changes are there.
2-      Execute cleanup scripts.


Notify the development team via emails (success/failure)


2          Jenkins

Jenkins is an open source continuous integration tool written in Java. The project was forked from Hudson after a dispute with Oracle.
Jenkins provides continuous integration services for software development. It is a server-based system running in a servlet container such as Apache Tomcat. It supports SCM tools including AccuRev, CVS, Subversion, Git, Mercurial, Perforce, Clearcase and RTC, and can execute Apache Ant and Apache Maven based projects as well as arbitrary shell scripts and Windows batch commands. The primary developer of Jenkins is Kohsuke Kawaguchi. Released under the MIT License, Jenkins is free software.
Builds can be started by various means, including being triggered by commit in a version control system, scheduling via a cron-like mechanism, building when other builds have completed, and by requesting a specific build URL.


2.1         Install Jenkins


2.1.1        Installing and running in a servlet container

You can install a servlet container like GlassFish and Tomcat, which can run as a service by itself, and then deploy Jenkins to it.

2.1.1.1       Download the war file from the below given URL.

2.1.1.2       Deploy the jenkins.war file to webapps directory of apache tomcat.

2.1.1.3       Restart the tomcat and wait till the war gets deployed.

2.1.1.4       Hit the following URL to open Jenkins. You will see the Jenkins GUI.

            Note: This process is common for all type of operating systems.


2.1.2        Installing and running as a windows/linux service

2.1.2.1       Download the executable file from the below given URL.

                                 Windows installer: http://mirrors.jenkins-ci.org/windows/latest
                                 Linux installer: http://pkg.jenkins-ci.org/redhat/
                                 You can find the installers for other OS at : http://jenkins-ci.org/

2.1.2.1       Install the package using “setup.exe” (windows installer)

2.1.2.2       Once installation completes, you can open the Jenkins GUI in the same way as discussed above.


                                   First look of installed Jenkins GUI:












3       Configuring Jenkins


3.1         Creating new job.

3.1.1   Click on the “New Item” to create a new job in the left side on Jenkins.

3.1.2      Select a type of project you want to create, read the description about each type if required. For e.g. I selected “Freestyle Project”.

3.1.3      You can also copy from existing job as well using the last option (Copy existing item), it will create the clone of given job.

3.1.4       Provide the name of the job, for e.g. “Test-Job” and click “OK”. It will create a job with given name and you will lang to the configuration page.


Here, you can provide the “Description” about the job. You can configure no. of builds    to display on the UI using “Discard Old Builds” option.
You can disable the builds using “Disable build“ option if you want because sometimes    you want to temporarily stop building a project.
You can also enable the option to execute the concurrent builds using the “Execute concurrent builds if necessary” option. If this option is checked, Jenkins will schedule and execute multiple builds concurrently (provided that you have sufficient executors and incoming build requests.) This is useful on builds and test jobs that take a long time,as each build will only contain a smaller number of changes, and the total turn-around time decreases due to the shorter time a build request spends waiting for the previous build to complete. 


3.1.5        Advanced Project Options
           
              This option contains following configurations.
·       Quiet period - If set, a newly scheduled build waits for this many seconds before actually being built. For e.g. you have provided 10 as a value then it will wait for 10 seconds. If not explicitly set at project-level, the system-wide default value (5) is used.
·       SCM checkout retry count - If a build fails to checkout from the repository, Jenkins will retry the specified no. of times before giving up. Default is “0”
·       Block build when upstream project is building - When this option is checked, Jenkins will prevent the project from building when a dependency of this project is in the queue, or building. The dependencies include the direct as well as the transitive dependencies.
·       Block build when downstream project is building - When this option is checked, Jenkins will prevent the project from building when a child of this project is in the queue, or building. The children include the direct as well as the transitive children.
·       Use custom workspace - For each job on Jenkins, Jenkins allocates a unique "workspace directory." This is the directory where the code is checked out and builds happen. Normally you should let Jenkins allocate and clean up workspace directories, but in several situations this is problematic, and in such case, this option lets you specify the workspace location manually.
One such situation is where paths are hard-coded and the code needs to be built on a specific location. While there's no doubt that such a build is not ideal, this option allows you to get going in such a situation.
Another situation where this is useful is when you are using the project type not to perform a software build, but execution of a certain batch task, perhaps as a cron replacement. In such case, you can use this option to map the relevant directory as the workspace, so that people can look at files through the Jenkins web UI, and you can kick relevant commands more easily.
If you are in a distributed build environment, unless you tie a job to a specific node, Jenkins may still move around jobs to different slaves. Sometimes this is desirable, sometimes this is not. Also, you can map multiple projects to have the same workspace, but if you do so, make sure concurrent executions of those jobs won't have nasty interferance with each other.
If this path is relative, it's resolved against the "remote FS root" directory of the slave, or $JENKINS_HOME on the master.
For instance, I am using a custom workspace as
C:\.jenkins\jobs\Test-Job\workspace

·       Keep the build logs of dependencies- If this option is enabled, all of the builds that are referenced from builds of this project  will be protected from log rotation.
When your job depends on other jobs on Jenkins and you occasionally need to tag your workspace, it's often convenient/necessary to also tag your dependencies on Jenkins. The problem is that the log rotation could interfere with this, since the build your project is using might already be log rotated (if there have been a lot of builds in your dependency), and if that happens you won't be able to reliably tag your dependencies.
This feature fixes that problem by "locking" those builds that you depend on, thereby guaranteeing that you can always tag your complete dependencies.


3.1.6        Source code management          

In this section source code repository will be configured, Jenkins will use the repository   URL to download the latest code for build process. Jenkins will provide the “Subversion” and “CVS” option out of the box. I will cover configuring SVN (Subversion) option here. 

         Configuring Subversion:
           Subversion (SVN) is most widely used as a source code management tool. 
           Select the Subversion option from the above shown radio buttons.
                       Follow the below steps for configuration
·         Provide the subversion “Repository URL”
·         Provide the credentials, use “Add” button to authenticate from repository.
·         Leave the other options as is.




·       Select the “Check-out Strategy” from the below options as per your requirement. I selected “Use svn update as much as possible” as my requirement is to only update the source code which is checked-out previously i.e. when you run the build for the first time Jenkins will check-out the latest code from the subversion repository and execute the build. When you run the build next time Jenkins will not checkout the whole code, only update operation will be performed as per my strategy opted.
·      Select the “Repository Browser” as “Auto

·         Leave the other options as is.


3.1.7        Build Triggers

In this section build triggers will be configured to trigger the build at some given events or time interval. Jenkins provides following type of build triggers.
·       Build after other projects are built - Set up a trigger so that when some other projects finish building, a new build is scheduled for this project. This is convenient for running an extensive test after a build is complete, for example. This trigger will watch for the job provided under “Projects to watch” option and triggers the build based on below given options.
This trigger provides 3 options
ü  Trigger only if build is stable
ü  Trigger even if the build is unstable
ü  Trigger even if the build fails
For e.g. 
·       Build periodically - This feature is primarily for using Jenkins as a cron replacement, and it is not ideal for continuously building software projects.
When people first start continuous integration, they are often so used to the idea of regularly scheduled builds like nightly/weekly that they use this feature. However, the point of continuous integration is to start a build as soon as a change is made, to provide a quick feedback to the change. To do that you need to hook up SCM change notification to Jenkins.
So, before using this feature, stop and ask yourself if this is really what you want.
In general, this option will not check for svn changes on the given time interval, it will only run the job iteratively based on the interval provided.
·       Poll SCM - Configure Jenkins to poll changes in SCM. This option will allow Jenkins to keep checking the code changes in the repository as per the configured time interval and if it founds changes then triggers the build.
For e.g.  If you have configured this option for time interval say “every-15-minute (H/15 * * * *) ” then Jenkins will check for the code changes in evry 15 minutes and if founds changes then triggers the build.
Examples:
# every fifteen minutes (perhaps at :07, :22, :37, :52)
H/15 * * * *
# every ten minutes in the first half of every hour (three times, perhaps at :04, :14, :24)
H(0-29)/10 * * * *
# once every two hours every weekday (perhaps at 10:38 AM, 12:38 PM, 2:38 PM, 4:38 PM)
H 9-16/2 * * 1-5
# once a day on the 1st and 15th of every month except December
H H 1,15 1-11 *
Note: Build will not be triggered untill there is a change in svn repository.
In “Build periodically” approcah Jenkins will execute the build periodically without taking care of code changes. Where “Poll SCM” will build only if there is a code change in the SVN.

3.1.8       Build


In this section you will add the build steps i.e. how the build will be executed.
For example, you would first execute a command to build the jar/war etc. then perform code analysis after that run Junit test cases and if everything completed correctly then you would deploy the build to a remote host where your application will run.

3.1.8.1       Building the project using “Windows batch command”

Runs a Windows batch script for building the project. The script will be run with the workspace as the current directory. The text you enter in the text box will be executed as a batch file, and a build will be considered a failure if at the end of the execution %ERRORLEVEL% is not 0. If you already have a batch file in SCM, you can just type in the path of that batch file (again relative to the workspace directory), and simply execute that. 

Here I am invoking a batch file which will process all the build steps as discussed example above 
        and ‘dev’ and ‘3.3.0.2’ are external params provided to “.bat” file which is environment and the build number for the build.

3.1.8.2       Building the project using “Execute shell"              

If you're using a shell script to do your build, you can either put these environment variables  directly into your shell scripts, or call them as parameters in your shell script. Below is an example how this can be done:


Runs a shell script (defaults to sh, but this is configurable) for building the project. The    script will be run with the workspace as the current directory. Type in the contents of  your shell script. If your shell script has no header line like #!/bin/sh —, then the shell configured system-wide will be used, but you can also use the header line to write script      in another language (like #!/bin/perl) or control the options that shell uses.
By default, the shell will be invoked with the "-ex" option. So all of the commands are      printed before being executed, and the build is considered a failure if any of the   commands exits with a non-zero exit code. Again, add the #!/bin/... line to change this   behavior.
As a best practice, try not to put a long shell script in here. Instead, consider adding the shell script in SCM and simply call that shell script from Jenkins (via bash -ex   myscript.sh or something like that), so that you can track changes in your shell script.


3.1.8.3       Building the project using “Invoke ant”

This causes Jenkins to invoke Ant with the given targets and options. Any non-zero exit code causes Jenkins to mark the build as a failure.
If you're using an Ant script to do your build, you may include environment variables in property settings. Click on the Advanced... button just below where you put the Ant targets you want to build. This will display the Properties box. Below is an example how to use the Properties box to set Ant properties with Jenkins Environment variables:












3.1.9   Post-build Actions

 In this section you can add the tasks to be performed after the build for e.g. publish junit test report, generate javadocs, publish pmd report sent email etc.

I have configured some options as given in below screen:






















4          Executing build

To execute the build you can click on “Build” button, it will execute the build based on the configurations provided as discussed above.




Once you click on “Build” button, Jenkins will start the execution of build. You can see the progress in the build history section.



You can see the build log by clicking on the build displayed in “Build History”, and then click on the “Console Output” link.






For further details visit: https://wiki.jenkins-ci.org/display/JENKINS/Use+Jenkins



5          Securing Jenkins using AD


Active-Directory Integration with Jenkins
Follow the steps given below to integrate Active directory with Jenkins
Step 1- Install the “Active Directory plugin 1.36 (This plugin enables authentication through Active Directory)”, if already not installed.
Step 2- Restart the Jenkins server after installation of plugin.
Step 3- Now, go to Manage Jenkins > Manage Plugins > Installed Tab
You will be able to see the “Active Directory plugin installed.”



















Step 4- Go to Manage Jenkins > Configure Global Security  and check “Enable Security












After clicking on “Enable Security”, below page will be displayed. Now you can choose the method to implement the security layer in Jenkins.

























Step 5- Let select the “Active Directory” option and also click on Advance button, as we are concerned about AD here. You will see following screen.











Step 5- Fill up the fields given ave with below given details, and then click “Test” button.


Configuration: (Here i am using fake inputs, you have to provide the correct inputs in order to connect to AD)
Domain Name:  xxxx.in
Domain Controller: test.xxxx.in
Site/Server: 10.192.168.26:389
Bind DN: abhimishra
Bind Password: passw0rd
Active Directory Integration Successful:














Roles Management:




Once everything is completed you will see, Log-In Screen


Provide your User Name & Password to Login.