git – Page 2 – DEVOPS DONE RIGHT

jgit-flow maven plugin to Release Java Application

Introduction

As a DevOps I need a smooth way to release the java application, so I compared two maven plugin that are used to release the java application and in the end I found that Jgit-flow plugin is far better than maven-release plugin on the basis of following points:

Maven-release plugin creates .backup and release.properties files to your working directory which can be committed mistakenly, when they should not be. jgit-flow maven plugin doesn’t create these files or any other file in your working directory.
Maven-release plugin create two tags.
Maven-release plugin does a build in the prepare goal and a build in the perform goal causing tests to run 2 times but jgit-flow maven plugin builds project once so tests run only once.
If something goes wrong during the maven plugin execution, It become very tough to roll it back, on the other hand jgit-flow maven plugin makes all changes into the branch and if you want to roll back just delete that branch.
jgit-flow maven plugin doesn’t run site-deploy
jgit-flow maven plugin provides option to turn on/off maven deployment
jgit-flow maven plugin provides option to turn on/off remote pushes/tagging
jgit-flow maven plugin keeps the master branch always at latest release version.

Now let’s see how to integrate Jgit-flow maven plugin and use it

How to use Jgit-flow maven Plugin for Release

Follow the flowing steps

Add the following lines in your pom.xml for source code management access
```
   scm:git:
  scm:git:git:
```

Add these line to resolve the Jgit-flow maven plugin and put the other option that will be required during the build

com.atlassian.maven.plugins
maven-jgitflow-plugin
1.0-m4.3
        
true
false
true
true
true
true
true
true
          
master-test
deploy-test

Above code snippet will perform following steps:

- Maven will resolve the jgitflow plug-in dependency
- In the configuration section, we describe how jgit-flow plug-in will behave.
- pushRelease XML tag to enable and disable jgit-flow from releasing the intermediate branches into the git or not.
- keepBranch XML tag to enable and disable the plug-in for keep the intermediate branch or not.
- noTag XMl tag to enable and disable the plug-in to create the that tag in git.
- allowUntracked XML tag to whether allow untracked file during the checking.
- flowInitContext XML tag is used to override the default and branch name of the jgit-flow plug-in
- In above code snippet, there is only two branches, master from where that code will be pulled and a intermediate branch that will be used by the jgit-flow plug-in. as I have discussed that jgit-flow plug-in uses the branches to keep it records. so development branch will be created by the plug-in that resides in the local not remotely, to track the release version etc.

To put your releases into the repository manager add these lines

<distributionManagement>
  <repository>
    <id><auth id></id>
    <url><repo url of repository managers></url>
  </repository>
  <snapshotRepository>
    <id><auth id></id>
    <url><repo url of repository managers></url>
  </snapshotRepository>
</distributionManagement>

Put the following lines into your m2/settings.xml with your repository manager credentials

<settings>
  <servers>
    <server>
        <id><PUT THE ID OF THE REPOSITORY OR SNAPSHOTS ID HERE></id>
       <username><USERNAME></username>
       <password><PASSWORD></password>
    </server>
  </servers>
</settings>

Start Release jgit-flow maven plugin command

To start the new release execute jgitflow:release-start.

Finish Release jgit-flow maven plugin command

To finish new release, execute mvn jgitflow:release-finish.
For a example I have created a repository in github.com. for testing and two branch master-test and deploy-test. It is assumed that you have configured maven and git your system.

In the deploy-test branch run following command
$ mvn clean -Dmaven.test.skip=true install jgitflow:release-start

This command will take input from you for release version and create a release branch with release/. then it will push this release branch into github repository for temporarily because we are not saving the intermediate branched

Now At the end run this command
$ mvn -Dmaven.test.skip=true jgitflow:release-finish
after finishing this command it will delete release/ from local and remote.

Now you can check the changes in pom file by jgitflow. in the above snapshot, it is master-test branch, you can see in the tag it has removed the snapshot and also increased the version. It hold the current version of the application.

And in the deploy-test branch it show you new branch on which developers are working on

Opstree SHOA Part 1: Build & Release

At Opstree we have started a new initiative called SHOA, Saturday Hands On Activity. Under this program we pick up a concept, tool or technology and do a hands on activity. At the end of the day whatever we do is followed by a blog or series of blog that we have understood during the day.

Since this is the first Hands On Activity so we are starting with Build & Release.

What we intend to do

Setup Build & Release for project under git repository https://github.com/OpsTree/ContinuousIntegration.

What all we will be doing to achieve it

Finalize a SCM tool that we are going to use puppet/chef/ansible.
Automated setup of Jenkins using SCM tool.
Automated setup of Nexus/Artifactory/Archiva using SCM tool.
Automated setup of Sonar using SCM tool.
Dev Environment setup using SCM tool: Since this is a web app project so our Devw443 environment will have Nginx & tomcat.
QA Environment setup using SCM tool: Since this is a web app project so our QA environment will have Nginx & tomcat.
Creation of various build jobs
- Code Stability Job.
- Code Quality Job.
- Code Coverage Job.
- Functional Test Job on dev environment.
Creation of release Job.
Creation of deployment job to do deployment on Dev & QA environment.

This activity is open for public as well so if you have any suggestion or you want to attend it you are most welcome

Tip : Setting up Git Jenkins integration on windows box

If you have ever tried setting up git as a version control system in a Jenkins installation on a windows box you would have faced an error message ssh key not available.

The reason behind this issue is that if you are using git with ssh protocol it tries to use your private key to perform git operations over ssh protocol & the location it expects is the .ssh folder at home directory of user. To fix this issue you have to create a HOME environment variable and point to your home directory where your .ssh folder exists after that restart Jenkins & now it should work fine.

Automation tips and tricks

As promised I’m back with the summary of cool stuff that I’ve done with my team in Build & Release domain to help us deal with day to day problems in efficient & effective way. As I said this month was about creating tools/utilities that sounds very simple but overall their impact in productivity & agility of build release teams and tech verticals was awesome :).

Automated deployment of Artifacts : If you have ever worked with a set of maven based projects that are interdependent on each other, one of the major problem that you will face in such a setup is to have the latest dependencies in your local system. Here I’m assuming two things you would be using a Maven Repo to host the artifacts & the dependencies would be SNAPSHOT dependencies if their is active development going on dependencies as well. Now the manual way of making sure that maven repo will always have the latest SNAPSHOT version is that every-time somebody does change in the code-base he/she manually deploy that artifact to maven repo. What we have done is that for each & every project we have created a Jenkins job that check if code is checked in for a specific component & if so that component’s SNAPSHOT version get’s deployed to maven repo. The impact of these utilities jobs was huge as now all the developers doesn’t have to focus on deploying their code to maven repo & also keeping track of who last committed the code was also not needed.

Log Parser Utility : We have done further improvement in our event based log analyzer utility. Now we also have a simple log parser utility through which we can parse the logs of a specific component & segregate the logs as per ERROR/WARN/INFO. Most importantly it is integrated with jenkins so you can go to jenkins select a component whose log needs to be analyzed, once analysis is finished the logs are segregated as per our configuration(in our case it is ERROR/WARN/INFO) after that in the left bar these segregations are shown with all the various instances of these categories and user can click on those links to go exactly at the location where that information is present in logs

Auto Code Merge : As I already told we have a team of around 100+ developers & a sprint cycle of 10 days and two sprints overlap each other for 5 days i.e first 5 days for development after tat code freeze is enforced and next 5 days are for bug fixing which means that at a particular point of time there are 3 parallel branches on which work is under progress one branch which is currently deployed in production second branch on which testing is happening and third branch on which active development is happening. You can easily imagine that merging these branches is a task in itself. What we have done is to create an automated code merge utility that tries to merge branches in a per-defined sequence if automatic merge is successful the merge proceeds for next set of branches otherwise a mail is sent to respective developers whose files are in conflict mode

Hope you will get motivated by these set of utilities & come up with new suggestions or point of improvements

Efficiently handling Code merge in Version Control System

One of the painful & mundane task that release engineers have to perform is to merge changes of one branch into another branch & in case of code conflicts the release engineer has to co-ordinate with all the developers to resolve those merge conflicts.

In our current setup the problem is more critical as development of two releases overlap with each other . We have a sprint cycle of 10 days where we have 5 days of active development after that code freeze is implemented & rest 5 days are only for big fixes. The next sprint starts just after the code freeze date of previous release. In ideal scenario this setup should work well but the biggest assumption behind successful execution of the process is their should be minimum code check-ins after code freeze & usually that doesn’t happens. This results in parallel development in 2 branches & therefore while merging two branches their are lot of code conflicts.

The real problem starts when we start merging code, as currently their are close to 100 developers working on the same code-base which means a huge list of files in conflict mode & you have to chase down each & every person to resolve those conflicts. To overcome the above-said problem we are planning to do 2 things.

First one is instead of doing merge after a long duration we are planning to increase the frequency of merge from once in 5 day to twice a day which would help us to reduce the list of conflicting files.

As I always strive to automate things as much as possible, the second part is to at-least create an automated tool that will perform a dummy merge of two branches and list out all the files that would result in conflict mode along with listing the last user’s who have modified the files in respective branch.

We are expecting 60-70% efficiency in code merge process, let’s see how things goes. Feel free to drop any ideas if you have or in case of any concerns :).

Although I tried to be as generic as possible, but just to let you know we are using Git as version control system.