Skip to main content

Automation tips and tricks January 2013

I'm starting a new blog series in which I'll be talking about various cool things or automations that I along with my team done in a month and what are my plans for next month.

Talking about January 2013, I've done following things

1.) Streamlining of environments : The big step in streamlining the environments is to change the owners of the application from root user to tomcat user & making ports of all the application consistent across environments i.e dev, qa, pt & staging. This will help me in my long term goal of introducing a server configuration tool most preferably puppet.
2.) Log Analyzer Utility : One of the major challenge that teams face is to get real time notifications of any exceptions that occur in the server logs, to overcome this problem we have written a log analyzer utility that will scan a log file backed by a meta file, this meta file have the information about who all should be notified for an exception. This utility is written in shell script and integrated with Jenkins CI server so that we can schedule the execution of this utility as per convenience, currently jenkins is executing this utility after every 15 minutes.
3.) System monitor : Off late we were facing challenge of servers getting disk out of space & when whole system goes down then only we were able to figure out the issue is due to disk space outage due to huge log files, to overcome this problem we have built a small shell utility that scans couple of folder's recursively and provide a list of top 10 files whose size is greater then a specified threshold. In our case we have set this threshold as 1 GB, also all these variables can be provided as input to this utility such as folder's to scan regular expression of files which needs to be considered the threshold value

This is what we have achieved in the month of January 2013 although these utilities seems to be but obvious and simple one but the effect they have in the productivity of the team is considerable.


Now plans for the month of February 2013, usually I choose those things which we are doing manually, this month we will be working on following things
1.) Utility which can perform automated merge if possible
2.) Utility that can automatically upload the artifacts to a central server(artifactory in our case)
3.) Integration of git common operations with Jenkins



Comments

  1. Good work!!

    For system monitoring: Look at Zabbix - it is a very good tool. We are using it for all kind of monitoring of our application/database/mail servers. It has capabilities of file system monitoring, application monitoring and web monitoring and support for scripting. There is good book about it - it costs around $8.

    For real time delivery of Exception messages - you can use the SMTP appender supported by log4j/slf4j. We use it for exception from certain packages.

    ReplyDelete
  2. thanks prakash, the idea behind creating these utilities is to fine tune them and control them as per our requirements for example for real time delivery of exception messages we can have different requirement for log file monitoring & different set of people looking into those exceptions

    ReplyDelete

Post a Comment

Popular posts from this blog

EC2 Ssh Connection Refused

When ssh: connect to host ip_address port 22 Connection refused



Unable to access server???
Exactly when you see the error - “ssh: connect to host ip_address port 22: Connection refused” while connecting your AWS EC2 Instance. In order to find solution of the problem, you will go to AWS forum and other channels where you need to answers several questions first. But it's very difficult to find the actual problem. In order to get clues what the problem is, we should provide as many details as possible about what we have tried and the results we are getting. Because there are hundreds of reason why a server or service might not be accessible, also connectivity is one of the toughest issue to diagnose, especially when you are hosting something critical on your box. I've seen several topics on this problem, but none offers a solution to it.  I was not aware for what should I look at first. So I walk through from the very basics and investigated the following thing Use of verbose while ss…

jgit-flow maven plugin to Release Java Application

Introduction As a DevOps I need a smooth way to release the java application, so I compared two maven plugin that are used to release the java application and in the end I found that Jgit-flow plugin is far better than maven-release plugin on the basis of following points: Maven-release plugin creates .backup and release.properties files to your working directory which can be committed mistakenly, when they should not be. jgit-flow maven plugin doesn't create these files or any other file in your working directory.Maven-release plugin create two tags.Maven-release plugin does a build in the prepare goal and a build in the perform goal causing tests to run 2 times but jgit-flow maven plugin builds project once so tests run only once.If something goes wrong during the maven plugin execution, It become very tough to roll it back, on the other hand jgit-flow maven plugin makes all changes into the branch and if you want to roll back just delete that branch.jgit-flow maven plugin doesn…

VPC per envrionvment versus Single VPC for all environments

This blog talks about the two possible ways of hosting your infrastructure in Cloud, though it will be more close to hosting on AWS as it is a real life example but this problem can be applied to any cloud infrastructure set-up. I'm just sharing my thoughts and pros & cons of both approaches but I would love to hear from the people reading this blog about their take as well what do they think.


Before jumping right away into the real talk I would like to give a bit of background on how I come up with this blog, I was working with a client in managing his cloud infrastructure where we had 4 environments dev, QA, Pre Production and Production and each environment had close to 20 instances, apart from applications instances there were some admin instances as well such as Icinga for monitoring, logstash for consolidating logs, Graphite Server to view the logs, VPN server to manage access of people.




At this point we got into a discussion that whether the current infrastructure set-u…