DevOps – Page 55 – DEVOPS DONE RIGHT

How to securely access your private app on cloud

One of the suggested practices in cloud administration is to always host your applications on a Virtual Private Cloud. Also, you should have a public subnet hosting the public facing apps, and a private subnet which hosts the private apps (like a database or a back-end service/app). To know more about why you need such kind of a setup, please read more about VPC.

This blog will talk about a scenario where you have multiple Virtual Private Clouds (hereafter referred to as VPC), and you need to access a private app hosted in one VPC from another VPC. An example of this scenario could be that you have a VPC for your staging environment and another VPC for production environment, then you’d like to sync the database from of production environment from the staging environment. In this case, it might not be straight forward to do this, as you might not be able to access the production database from outside the production VPC.

One of the solutions for this problem would be to first take a dump of the production database on one of the public facing machines in the production VPC, and then copy that dump to a public facing machine in the Staging VPC and finally applying this dump to the private database of Staging environment. This approach will work, but it would not be a perfect solution, as you have to copy the db dump between VPC’s.

A much better approach would be if you could directly connect to the production database from the Staging VPC & execute the dump & restore command, for that you need direct access of production database from staging environment. This approach is called port-forwarding. We configure port-forwarding at one of the public facing machines(NAT is the preferred one) in the production VPC in such a manner that if a request comes on this machine at port x it will be forwarded to port y on a private facing machine in the production VPC which is the database production in this case.

In the next blog I will talk about other alternate approaches that can be used to solve this problem.

Puppet module to setup nodejs deployment 2

As I said in the previous blog Puppet module to setup nodejs deployment, the nodejs module was for providing the basic infrastructure for automated node app’s deployment & as promised I’ve released the next module “nodeapp” that can be used to setup a node app on the target server.

First of all I’ll talk about what this module will do to facilitate the automated deployment of a nodejs app, as already discussed we are following a convention that all the node app’s code will be present at /home/nodejs/ which is referred by startNodeApp.sh script so we create the directory of nodejs app. The deployNodeApp.sh script was using the upstart to manage the nodejs app instance i.e starting/stoppping the nodejs app, the nodeapp module takes care of creating the require upstart configuration at /etc/init/.conf. Also we use monit to monitor the nodejs app’s so that we can start/stop the nodejs app’s using the web ui of monit & also see various stats such as cpu, memory, load.. consumption of nodejs app.

This nodeapp module is a userdefined type which takes the name of nodeapp as an argument, as a result of which you can setup any number of nodejs app’s on a system.
i.e nodeapp{‘search-demo’: app_name => “search-demo”}
This entry will create below files

/etc/init/search-demo.conf : An upstart configuration file, using which search demo nodejs app can be managed as a service.

#!upstart
description “node.js search-demo server”
author      “sandy”

start on startup
stop on shutdown

script
    export HOME=”/home/nodejs”

    echo $$ > /var/run/search-demo.pid
    exec sudo -u nodejs /home/nodejs/startNodeApp.sh search-demo
end script

pre-start script
    # Date format same as (new Date()).toISOString() for consistency
    echo “[`date -u +%Y-%m-%dT%T.%3NZ`] (sys) Starting” >> /var/log/search-demo.sys.log
end script

pre-stop script
rm /var/run/search-demo.pid
echo “[`date -u +%Y-%m-%dT%T.%3NZ`] (sys) Stopping” >> /var/log/search-demo.sys.log
end script

/etc/monit/conf.d/search-demo.monit : A monit configuration file, using which search-demo nodejs app can be monitored & even automatedly restarted

check process search-demo with pidfile /var/run/search-demo.pid
stop program = “/sbin/stop search-demo”
start program = “/sbin/start search-demo”

So using these 2 modules nodejs & nodeapp you can make any system up & running for nodejs app’s automated deployment

Puppet module to setup nodejs deployment

I would like to share my puppet module to setup nodejs deployment infrastructure on a linux box. This module performs the basic setup required to facilitate the automated deployment of a nodejs app. Very soon I’ll be introducing another generic puppet module that will run on top of this module & provide a full fledged automatic deployment of any node app. To view the source code of this module you can refer my github repository.

Let’s talk about what this module actually does. First of all we create a nodejs user which we will use for all deployment related activities of all the node app’s, as a convention we have created a folder /home/nodejs/nodeapps this folder will contain all the code of our node applications.

This modules adds 2 scripts as well the first one is deployNodeApp.sh, deployNodeApp.sh is a generic script that assumes that node app code will be present in tar form at /home/nodejs it will clean existing code of nodeapp at /home/nodejs/nodeapps untar the code at corresponding directory of node app & restart the node app. As another convention we are using upstart for managing the node app i.e starting & stopping the node app I’ll talk about the upstart configuration in my next blog where I’ll talk about generic puppet module for a node app. Another script startNodeApp.sh will take care of starting the node app after doing some per-processing such as loading some environment specific properties of node app which we don’t want to commit in the codebase i.e want to separate it out from deployment process choosing a specific version of node.

This module also takes cares of installing nvm for nodejs user so that nodejs version can be managed locally for this user or app.

Though we already have a puppet module for nodejs, but I had some specific requirements which I wanted to handle that’s why I’ve created this module.

Let me know if you have some points of improvement in this module, one thing that I wanted to add in this module is to add npm installation but it had some other dependencies also I had some doubts whether I should have npm as part of nodejs module or not.

System Monitoring

One of the main task of a system administrator is system monitoring, system monitoring usually involves monitoring the ram & disk space usage of the system …. In this blog I’ll be talking about my experience as a system admin & how I do it.

Usually system monitoring is divided into 2 parts Continuous system monitoring and troubleshooting system issues when system crosses a threshold value & you have to figure out the issue & try to resolve it.

In continuous system monitoring a system is put under continuous monitoring i.e the system ram usage is within defined limit or not, the disk space occupancy should not cross a predefined threshold …. To achieve continuous monitoring you can use couple of tools available in market such as nagios, omd we are primarily using these tools their would be other tools available also for this purpose.

Continuous system monitoring serves one purpose where they notify about any deviation from the expected state of the system, the next step is to troubleshoot this issue & resolve it accordingly. As a first step I usually execute top command, top is a very powerful command apart from just viewing the processes activity in real you can do a lot of things i.e

If you want to add/remove fields : press f & then you can choose the fields to add/remove
If you want to change ordering of fields : press O & then you can move fields
If you want to change the sort order : press F or O

there are lot of other options available as well, if you want to explore them pressing h will provide you a list of all the options.

You can also read about htop, htop is an advanced form of top where you can view some graphs as well though I’ven’t used htop so much but I’m planning to 🙂

One thing to note sometimes you are not able ot run top command due to high resource utilization, in that case you have to use cat /proc/loadavg to view the load on the system & cat /proc/meminfo to view current memory state of the system.

One of the useful command if top doesn’t work
ps -eo pmem,pcpu,vsize,pid,cmd | sort -k 1 -nr | head -5
This command will give the top 5 processes by memory usage.

Also there are couple of other commands that you can use
free : To view the memory usage of system
df : To view the file system information
du : To view the disk usage

One tip : To increase the memory of system you can create a swap memory & it is always recommended to create a swap on a partition only. Another best practice for swap area is if your system RAM is below 8 gb your swap area should be double of your ram otherwise it should be half of your RAM size

Tip : Setting up Git Jenkins integration on windows box

If you have ever tried setting up git as a version control system in a Jenkins installation on a windows box you would have faced an error message ssh key not available.

The reason behind this issue is that if you are using git with ssh protocol it tries to use your private key to perform git operations over ssh protocol & the location it expects is the .ssh folder at home directory of user. To fix this issue you have to create a HOME environment variable and point to your home directory where your .ssh folder exists after that restart Jenkins & now it should work fine.