Monday, August 8, 2016

Stunnel a Proxy to ship the log on SSL


Few days ago I got a task to create the SSL connection with logstash redis plug-in with Azure Redis. As we are shipping the logs form the several data center to the Azure Redis. So logs must be shipped on SSL connection. There is no provision to create SSL connection through logstash redis plug-in thats why logstash redis plug-in is not able to make SSL connection with Azure redis.

To resolve this problem we have to setup the stunnel as proxy front of the logstash redis plug-in. Stunnel can create SSL connection with Azure redis and Stunnel provide non-ssl connection for the logstash redis plug-in.
Azure redis provide two type of connections, SSL on 6380 port and non SSL on 6379 port and also provide primary and secondary key(password).


Install Stunnel on the ubuntu. 
$ sudo apt-get install stunnel  


Create a configuration for stunnel /etc/stunnel/stunnel.conf and put the following lines into the configure file.
setuid = root
setgid = root 
pid = /var/run/ 
debug = 7 
output = /var/log/stunnel4/azureredis.log 
options = NO_SSLv2 
options = NO_SSLv3 
  TIMEOUTidle = 30 

Restart the stunnel4
$ service stunnel4 restart

Install redis-cli to test the connection try to ping the redis azure it should reply as PONG
$ redis-cli -a <AZURE_REDIS_PASSWORD>> ping 

Now you can make SSL connection with azure redis on SSL.

Thursday, April 21, 2016

Kitchen Chef's diagnosis center


This time we familiarize you with the backings of chef kitchen. Chef kitchen provides you facility of trial and verification of your cookbooks over different  environment so that you can confidently use  them on your targeted infrastructure. This may be getting boring some times but here is always some seeds in oranges.  


This blog requires an initial information about Git and Vagrant. This blog uses centos7   as platform. It needs basic understanding of chef, it's cookbooks and chef kitchen. To know about chef cookbooks and work with  chef kitchen follow our previous blogs of this series Chef Start here with ease...

Setup Kitchen

To setup kitchen chase same procedure as we cater in our previous blog Chef-Kitchen  Do it simply .

Backings of  Kitchen

Chef kitchen is the diagnosis center of chef.  Here  you can test the authenticity of your cookbooks on different platforms and confidently use them on your infra. Kitchen achive this by using its configuration file where you define all the things which are necessary to run the complete cycle of kitchen.


This file contains all the required stuff to  run chef kitchen. This file is divided into four major sections.

  • Driver: This is  the name of a driver that will be used to create platform instances used during cookbook testing. This is the default driver used for all platforms and suites unless a platform or suite specifies a driver to override the default driver for that platform or suite; a driver specified for a suite will override a driver set for a platform.

  • Provisioner: This specifies how the chef-client will be simulated during testing. chef_zero andchef_solo are the most common provisioners used for testing cookbooks

  • Platforms: This is a the name of a platform on which Kitchen will perform cookbook testing, for example,ubuntu-12.04 or centos-6.4; depending on the platform.
  • Suites: This is a collection of test suites, with each suite_name grouping defining an aspect of a cookbook to be tested.

.kitchen directory

This directory holds logs for every kitchen run for each platform entry in .kitchen.yml file. This folder also holds keypair for the ssh into your virtual environment.

Analysis of Kitchen

Kitchen performs its own complete cycle of testing via different phases.  Kitchen has its five phases i.e., create, converge, login, verify and destroy.  Each phase has its own significance and some specific task is gonna performed in every phase.


In this step chef’s kitchen tool creates a virtual instance. This virtual instance could use cloud or any other virtualization technology. Chef supports cookbook testing across many cloud providers and virtualization technologies.


This phase is responsible for application of your cookbook on virtual  instance. Here your all  cookbooks deployed into the virtual instance, though in next steps you can verify the complete functioning of your cookbooks.


This step creates a ssh session into  your machine and provide you a login into it. So that you can run your test to verify the proper functioning of your cookbook.


In this  step you manually perform all checks so that you can certify the authenticity of your cookbook over all platforms.


This is the final step  of your kitchen testing cycle. Here you destroy your virtual environment after entire testing phases.

Here is also a combine command for all these phases i.e.,  kitchen test. This command club all the commands in below listed manner.
  • Kitchen destroy
  • Kitchen create
  • Kitchen converge
  • Kitchen verify

Hass finally this over!!  I know you too get frustrated with the theories. We now aware with the backings of Chef Kitchen.  

Be Warned: I Am Bored. This Could Get Dangerous.


Thursday, April 14, 2016

Chef-Kitchen Do it simply..

If you are from agile background you would be already aware about the importance of testing your code as you develop. As DevOps SCM tools are maturing where we try to have our complete infrastructure as code it becomes a dire necessity to have our infrastructure code to be tested as well. In this blog I'll showcase how I'm using the testing capability of Chef to test our Chef code.


This blog assumes that you have a basic understanding of Chef, Docker, Git and Vagrant. This blog is written in consideration with centos as platform. You can follow same procedure for ubuntu with some basic changes.

Setup Kitchen
Clone from our github repository to spin up a vagrant with kitchen and other chef tools.

  • Go to testkitchen directory.  

$ cd  Chef/centos/testkitchen

This directory includes a Vagrantfile, a cookbook and a knife.rb file.

  • View vagrantfile for provisioning.
This Vagrantfile have some basic vagrant provisioning with shell.
$ cat Vagrantfile

This file provision your vagrant using shell.

As this block contains a lot of stuff. FIrstly this need two rpm files to be placed in current directory (i.e. vagrant/chef/centos/testkitchen). Download these rpm using following commands. This vagrantfile uses centos/7 as base box.

$ wget

Next in this block we are installing docker and rvm to manage our ruby version to 2.2.4.

  • Spin Up Vagrant box
$ vagrant up

This will perform everything for you. And creates a running vagrant with centos7 platform. Next login into this vagrant box using following command.

$ vagrant  ssh

Your testing environment is ready and you are all set to use kitchen to test and verify your cookbooks in different platforms using docker container.  

Test your first cookbook  

Change directory to /vagrant. /vagrant directory is shared between host and guest machine.

$ cd /vagrant

This cloned repo has its first cookbook “helloworld” in cookbooks folder. This cookbook creates a directory and place a hello.txt file in it.

Go to /vagrant folder and start using kitchen.
  • Initialize kitchen

$ kitchen init --driver=docker

This command initialize mandatory stuff to start with your kitchen. This creates  .kitchen.yml, chefignore file and a test folder.
.kitchen.yml file is the main configuration file for kitchen. This file contains mainly 4 blocks driver, provisioner, platforms and suites.
    • Driver:  This is the name of driver that is used to create new node for testing
    • Provisioner: This is the chef provisioner to simulate test, chef-zero and chef-solo are the options.
    • Platforms: This is the type of platform on which kitchen runs test eg. ubuntu-14.04 and centos-7.1
    • Suites: This is the collection of test suites with runlist and attributes.

Chefignore file determine which files to ignore when uploading cookbooks.
Test folder contains all test files running over the cookbooks.

  • Install required gems
Install required gems by using bundle. As your directory contains a Gemfile where gems are described with their specific version.  
$ bundle install

Then to verify run following command.

$ gem list kitchen
As this install two essential gems kitchen-docker and test-kitchen.

  • List all  your nodes.
$ bundle exec kitchen list
This command list all your nodes with last action performed on them.

Start kitchen cycle

  • Create your node using following command
$ bundle exec kitchen create

    • This command pulls image of docker if they are not present locally.
    • Install  openssh-server and other essential packages in newly created docker container.
if get “Kitchen is finished.” as final output. Now verify with kitchen list once again.

  • Converge your node
This step deploys your cookbook to newly created node. So let's tell kitchen which cookbook to run. Edit .kitchen.yml file and add recipe in runlist attribute of suits block.
Now run following command to converge your node.

$ bundle exec kitchen converge

    • Chef solo and dna.json got prepared and pushed into node /tmp/kitchen directory
    • This will put cookbook into the node and install chef by performing an Omnibus package installation.
    • Chef run initiated with information in .kitchen.yml file.
Verify again with kitchen list.

  • Login into node
$ bundle exec kitchen login  default-ubuntu-1404
$ bundle exec kitchen login  default-centos
    • Login into node and manually verify /data directory and hello.txt file in it.

  • Verify your cookbook
Look into test directory structure.

As this directory contains “default” folder this is corresponding to your suit name i.e. default. Now create a directory structure to put files in particular location.

    • Use Bats to create test

$ mkdir -p test/integration/default/bats

Busser is the component which provides facility of testing in your node. Bats directory tells kitchen to which Busser runner plugin needs to be installed on the remote instance. In other words the bats directory name will cause Busser to installbusser-bats from RubyGems.

      • Write first test file.
$ vim  test/integration/default/bats/hellotest.bats

#!/usr/bin/env bats

@test "/data directory found in path" {
 run stat /data
 [ "$status" -eq 0 ]

@test "hello.txt file found in path" {
 run stat /data/hello.txt
 [ "$status" -eq 0 ]
      • Run kitchen verify

$ bundle exec kitchen verify

Now verify again with kitchen list, and see last action.

    • Use serverspec to create test
$ mkdir -p test/integration/default/serverspec

      • Write a test
$ vim  test/integration/default/serverspec/hellotest_spec.rb

require 'serverspec'

# Required by serverspec

set :backend, :exec

describe "file hello.txt status" do
 it "file hello.txt" do
   expect(file("/data/hello.txt")).to exist

      • Run kitchen verify

$ bundle exec kitchen verify

Now verify again with kitchen list, and see last action.

  • Destroy your node
$ bundle exec kitchen destroy

This will destroy your node.

Kitchen test

$ bundle exec kitchen test

Kitchen test is an option to perform complete kitchen cycle with one command. This destroy existed nodes and then create, converge, verify node with your predefined suit in .kitchen.yml. And finally it destroy the node.

Now your cookbook is tested. So feel confident to use it.