Skip to main content

Puppet module for setting up Multiple mongo's with replication

In this blog I'll be talking about a puppet module, that can be used to installing multiple mongo's with replication on a single machine. Since I'm very new to puppet so you may find this module very crude, but it works :). Their were couple of puppet module already available but most of them are only for installing a single instance of mongo at a machine & I've a specific requirement of installing multiple instances of mongo having master slave replication between them. As I already said that this module may be quiet crude or basic so please bear with that & my approach may also seem a bit unconventional so please let me know what all can be improved in this module or how things could have been done in a better way.

So let's start with the actual details first of this module is hosted on github(https://github.com/sandy724/Mongo), if you want to look at the source code you can clone it from github. For installing mongo you would be executing the command
puppet apply -e "class {mongo:port => , replSet => ,master => ,master_port => ,}"

Command for installing master
puppet apply -e "class {mongo:port => 27017, replSet => sdrepsetcommon,master => master, master_port => 27017,}"

Command for installing slave
puppet apply -e "class {mongo:port => 27018, replSet => sdrepsetcommon,master => slave,master_port => 27017,}"

Before going into the details what all this module is doing I will share some details of mongo
  • You can start mongo by executing mongod command
  • You can provide a configration file which contains details such as
    • log directory where mongo would be generating the logs
    • port at which mongo would be listening for requests
    • dbpath where mongo would be storing all the data
    • pidfilepath containg process id of mongo instance, that would be used to check whether mongo is running or not
    • replSet name of the replicaset
  • You need to have a mongo as a service installed in you system to start an instance of mongo
  • For replication you need to execute rs.initiate command on the master mongo
  • For adding another instance into replication you need to execute rs.add(":")  command on the master mongo
   
Now let's go into more details what all this component does, I'll be listing down all the steps in bulleted points
  • As you can figure out this module is expecting few parameters :
    • port : port at which mongo would be listening,
    • replSet : name of replicaset which would be used for managing replication
    • master : A string parameter which would signify whether the mongo setup is for master or slave
    • master_port : Port at which master instance of mongo would be listening
  • First of all we create a mongo user
  • Parent Log directory for the mongo instance is created if it doesn't exists with mongo user as owner.
  • Mongo db directory is created under /data/mongo with a naming convention replSet_port, i.e if replSet parameter is sdrepsetcommon & port is 27017 then the data directory for this mongo instance would be  /data/mongo/sdrepsetcommon_27017. This directory would be owned by mongo user.
  • A mongo service would be installed if not already their.
  • A mongo restart shell script is also placed at the mongo db directory
  • A file is also placed under the mongo db directory that have a mongo command to setup replication, this file is created conditionally depending on whether we are setting up a master or slave instance.
  • Finally the replication command is executed on mongo server & restart script is also executed

This concludes the setting up of a mongo instance on a machine.

Just for more details to start mongo we are using mongod -f command, this configuration file is saved as a template & the mongo modules processes the template with the values passed & creates the desired mongod.conf. In our case we are evaluating following properties of mongod.conf : logpath, port, dbpath, pidfilepath, replSet

Comments

Post a Comment

Popular posts from this blog

EC2 Ssh Connection Refused

When ssh: connect to host ip_address port 22 Connection refused



Unable to access server???
Exactly when you see the error - “ssh: connect to host ip_address port 22: Connection refused” while connecting your AWS EC2 Instance. In order to find solution of the problem, you will go to AWS forum and other channels where you need to answers several questions first. But it's very difficult to find the actual problem. In order to get clues what the problem is, we should provide as many details as possible about what we have tried and the results we are getting. Because there are hundreds of reason why a server or service might not be accessible, also connectivity is one of the toughest issue to diagnose, especially when you are hosting something critical on your box. I've seen several topics on this problem, but none offers a solution to it.  I was not aware for what should I look at first. So I walk through from the very basics and investigated the following thing Use of verbose while ss…

jgit-flow maven plugin to Release Java Application

Introduction As a DevOps I need a smooth way to release the java application, so I compared two maven plugin that are used to release the java application and in the end I found that Jgit-flow plugin is far better than maven-release plugin on the basis of following points: Maven-release plugin creates .backup and release.properties files to your working directory which can be committed mistakenly, when they should not be. jgit-flow maven plugin doesn't create these files or any other file in your working directory.Maven-release plugin create two tags.Maven-release plugin does a build in the prepare goal and a build in the perform goal causing tests to run 2 times but jgit-flow maven plugin builds project once so tests run only once.If something goes wrong during the maven plugin execution, It become very tough to roll it back, on the other hand jgit-flow maven plugin makes all changes into the branch and if you want to roll back just delete that branch.jgit-flow maven plugin doesn…

VPC per envrionvment versus Single VPC for all environments

This blog talks about the two possible ways of hosting your infrastructure in Cloud, though it will be more close to hosting on AWS as it is a real life example but this problem can be applied to any cloud infrastructure set-up. I'm just sharing my thoughts and pros & cons of both approaches but I would love to hear from the people reading this blog about their take as well what do they think.


Before jumping right away into the real talk I would like to give a bit of background on how I come up with this blog, I was working with a client in managing his cloud infrastructure where we had 4 environments dev, QA, Pre Production and Production and each environment had close to 20 instances, apart from applications instances there were some admin instances as well such as Icinga for monitoring, logstash for consolidating logs, Graphite Server to view the logs, VPN server to manage access of people.




At this point we got into a discussion that whether the current infrastructure set-u…