Needless to say an automated system, no matter how big or small, must be designed with scale in mind. We’ll talk about laying foundation for a robust and malleable setup which is a useful read for everyone.
In my experience as DevOps and SRE, I’ve enjoyed quite a lot of things this profession offered. From the satisfaction of fulfilled curiosities to the anxiety of unforeseen mishaps, it delivered one day after the other. The nervousness in the face of new challenges, happiness on receiving appreciation, thrill during troubleshooting, the pride after a successful implementation, and a lot more. But the one I found myself seeking was boredom. Yes, plain old silence where no surprises are met, everything runs exactly as it should and you enjoy listening to dropping pins. This is especially true when the system in question is your own design. There’s no greater sentiment. It is like watching abird that you’ve freed soar.
In our day-to-day lives, we take a lot of things for granted. Our body, our friends, water, the food we eat, clothes we wear, IP address of our host, FQDN of our favorite websites, etc. In our ignorance, we forget to consider how our lives would be without them. Let’s discuss the leisure of friends, food, etc sometime later. For now, we’ll limit our thoughts to how DHCP and DNS are managed in AWS, both of which are the backbone to connections among hosts and servers over a network.
HAProxy is one of the most frequently used and efficient tools out there for load-balancing. It is highly configurable and can handle almost all of one’s needs to set up a HA, scalable infrastructure in both, HTTP and TCP. Its clientele is a testament to that as it is used and recommended by various heavy-hitters in the industry like Airbnb, Github, instagram, reddit, etc.
Right off the bat, I want to say that, this blog does not cover installing and configuring ElastAlert in the usual sense, i.e. working with pre-existing rules. It helps, I hope, in understanding the requirements for adding one’s own rule. Continue reading “Make Your Own Rules, ElastAlert Style”
Imagine you wake up one day, open your front door, and find huge wooden logs of different shapes and sizes placed in your front yard. On one of them is a note that says, “Well, here have ’em, all the logs, both structured and unstructured. Please deliver all the furniture by the end of the week or else … Now, if you’re inexperienced in “woodwork”, you need to start thinking. This is, more or less, how I felt when I was first assigned a task like this.
At the time, I faced a fair amount of difficulties bringing the logs from servers to visualize in kibana. The area which was most troubling and rewarding was figuring out logstash. This blog is a helpful summary of what I learned about logstash and how it helped. So, Ladies, Gentlemen, and others, sit tight and read on.