As we promised in our previous blog Prometheus as Scale – Part 1 that in our next blog we will be writing about the implementation part of Cortex with Prometheus, so here we are with our promise. But before going to the implementation part, we would suggest you guys go through our first blog to know the need for it.
Previously we talked that Prometheus is becoming a go-to option for people who want to implement event-based monitoring and alerting. The implementation and management of Prometheus are quite easy. But when we have a large infrastructure to monitor or the infrastructure has started to grow you require to scale monitoring solution as well.
Prometheus has gained a lot of popularity because of its cloud-native approach for monitoring systems. Its popularity has reached a level that people are now giving native support to it, while developing software and applications such as Kubernetes, Envoy, etc. For other applications, there are already exporters(agent) available to monitor it.
Since I have been working on Prometheus for quite a long time and recently have started doing development on it, I was confident that I can handle any kind of scenario in it. Here, in this blog, I am going to discuss a scenario that was a very good learning experience for me.
A while back we got the requirement for working on Apache Druid. By working on Apache Druid, We mean setup, management, and monitoring. Since it was a new topic for us we started evaluating it and we actually find it has a lot of amazing features.
So for the people who don’t have an idea about Druid and just starting with Druid. Let me give a quick walk-through of it.
Elasticsearch is a scalable, full-text search and analytics engine based on Apache Lucene. It is Java-based and allows you to store, search, and analyze big volumes of data quickly in near real-time. It can search and index the document files in diverse formats. Lucene is the underlying technology that Elasticsearch uses for extremely fast data retrieval.
As flexible, scalable, and useful as Elasticsearch is, monitoring your cluster can help you ensure that the cluster is appropriately sized and handles all operations efficiently.