Elasticsearch Backup and Restore in Production

ES backup and restore using AWS S3

We were fortunate enough to get an opportunity to do an Elasticsearch cluster snapshot and restore on a production highly active cluster. The indices we needed to restore were around 2 – 3 TB in size.

Our task was to take a snapshot from an old cluster (v 6.4.2) which had several huge indices and restore a few of them to a new cluster (v7.9.2). This endeavour was supposed to bring the load down from the old cluster.  

Continue reading “Elasticsearch Backup and Restore in Production”

Elastic SIEM – An Event Tracking Feature

SIEM with ELK – san3ncrypt3d – Making cybersecurity a habit & Privacy a Goal

 

Torture the data, and it will confess to anything.

Ronald Coase

WHAT IS ELASTIC SIEM

Elastic SIEM (Security Information and Event Management) is a new feature provided by Elastic NV. Using Elastic SIEM we can track and maintain important events that concern us.

Events are actions that reflect something that has happened.

Continue reading “Elastic SIEM – An Event Tracking Feature”

Elasticsearch Cluster Monitoring

Elasticsearch is a scalable, full-text search and analytics engine based on Apache Lucene. It is Java-based and allows you to store, search, and analyze big volumes of data quickly in near real-time. It can search and index the document files in diverse formats. Lucene is the underlying technology that Elasticsearch uses for extremely fast data retrieval.

As flexible, scalable, and useful as Elasticsearch is, monitoring your cluster can help you ensure that the cluster is appropriately sized and handles all operations efficiently.

Continue reading “Elasticsearch Cluster Monitoring”

Elasticsearch Garbage Collector Frequent Execution Issue

Have you noticed an unexpected unallocation of Shards happening at a duration of 1 hour resulting in Cluster state switching from Green > Yellow > Red > Yellow > Green?. During this transition, ES becomes unreachable and the API calls start responding with non 200 code.

Image for post

Environment Continue reading “Elasticsearch Garbage Collector Frequent Execution Issue”

Make Your Own Rules, ElastAlert Style

 

Right off the bat, I want to say that, this blog does not cover installing and configuring ElastAlert in the usual sense, i.e. working with pre-existing rules. It helps, I hope, in understanding the requirements for adding one’s own rule. Continue reading “Make Your Own Rules, ElastAlert Style”