Protected EFK Stack Setup for Kubernetes


In this blog, we will see how we can deploy the Elasticsearch, Fluent-bit, and Kibana (EFK) stack on Kubernetes. EFK stack’s prime objective is to reliably and securely retrieve data from the K8s cluster in any format, as well as to facilitate anytime searching, analyzing, and visualizing of the data.

What is EFK Stack?

EFK stands for Elasticsearch, Fluent bit, and Kibana.

Elasticsearch is a scalable and distributed search engine that is commonly used to store large amounts of log data. It is a NoSQL database. Its primary function is to store and retrieve logs from fluent bit.

Fluent Bit is a logging and metrics processor and forwarder that is extremely fast, lightweight, and highly scalable. Because of its performance-oriented design, it is simple to collect events from various sources and ship them to various destinations without complexity.

Continue reading “Protected EFK Stack Setup for Kubernetes”

On-Premise Setup of Kubernetes Cluster Components (Offline Mode) – PART 2

In the previous blog, we discussed setting up Offline Kubernetes Cluster over on-premises servers. After setting up the Kubernetes cluster we need to have some basic components to manage the orchestration and monitoring of the Kubernetes Cluster which will help Horizontal Pod Autoscaler and Vertical Pod Autoscaler to get information about CPU/Memory. Also, we have to limit access to all the components and Microservice we have set up for the SSO tool.

To begin with, we need a service mesh tool to manage the traffic flow between multiple microservices and We have many tools for this like Istio, Linkerd, Cilium Service Mesh, Consul connect, etc. Here I am considering Istio.

 Firstly, We will be talking Istio Setup over Kubernetes Cluster.

Istio is an open source service mesh that helps organizations
run distributed, microservices-based apps anywhere.
Istio enables organizations to secure, connect, and monitor
microservices, so they can modernize their enterprise apps more
swiftly and securely.
Istio allows organizations to deliver distributed applications at scale. It simplifies service-to-service network operations like traffic management, authorization, and encryption, as well as auditing and observability.

Continue reading “On-Premise Setup of Kubernetes Cluster Components (Offline Mode) – PART 2”

Group-Based Authorization in GitLab

Why Group-Based?

In an organization, there are multiple projects, and every project has multiple users every user has a different role to perform, based on the role whether he is owner, maintainer, developer, reporter, or guest we assign the role to that user, but the main problem is that when we have to use those users to the different project then we have to do all the same task again. There is a better way to manage users in GitLab by creating groups and assigning those groups to the project.

What is GitLab Group?

In GitLab, we use groups to manage one or more related projects at the same time. We can use groups to manage permissions for your projects. If someone has access to the group, they get access to all the projects in the group. We can also view all of the issues and merge requests for the projects in the group, and view analytics that shows the group’s activity. We can also create subgroups in a group.

Continue reading “Group-Based Authorization in GitLab”

Deploying Azure Policy using Terraform Module

While working on Azure, you might come across a requirement that says the resources being deployed should be in accordance with the organization’s policies. Suppose you might want to grant a particular or a set of permissions on the resource group or on the management group so that the owner of it should be restricted like denying deploying of resources by enforcing resource tagging, region enforcement, allowing approved Virtual machines (VM) images, and many more. 

In this blog, we will try to resolve these issues by applying Azure policies. 

First, let’s get familiar with the azure policy.

The azure policy is a service that has been designed to help you enforce different rules and to act based on the rule’s effect on your Azure resources. You can use it to create, assign and manage policies. Azure policy evaluates your resources for non-compliance with assigned policies and performs the assigned effect. 

Continue reading “Deploying Azure Policy using Terraform Module”

Trigger Jenkins Job using AWS Lambda triggered by S3 Event

The problem that we faced:

There was a cron scheduled in our environment at a particular time which used to run at 8AM. This cron read data from the s3 bucket and used to succeed but failed when there were no objects present.

Problem this Integration solves:

  • Eliminates the need of keeping a scheduler in the system. Lambda will work as a Scheduler.
  • No manual intervention will be required in case the file from one service is delayed by a minute or an hour due to any reason. 
  • Data exchange and processing between teams become more efficient and less error-prone.
  • Eliminates the need of checking the presence of files on S3. It all gets automated.

Workflow:

  1. The object is uploaded to S3
  2. S3 event triggers a Lambda function
  3. Lambda starts Jenkins job using Jenkins API
Continue reading “Trigger Jenkins Job using AWS Lambda triggered by S3 Event”