Saturday, September 16, 2017

EC2 Ssh Connection Refused

When ssh: connect to host ip_address port 22 Connection refused




Unable to access server???
Exactly when you see the error - “ssh: connect to host ip_address port 22: Connection refused” while connecting your AWS EC2 Instance. In order to find solution of the problem, you will go to AWS forum and other channels where you need to answers several questions first. But it's very difficult to find the actual problem.
In order to get clues what the problem is, we should provide as many details as possible about what we have tried and the results we are getting. Because there are hundreds of reason why a server or service might not be accessible, also connectivity is one of the toughest issue to diagnose, especially when you are hosting something critical on your box.
I've seen several topics on this problem, but none offers a solution to it.  I was not aware for what should I look at first. So I walk through from the very basics and investigated the following thing
Use of verbose while ssh
    $ ssh -vvv user@x.x.x.x
This didn’t help me as I haven't found any meaningful information except connection refused.
  • After that I looked for my security groups, well they haven’t provide me any  hint for further steps.
  • Then I tried telnet at port 22 from my public and private network which was again a hard luck for me.
    $ telnet X.X.X.X 22
  • Tried creating AMI and building new instance of it.
  • I've mounted the EBS of a broken instance on a running instance, look for the file configuration of my ssh.
           $ cat /etc/ssh/sshd_config
          and compare that with running instance.
  • Also checked for the entries in /etc/fstab, but entries were all perfect as per knowledge.
  • Tried starting the instance from the broken instance, but again the same error occured on the screen.
Coming to AWS UI console :-
  • Further moved over the AWS UI, under Action I found option to put user data
action.png

So below entry were made
#cloud-config:
snappy:
ssh_enabled: True


  • I had gone through different option in UI , just went through the system logs
    action-4.png

          And found that the issue is with swap, which is showing error while mounting.
  • So I stopped the broken instance and mount the broken ebs volume to the running one and commented the  swap entry from /etc/fstab
fstab2.png
  • Finally I found that my instance is up and running, again I looked for the system logs in aws UI, where login was prompt was able to access my instance again.

Conclusion :-
If you come across any such error then follow the AWS console of the machine & look for the issue and get to the core of the problem.

Monday, July 24, 2017

Logstash Timestamp

Introduction

A few days back I encountered with a simple but painful issue. I am using ELK to parse my application logs  and generate some meaningful views. Here I met with an issue which is, logstash inserts my logs into elasticsearch as per the current timestamp, instead of the actual time of log generation.

This creates a mess to generate graphs with correct time value on Kibana.
So I had a dig around this and found a way to overcome this concern. I made some changes in my logstash configuration to replace default time-stamp of logstash with the actual timestamp of my logs.

Logstash Filter

Add following piece of code in your  filter plugin section of logstash's configuration file, and it will make logstash to insert logs into elasticsearch with the actual timestamp of your logs, besides the timestamp of logstash (current timestamp).

date {
  locale => "en"
  timezone => "GMT"
  match => [ "timestamp", "yyyy-mm-dd HH:mm:ss +0000" ]
}



In my case, the timezone was GMT  for my logs. You need to change these entries  "yyyy-mm-dd HH:mm:ss +0000"  with the corresponding to the regex for actual timestamp of your logs.

Description

Date plugin will override the logstash's timestamp with the timestamp of your logs. Now you can easily adjust timezone in kibana and it will show your logs on correct time.

(Note: Kibana adjust UTC time with you bowser's timezone)

Friday, February 3, 2017

Classless Inter Domain Routing Made Easy (Cont..)

Introduction :

As we had a discussion  about Ip addresses and their classes in the previous blog,we can now start with Sub-netting.

Network Mask /Subnet Mask -

As mask means to cover something,
IP Address is made up of two components, One is the network address and the other is the host address.The Ip Address needs to be separated into the network and host address, and this separation of network and host address in done by Subnet Mask.The host part of an IP Address is further divided into subnet and host address if more subnetworks are needed and this can be done by subnetting. It is called as a subnet mask or Network mask as it is used to identify network address of an IP address by performing a bitwise AND operation on the netmask.

Subnet Mask is of 32 Bit and is used to divide the network address and host addresses of an IP.
In a Subnet Mask all the network bits are set to 1’s and all the host bits are set to 0’s.
 
Whenever we see an IP Address - We can easily Identify that
WHAT IS NETWORK PART OF THAT IP
WHAT IS THE HOST PART OF THAT IP
FORMAT :
mmmmmmmm.mmmmmmmm.mmmmmmmm.mmmmmmmm
(Either it will have 1 or 0 Continuously)
EXAMPLE :
A Class Network Mask
In Binary : 11111111.00000000.00000000.00000000         - First 8 Bits will be Fixed
In Decimal : 255.0.0.0
Let the IP Given is - 10.10.10.10
When we try to Identify it we know that it belong to class A, So the subnet mask will be : 255.0.0.0
And the Network Address will be : 10.0.0.0

B Class Network Mask  
In Binary : 11111111.11111111.00000000.00000000           - First 16 Bits will be Fixed
In Decimal : 255.255.0.0
Let the IP Given is -150.150.150.150
When we try to Identify it we know that it belong to class B, So the subnet mask will be : 255.255.0.0
And the Network Address will be : 150.150.0.0

C Class Network Mask  
In Binary : 11111111.111111111.11111111.00000000           - First 32 Bits will be Fixed
In Decimal : 255.255.255.0
Let the IP Given is - 200.10.10.10
When we try to Identify it we know that it belong to class C, So the subnet mask will be : 255.255.255.0
And the Network Address will be : 200.10.10.0

Subnetting :

The method of dividing a network into two or more networks is called subnetting.
A subnetwork, or subnet, is a logically subdivision of an IP network
Subnetting provides Better Security
Smaller collision and Broadcast  Domains
Greater administrative control of each network.
Subnetting - WHY ??
Answer : Shortage of IP Addresses
SOLUTIONS : -
1) SUBNETTING - To divide Bigger network into the smaller networks and to reduce the wastage
2) NAT -  Network Address Translation
3) Classless IP Addressing -
No Bits are reserved for Network and Host

**Now the Problem that came is how to Identify the Class of IP Address :**
Let a IP Be : 10.10.10.10
If we talk about classful we can say it is of class A But in classless : We can check it through subnetwork mask.
255.255.255.0
So by this we can say that first 24 bits are masked for network and the left 8 are for host.
Bits Borrowed from Host and added to Network
Network ID(N)
Network ID(N)
Host ID(H)
Host ID(H)

Network ID(N)
Network ID(N)
Subnet
Host ID(H)

Network ID(N)
Network ID(N)
Subnet
Subnet/Host

Let we have a
150.150.0.0 - Class Identifier/Network Address
150.150.2.4 - Host Address - IP GIVEN TO A HOST
255.255.255.0 - Subnet Mask
150.150.2.0 - Subnet Address

CIDR : Classless Inter Domain Routing


CIDR (Classless Inter-Domain Routing, sometimes called supernetting) is a way to allow more flexible allocation of Internet Protocol addresses than was possible with the original system of IP Address classes. As a result, the number of available Internet addresses was greatly increased, which along with widespread use of network address translation, has significantly extended the useful life of IPv4.


Let a IP be - 200.200.200.200
Network ID(N)
Host ID(H)
--------24 Bit -------- -------8 bit -----------
   
Network Mask tells that the number of 1’s are Masked
Here First 24 Bits are Masked
In Decimal : 255.255.255.0
In Binary : 11111111.11111111.11111111.00000000
   Here the total Number of 1’s : 24
So we can say that 24 Bits are masked.

This method of Writing the network mask can be represented in one more way
And that representation is called as CIDR METHOD/CIDR NOTATION

CIDR  - 200.200.200.200/24
24 : Is the Number of Ones - Or we can say Bits Masked
Basically the method ISP’s(Internet Service Provider)use to  allocate an amount of addresses to a company, a home

EX :
190.10.20.30/28 : Here 28 Bits are Masked that represents the Network and the remaining 4 bits represent the Host
/ - Represents how many bits are turned on (1s)

CLASS C SUBNETTING :

Determining Available Host Address :

<-----------------------NETWORK-----------------------------------><--------HOST----------->
200
10
20
0
11001000               00001010               00010100                 00000000 - 1
                                                                                              00000001 - 2     
                                      00000011 - 3
                                                                          .
                                                                                                    .
                                                                                                    .
                                                                                              11111101 - 254
                                                                                              11111110 - 255
                                                                                              11111111 - 256     
                                                                                                                    -2
                                                                                                               ---------
                                                                                                                   254

    2^N - 2  = 2^8 -2 = 254
           (Coz we have 8 bits in this case)               - 2 (Because 2 Address are Reserved)

254 Address are available here

FORMULAS :
Number of Subnets : ( 2^x ) - 2     (x : Number of Bits Borrowed)
Number of Hosts : ( 2^y ) - 2         (y : Number of Zero’s)
Magic Number or Block Size = Total Number of Address : 256 - Mask

Let a IP ADDRESS BE 200.10.20.20/24
Number of subnets : 5

Network Address   :

200
10
20
20

255
255
255
0
(as total Number of 1’s : 24)
IP in Binary
11001000
00001010
00010100
00010100
MASK
11111111
11111111
11111111
00000000

And Operation in IP And Mask
11001000
00001010
00010100
00000000

In Binary
200
10
20
0

As we need 5 Subnets :
2^n -2 => 5
So the value of n = 3 that satisfies the condition
So, We need to turn 3 Zero’s to One’s to create 5 subnets
200
10
20
0

11001000
00001010
00010100
00000000
11001000
00001010
00010100
11100000
 (3 Zero’s changed to 3 one’s)    
200
10
20
224
                                                                                  


Subnet 0   
200
10
20
0/27  

Subnet 1                                           +32 - Block Size
200
10
20
32/27

Subnet 2                                            +32
200
10
20
64/27

Subnet 3
200
10
20
96/27

Subnet 4
200
10
20
128/27

Subnet 5   
200
10
20
160/27

Subnet 6
200
10
20
192/27

Subnet 7
200
10
20
224/27


How to Put Host ADD.
Subnet 0   
200
10
20
0/27  
Subnet Broadcast Number 0
200
10
20
31 /27  
Subnet 1                                           +32 - Block Size
200
10
20
31/27

200
10
20
32/27

200
10
20
33/27
                                                          .
                                                          .
                                                          .
200
10
20
62/27
Subnet Broadcast Subnet 1
200
10
20
63/27

200.10.20.33 ….and so on till 200.10.20.62   - 13 Host can be assigned IP Address.

Conclusion :

As the world is growing rapidly towards digitalization, use of IP Addresses is also increasing, So to decrease the wastage of IP Addresses, the implementation of CIDR is important that allows more organizations and users to take advantage of IPV4.