I'm a novice in infrastructure management, but I really liked this field that's why I plunged into this domain and now I'm really loving it because of such challenges. Now let's cut the crap and straightaway jump to the point, I've figured few of the best practices that you should always do while configuring your "SECURE" linux server:
- Don't use default ssh port for login into the system, or best you can have a policy where you will change your ssh port every month or 2 month.
- To go a step forward disable the password based login and just enable key base login.
- Use some intrusion prevention framework, I've figured out fail2ban is a good one.
- Keep all non public facing machines on private ip.
- In case of public machines only open those ports which are actually required.
- User firewall to it's maximum effect. Iptables can be a good option.
- Have a strong alert system that can monitor your system and raise an alert in case of any suspicious activity. We use Icinga.
Though this list may not cover all the required things that you can take care of, but it can serve as a very good starting point. Also I would love to hear more suggestions that can be used.