Investigating your Linux box for vulnerabilities

A server in a live environment is always susceptible to malware attacks, code vulnerabilities, hacking, rootkits and what not else !

Analyzing your server for these back-doors and pin-pointing the exact issue always poise a big problem to the server owners or their administrators. Thankfully Linux can provide you with many results, logs and statistics which can help you in a great way. Lets look at some of the things which you can do when you feel like your system is compromised..

  • Unintended processes running can at-times increase the server load. Using the command # top -c,  check the current server load and analyze if you can find any unfamiliar processes or process paths running.
  •  Check for the process tree in your server and see if you can spot any unusual process/paths.  Use the command # pstree -p. If you find an unsual process get the PID of it and alanyze using lsof command, which will lead to current working directory ( cwd ) of the process.

Just for an example, consider you find lots of Perl process running, the root perl process has got the PID ‘9905’. Use the command # lsof PID to check more about this.

=================

# lsof -p 9905
COMMAND  PID USER   FD   TYPE     DEVICE    SIZE      NODE NAME
perl    9905 root  cwd    DIR       x,xx  6770688  33825697 /tmp
perl    9905 root  rtd    DIR       x,xx    4096  33465149 /
perl    9905 root  txt    REG       x,xx   13696  41518574 /usr/bin/perl

=================

Checking this shows you something, an invalid/malicious Perl script is running from /tmp, which should not happen at all.  Analyze the files in /tmp and make sure you clean up the unwanted ones.

  • Use # ls -al /tmp ( followed by grep arguments ) to check for the files in /tmp.
  • Another recently noted process in # pstree -p, is lots of hosts commands being carried out. Capture the PID of one of the process with the name ‘hosts’ and run an lsof on it. Probably you might see something like :

================

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME

host 7909 nobody cwd DIR 0,32 4096 117770255 /home/domain/public_html/blog/wp-content/uploads

================

The host command, which is used for resolving DNS, is being executed and some scripts is coercing it to do HTTP requests, which is seen as the issue here. Blocking the scripts found in the location ( cwd ) will help in this case. Analyze if it was uploaded via any unsecured plugins ( mostly happens with WordPress )

  • Check the IP’s making connections to you server.  Use the command given below to find the number of tcp/udp flow :

netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

If you find the connections from an IP is not on the normal side, check for what it is trying to access :

# netstat -plan | grep IP

And block ’em in the firewall if it looks invalid.

  • If you are using a cPanel server, you can find what exactly it is trying to access if its to the web-server.

# grep -i -r IP /usr/local/apache/domlogs*

There are many occasions in which the IP would be trying to brute-force attack the logins such as WP-login.php. In those cases, you can find that IP is ‘POST’ ing the logins to the login.php page.

  • Changing the username of WordPress from the default ‘admin’ to a stronger one can also help.  These sort of attacks can increase the server load as well as compromise the server security, if the brute-force attack is successful.
  • Find out the most accessed domain for a particular day by giving the following command :

grep -r '14/Dec/2014' /usr/local/apache/domlogs* |awk {'print $1'}|cut -d: -f1|sort -n|uniq -c|sort -n

^ Date is to be given in this format.

  • Find out the most accessed IP for the domain which you get from the above result :

# grep -r '14/Dec/2014' /usr/local/apache/domlogs/domain.tld |awk {'print $1'}|cut -d: -f1|sort -n|uniq -c|sort -n

  • Checking the root history ( cat /root/.bash_history ) can help you to see if your server is root hacked. Also check # last command to see if you can spot any unfamiliar IPs, which tried to login to your box.
  • You can use rkhunter to scan for possible rootkits and local exploits in your linux based box.  It also performs checks to see if commands/paths have been altered , if the system startup files have been modified, and various checks on the network interfaces, including checks for listening applications.

Once rkhunter is installed, you can scan the system using the command # rkhunter --check and find if it returns any negative result.

So, let the Sherlock Holmes investigation start !

Control panels for production servers

When it comes to selecting a control panel for your hosting server, you might be in a perplexed stage. The industries hit-list items are cPanel, DirectAdmin and Parallels Plesk. I have played with these 3 and most of the times when I play longer, they screw me up. The one I’m comfortable with is cPanel and let me explain WHY !

cPanel :

Most colorful, most graphical user-interfaced and easy to understand/use — cPanel offers stuffs more than any other control panels has to.

They have many many custom Apps tied to their interfaces and are highly configurable by the users. This is a serious plus-point when compared with Plesk or DirectAdmin. Yes, its true, more features means more bugs. However, there will be a workaround/fix for them at the earliest. Newer versions with new fixes/features get released in the blink of an eye, while you have to wait for long for DirectAdmin or Plesk.

cPanel is very widespread, their forums answer about 99% of your questions/issues. If not their support ( the quickest among the lot ) will do it for you.

Upgrading your services ( like PHP or MySQL ) are very much simple when it comes to cPanel. The tool integrated to provide these upgrading tasks can include custom modules and you can run your server with 2 versions of PHP, set both of them with different PHP handlers. Cant imagine something like this in Plesk/DirectAdmin Yes, it is possible, but 90% of the time, things break leaving you with no option other than restoring the entire system. I still remember me sitting hours to install 2 versions of PHP in Plesk.

Quick installation setup’s using scripts is a great feature of cPanel, while you have to depend on yum commands in Plesk or DirectAdmin. Just take the example of PHP modules. How much headache’s it can cause in Plesk/DirectAdmin when you do not have the required versions of modules in the repo’s.

When it comes to emailing,  creating email accounts, tieing them with SpamAssasin or BoxTrapper , changing the mail server port, even the interface IP and so on… they are very much simple and time saving. On the other hand, when you attempt to do these stuffs in Plesk or DirectAdmin, it will present you with lots of issues and time-loss.

Creating a reseller account, assigning permissions to them, creating different packages to meet your needs — these are presented to you using a simple UI in cPanel/WHM. You would not have to worry about searching online for help on this. For a person who is not so acquainted with shell environment, cPanel will do you great heavens.

The Bandwidth statistics, raw access logs and much more logs, which can be viewed from the front-end give you a clear picture on the traffic/data with respect to your accounts.

One of the things which I like about cPanel is its system of Backup/restore websites. Either you can manually backup the required accounts or allow cPanel to do it automatically. When you have a backup file, suppose your developers screws off the entire site, you can quickly restore – without any headaches.. Something that’s missing in other panels ( ‘I meant the term – without any headches’ )

Also, migrating your domains b/w cPanel-based systems is quite wonderful and I guarantee you 99% success all time time. With others, I cant recollect the pains i go through with.

Upgrading your cPanel version is as simple as # /scripts/upcp.   With DirectAdmin too it is quite easy with the custom scripts located in # /usr/local/directadmin/custombuild/.. Plesk too has got a script,   # /usr/local/psa/admin/sbin/autoinstaller. But with plesk, you will have to answer a 100 questions before upgrading and higher the chances to fail.

cPanel has two interfaces, the WHM ( Web Host Manager ) for the root account as well as resellers accounts and the usual cPanel interface for the domain owners. These two are very very different in looks, with different login pages. On the other hand, Plesk and DirectAdmin, got single login for Admins and users ( with different menu’s )

Plesk – runs in both LINUX/WIN server’s. Its more of like a windows control panel ( what you see in desktop versions ), less features and might take more time for you to get acquainted with it. For Plesk, as features are less, bugs are less and things move on stable until you try to alter the usual builds. Not much help you will find in online forums.  Plesk support can be messy at times. They require you to have your server build only from the OS repo’s. If you install any 3rd Party repo or if you Plesk version is quite old, then its a goner. Forget about getting your issue fixed.

Plesk is for light-hearted guys, still living with black & white frames and quiet ones.

cPanel and Plesk are on the expensive side, whereas DirectAdmin is cheap. If you are thinking of running a small hosting environment, with not much screamers, you can go with DirectAdmin — Quite simple, efficient, not much features ( less bugs ) and stable.

So, I prefer cPanel over Plesk for real production environments and DirectAdmin over cPanel and Plesk for a startup environment !

Try their demo’s  at :

cPanel demo : http://cpanel.net/demo/

Plesk demo : http://www.parallels.com/products/plesk/hands-on-demos/

DirectAdmin demo : http://www.directadmin.com/demo.html

Ubuntu 14.04 LTS – A quick review

Finally after much hype , the latest LTS’d version of Ubuntu is here, 14.04. I had been a Debian fan for long and was waiting eagerly for the release of 14, with a hope Ubuntu would eradicate the common bugs and issues which i have been facing in late versions.  It comes out with GNOME 3.10, although many expected GNOME 3.12 desktop would be shipped in. Yes Ubuntu 14 looks just like the previous versions, whether it be 13.10 or 12.04. Also, Unity ( shell interface for the GNOME ) hasn’t been upgraded to 8 and the latest version of Ubuntu is running with unity 7. 

In this case however, there  is an enhanced support for online accounts.

onlineaccount1

 

Just provide your online account and you will not have to bother about signing in separately to email clients or chat clients. Just grant Ubuntu access and you find it running smoothly. That said, the online accounts is easily optimized for providing better UI than the previous ones. The dash and searches — you will love them as more specific search needs are met both within your system and over the internet.

Look-wise, nothing great have changed with Ubuntu 14.04. Performance wise, speed of operation seems to be improved.

Bugs ? 🙂 with a thought, i booted Ubuntu 14 for the first time and this is what I got!

“Error: diskfilter writes are not supported.
Press any key to continue…”

Check out this here, if you face this issue :

https://bugs./ubuntu/+source/grub2/+bug/1274320

Glad news is that Canonical will be supporting this desktop Linux for five years. Coming to the technical aspects, the kernel used is the Linux 3.13 kernel, which focus on enhancing the security aspects as well as  limiting the power consumption on individual components.

terminal

 

There has been a tremendous effort in order to save the power drain which existing in some arch’s for the previous versions.  According to the Ubuntu feature’s list, there is an introduction of the IntelPowerClamp driver for improved power efficiency. 

And yes, I’m able to see appreciable power efficiency in my Thinkpad-510.

I have not yet tried much custom apps on Ubuntu. The LibreOffice suite has been updated to the latest version, 4.2, in which especially LibreOffice Calc has got more features and looks simple which might slowly rub the headache of user’s who always wanted MS-office when thinking of the complicated behavior of Libreoffice.

All in all, it looks promising. A revised version of older Ubuntu with considerable improvements. However, will this be a replacement for winXP ? Have to wait and watch !