So you’ve discovered that all of a sudden your server load has shot and your email inbox is getting filled up with hundreds of bounce backs. You sir may be spamming! Now if you are a spammer, this isnt really much of a shock.
But if you’re not a spammer you may be wondering what happened. Well odds are your website got hacked or your personal machine has a virus/malware. Now usually if your website gets hacked its because you are using an outdated version of your CMS software. Because wordpress doesn’t auto update and we don’t always login every day (especially if your wordpress just hosts a static website) it can be hard to keep up with the constant updates. And of course in doing so all the little hackers out there are now able to exploit whatever security holes you didn’t patch. On top of this it isn’t just the wordpress core we have to worry about, but also the plugins and the themes. Joomla and drupal and practically all other CMS’s follow the same logic. Keep your apps up to date and your chances of being compromised slim down quickly. What happens though when you are compromised? Usually the attacker places a php file on the server that acts as part of a ddos or a script that sends out a ton of spam. If your personal machine was compromised then whatever application you are running to connect to your email (such as outlook, thunderbird, etc) is usually used (or they just grab any IMAP/SMTP connection info) and use that to start spamming.
Ok, so we know that someone on the server is spamming. We don’t know if its a script or if its because someone’s personal machine got attacked. Lets take a look a couple one liners to help out with this. First, lets look at a command which searches for all external logins (meaning the personal local machine was compromised)
$ exigrep @ /var/log/exim_mainlog | grep _login | sed -n 's/.*_login:(.*)S=.*/1/p' | sort | uniq -c
1 test@example.com
3 info@example.com
59 notaspamer@example.com
So this will exigrep through our mail log and return any line containing an @ (meaning pretty much everything) cut out the dovecot_login or courier_login (whichever one you use) and then sort it and count how many instances there are. In this case you can see that the email account notaspamer@example.com is sending much, much more than the other two email accounts it found. This doesn’t immediately mean that its a spamming account, it could be legitimate of course but it gets you on the right path.
Now lets look at a few one liners for checking which user/account has been hacked:
$ exigrep yahoo /var/log/exim_mainlog | grep U= | sed -n 's/.*U=(.*)S=.*/1/p' | sort | uniq -c
3 user1 P=local
74 user2 P=local
So here the user “user2” is sending the most email on the system so we know that this user is likely responsible for the spam. Lets see if we can track down the script!
grep "cwd=" /var/log/exim_mainlog | awk '{for(i=1;i<=10;i++){print $i}}' | sort |uniq -c| grep cwd | sort -n | grep /home/Running this will look at any lines in the exam log that contains the “cwd” string. This should help narrow it down the folder where the spam is happening. But we can get even more specific! Note that this command doesn’t have as high of a success rate as the previous ones but when it works it saves so much headache.
grep X-PHP-Script /var/spool/exim/input/*/*-H | awk '{print $3}' | sort | uniq -c | sort -nrNow unlike the others this actually searches the active email queue. So if you have hundreds or thousands of email queued up (and you can check this by running exam -bpc) this should work. It looks for the X-PHP-Script field in the header of the emails. This should be enabled by default in cpanel, if not it can be enabled in the whm. But anyways this should again sort and count exactly which script sent the email! Pretty cool, right!
