Okay let's say that hypothetically speaking, you want to upgrade a server. You order the server, and it gets delivered in the morning, after some delays. You login to the freebsd 8.2 64bit box, and you start to upgrade ports, and then recompile kernel, and upgrade to the latest patch level.
Then you install cPanel, and once you setup the system mails to your catchall mailbox, you notice something weird. A binary exim-update is running on the server every minute, trying to open a reverse shell to another server.
You become puzzled, and you start checking the logs.
What do you see ?
Well you see that only ip addresses from the datacenter connected to the box besides you. You also notice that who connected to the server did "unset HISTSAVE" (thanks csh shell log timestamps), you also notice that the exim-update binary was uploaded to the box within 15 minutes from when that login from the datacenter was made.
You also notice that the reverse shell tries to connect to a server owned by the datacenter.
Let's also say that you have not used your normal admin/root password when you asked for the server installation, because that's a password you setup only after the box is completed by you, but a complete unique password, that only you and the datacenter guys knew.
Assuming the facts above. What picture do you get in your head? Any logical explanation ?
For the argument's sake, the server mentioned is already being reinstalled now, by one of the techs i actually trust but ... just saying ... how would you feel ?
As weird as this may sound....this is not uncommon. I've had the same thing happened to me a while back from a vps provider. It's usually that the provider outsource their support and their support staff makes a little cash by giving away your credentials.
I also didn't jump into conclusions, but I did leave the vps provider.
Well the dc said that they have a script that puts up the ip aliases, and that script does the unset ... but as far as i know, any shell script if you run it in a shell, you will get logged the initial command that you did to run the script, not the individual commands the script does like ...
./script and the script has several commands in it, the .history or .bash_history will only show ./script as logged command, and not the individual commands that i have listed in the script.
as for the rest ... they are looking into it. (they found nothing so far. i asked them to investigate who logged in at that time, and who owns the other server the reverse shell was supposed to connect to.) it's a pretty fun story heh ? I guess i have to say thanks to my background - i come from shell hosting - had to deal with all kinds of kiddies, exploits, hack attempts .
thanks for liking our pricing try it out, and see if you like it even more
"You also notice that the reverse shell tries to connect to a server owned by the datacenter."
Is it actually owned by the datacenter (like RWHOIS/rdns -> .office, .support, CompanyName Internal Infrastructure) or is it maybe another compromised system autoscanning its own subnet?
no. the thing is the backdoor doesn't like perl threaded. i had to install perl threaded on this box, and therefore it was not running properly.
i analysed the backdoor, and it was only supposed to open a reverse shell, and connect to a hard coded ip address(the server owned by the datacenter), not scan a subnet. i can make the difference between a scanner and a backdoor.