One of my servers has been hacked today.
tera-byte has taken it down to try and restore it. Does anybody have any idea how long this can take ?
There are 125 sites on that raq :bawling:

The hacker left a new index file at several sites, but did not format the disk or anything.
He's using a anonymous email adres, wich i'suddenly feel should be forbidden by law !

I had all patches installed except for the last 3.

Man i wish i could get this guy in my hands, if i would find out exactly where he lives i would buy a ticket to Indonesia immediatly !

:angry:

Do you know how the intruder got in?

The bind vulnerability should have been fixed before that last patch. Was it the ssh crc-32 vulnerability? I still get snort reports showing alot of scanning for that one.

I have no idea, i guess Tera-byte will take a look at the log files.
I wasn't running DNS on it, so i guess it won't be BIND.
The message said ...GoTRooT...

I would VERY much like to know i how i can prevent this in the future.

How much are they charging to clean up your RAQ?

I've posted warnings on the forum a couple of times about the sshd crc-32 problem. There are automated tools for finding and getting into hosts using this vulnerability. I moved my sshd port and removed ssh level 1 compatibility. Also loaded oepn ssh v3 package from one of the unofficial package sources. According to my logs, these attacks are dying down now.

I'm hoping their aren't any more vulnerabilities to catch us RAQ-ers unaware.

Tera-byte is charging $50,- an hour to restore.
I would gladly pay double if that would speed things up :(

So i understand SSH is unreliable ?

no, SSH is pretty re-liable
it's just telnet that is less secure

so if i would disable shell access on the main account (and all others) then i would be ok ?
Thought i was save as long as i logged in with SSH

see my posts at:
http://www.webhostingtalk.com/showthread.php?s=&threadid=24355
http://www.webhostingtalk.com/showthread.php?s=&threadid=8170&perpage=15&pagenumber=2

ssh does not send username and password in clear text(telnet does). This is important because of the amount of hacking being done on routers these days.

ssh is good. shell access is good. There is a ssh vulnerability, but it is easily fixed. It looks like the vulnerability has been around a while, but in august some hacker allegedly started selling(!) an automated tool. The crc-32 vulnerability is still listed in the top at the cert advisories (see my previous thread). My cobalt was configured to fallback to ssh protocol v1, and I believe this made it vulnerable.

I also move the port for ssh. If someone is scanning hundreds of systems for port 22, they may take a closer look at how to get into my box if I show up on the list. (a little obscurity never hurt...)

Hope that helps.

Thank you Monkey Boy,
sound easy enough, i'll do this as soons as my raq comes back up..

in language that me, joe bloggs, can understand. How do you change a port on the raq, if it helps cut down on potential hacks, I want in!

Well here's a update from tera-byte:


A recent hack exploiting security issues on cobalt raqs has been
employed in an attack effecting your server
(an SSH client on port 20000, was common to all of the compromised
servers, and
a scan of your server indicated
that this was indeed running) This port has since been blocked at the
firewall and the ssh client disabled on the
effected servers.



So indeed SSH....
Now for everyone trusting the the raq's backup in use with the tera-byte backup server :

Please note that any backups done with the cobalt backup/restore utility
will not restore to a site that has not yet been created. (sites also
need to be recreated in the same order to use the backup/restore)
In fact we do not recommend depending on this data as your restore
method.

So there goes my weekly backups of my 2 servers :(
I know there's been a lot of talk about backup scripts, they all seemed a bit to complecated for me. Can anyone offer a simple and affordable and most of all RELIABLE backup solution ?

Ok, this is BAD!

I got an eMail from tera-byte saying my site had port 20000 active and that they have firewalled it and removed SSH from my system.

I didn't notice any changes to my pages at all.

The question is, if someone installed some malicious code on my system, how do i check for that? Is there a start-up code I should look at? I can't afford to pay tera-byte and as a whole my site seems uneffected.

I had all but the last security patch installed, and reading it, it only seemed to apply to local users (which I don't have). I didn't use bind (using tera-byte as dns).

How do I even log into my site now since SSH has been removed by tera-byte! I can't even install a new SSH as it won't install. And telnet sure isn't safe.

And while I'm at it. How secure is the site maintainance login? and how about the webalizer login, ftp login, etc... ?

If my site was comprimised, what can I do on my own now given the aforementioned circumstances???

blight:
The terrabyte notice said the ssh client was removed. It sounds like the ssh server should still be available for you to login to.

http://vito.pointclark.net/cobalt/chkrootkit.html
this checks for a "root kit" that would indicate you have been breached.

http://vito.pointclark.net/cobalt/snort.html
snort writes to your logs when you are being scanned, etc. I have been using snort. Snort shows the number of attempts to connect to my port 22 (formerly my sshd port) have dropped to 4 over the last month. Previously, i was seeing 4-10 a week.

joe bloggs:
easy fix:
login. su to root. type:
pico -w /etc/ssh/sshd_config

at the top of the file, change:
Port 22
to:
Port 8xxx
(but choose an actual number in place of the small x characters)

Also change:
Protocol 2,1
to:
Protocol 2

exit out of pico (control+x). Yes to save.

type:
/etc/rc.d/init.d/sshd restart

Also be sure to write down the new port and change it in your ssh client software(putty or MacSSH, probably)!


Every morning, I check:
http://www.cert.org/current/current_activity.html and securityfocus.com, hoping to find out about issues before crackers do!
(The current CDE subprocess vulnerability does not apply to RAQs)


There are alot of knowledgeable folks here on the forums. Search for snort.

About backups, I use the perl script that was given out here on the forums - it works great (search the forums), but i have not restored from it.

I (I) would have them reformat your system that way you can be 100% sure there are no root-kits installed.

They will do that, the downside is that it costs $300.
They take the sites of and restore them to a fresh raq.

I asked them to reformat with all the security patches with no backup at all (I'll reinstall everything myself). Hopefully they won't charge me $300 for that.

They already did it, took them under 6 hours, so it wasn't that bad.

6 hours :eek2:
And that is even without the restore ?????
I pay the $300 for a restore, but hope they can do it in less then 6 hours.
I gave them a ok for a restore, but haven't heard from them since then, i asked if they could inform me when they gonna take it offline, that was yesterday, but no reply yet, guess they are very busy now restoring all the hacked boxes.

Now what bothers me a bit is that this vulnerability was known by others at the forum, Monkey-boy even posted the solution.
I'm renting raqs now for exactly a year and this was the second time my server was among a bunch of hacked servers.
first time it was BIND after wich TB banned running DNS on your server.
This also was a very well known problem with that version of BIND, but ofcourse most users like me don't know about that.
I know i should watch sites like securityfocus.com, but i don't know what i'm reading when i take a look at these sites.

Is it a strange suggestion if company's like TB and RS post such problems and the solutions as soon as they find out in a newsletter or forum ?
I'm sure they watch these sites, and they DO know what they are reading, they also know the solution.

I know the raq's safety is the responsebility of the owner, but their target audience just don't know enough about these things, and it would be much less trouble then restoring all these raqs twice a year.

dutchie:
I sent them an email to format it, then left.

when I came back 6 hours later, it was reformatted. No idea how much time it actually took. I doubt the actual work took more than an hour. Probably 5 minutes if they just used a cloned cd of some sort.

As for security update.

I actually read it, and the bug seemed to relate to users of your system being able to obtain access. My site didn't have any user other than myself, so I have no idea how it was hacked.

After the format, I changed the SSH code/port, modified the admin password so that it doesn't match the root password (something I guess I should have done long ago).

I really hope that's enough to stop whatever they were using to hack the site.

Well how it was hacked is no mystery, the hacker posted it on a site from wich i suddenly got a lot of traffic..

Interesting ports on myhackedomain.com (216.234.xxx.xxx):
(The 1 port scanned but not shown below is in state: closed)
Port State Service
80/tcp open http

Remote OS guesses: Linux 2.1.19 - 2.2.17, Linux kernel 2.2.13, Linux 2.2.14
Uptime 0.423 days (since Wed Jan 23 16:56:01 2002)

# Nmap run completed at Thu Jan 24 03:05:26 2002 -- 1 IP address (1 host up) scanned in 17 seconds

This is the sites name, now i should call on you wizards here to bring this site down, but whats the use..
http://safemode.org/body_index.php

I'd rather take legal action if i knew how..

Would putting a firewall infront of your racks make it more secure. I hate hackers and a my cobalt raq3i got hacked and the guy put Casino sites on it and spammed it. I ended up getting allot of angry e-mails (That's How I found out) I disabled ftp and telnet after that.

I wish raqs would auto update themselves but then again. I heard some updates causes some raqs to reboot non stop. Would a netscreen help keep hackers out of Cobalts?

If you disable FTP, how exactly are you to update the site?

Hi,

> If you disable FTP, how exactly are you to update the site?

You might want to checkout sftp :-)

Cu, Moritz

hiiiiiiiiiiii!!!!!!!!!!!!!!!!!!!!!! :wavey:

I use SSH and have telnet disabled. What else should raq sites do to protect themselves?

As a bare minimum:

Install ipchains, pmfirewall, logcheck, use SSL for the admin panel.
keep up to date with all security patches.