Web Hosting Talk







View Full Version : Why do all data centers screw you during DoS


NexDog
01-17-2004, 11:37 AM
They all the same. No communication, they don't care. Managed, unmanaged, it's all the same. And from a provider I've recommended a hundred times. Pfffft.

XYPHEN
01-17-2004, 11:39 AM
Yes I noticed the same thing, I was getting DoS and they terminated my service without listening to what I have to say, I had to switch services back then, now I just do what I can do prevent DoS myself.

NexDog
01-17-2004, 11:44 AM
All they need to do is respond to email. Not leave me hanging for SEVEN hours during which time I've emailed 3-4 times. And then when I send in a severely pissed off new ticket, the tech tells me to watch my tongue or I'll get me account terminated and tells me off for sending in multiple tickets.

Erm....hello? If they responded to the original ticket there would be no pissed off new ticket. :crazy:

qm8309
01-17-2004, 11:47 AM
may we know which dc is it?

Mrdredd
01-17-2004, 11:49 AM
There are providers who respond and are available and will do everything possible to mitigate a DOS or any other type of attack - it all depends on which provider you are with.

NexDog
01-17-2004, 11:55 AM
This provider was supposed to be one. Spouting "managed" and all that nonsense. There's no such thing as "managed", just the provider's interpretation of it and that is "I'll say managed so I can take more cash". They actually do nothing for us really as we are light maintenance as we have a bunch of sysadmins but wouldn't it be nice to smell something of that managed in cases like this?

Thanks to WHT for letting me vent. And no, I'm not saying who it is (unless they continue to shine in the non-communication department). Those who know who we use will figure out who it is.

NexDog
01-17-2004, 12:20 PM
It appears that screaming does help. 12 hours later and the block is lifted. Almost by magic, heh. Client screams, someone decides to check and lo and behold, the attack is finished. Probably finished a few minutes after the night shift left about 9 hours ago. Big thumbs down on how this was handled. No communication, no monitoring and no investigating the packets to find source. Bleh.

dynamicnet
01-17-2004, 03:08 PM
Greetings:

1. If you have your servers at quality data centers like Rackspace.com, NTT/Verio, and the like, chances are high these events are rare.

2. If you apply the recommended sever security layers to your machines, DoS attacks can be minimized to a degree (DDoS typically is better handled by hardware devices geared towards DDoS).

Thank you.

Mfjp
01-17-2004, 03:31 PM
For them to filter out any DOS attack, they'll have to have access to the routers. There are just these providers out there that charges a bit more because they offer management of the OS and such for you. They may not run the network.

chet
01-17-2004, 03:57 PM
Has anyone seen servermatrix's flood guard in action? I am interested to see how well that works.


thanks

NationHosts
01-17-2004, 04:43 PM
so if you own a switch and a rack and servers how would your provider respond if your getting DDoSed? their gonna pull all your servers?

2Grumpy
01-17-2004, 06:22 PM
Well hostnexus.com traces into EV1 but by your words I was assuming you were dealing with colocation since you said "data center" rather than "dedicated server provider" plus you used the words "managed" and of course EV1 is a lotta things but not moanaged so am I right in assuming this isn't EV1?

And to get back on topic, yes I agree, it's a REAL pain when a dos/ddos happens the first thing they do is cut off stuff, and then soem may or may not do much else. So far GNAX has been very good to us about helping to get DDOS's handled, from memory we've had 1 real hardcore bout with a ddos and a handful (3 or 4) small bouts.

Luckily we haven't had to deal with many ddos's the one or two I've had in EV1 were hardly ddos's like you see, it was more like wanna be ddos's that didn't hardly register on the scale :) Don't reckon we've had any within HE other than HE's own problems with broadcast crap and other stuff originating IN HOUSE (ditto EV1 I can remember at least one or two occasions where someone else in our subnet was the problem).

illogix
01-17-2004, 06:31 PM
I don't think NexDog is talking about EV1. I remember from older threads that his company was using Rackspace.

I don't know if that is still the case thought.

And I'm sure he knows that EV1 is not managed so he is most likely talking about Rackspace or another managed provider.

mdrussell
01-17-2004, 07:12 PM
Originally posted by NexDog
All they need to do is respond to email. Not leave me hanging for SEVEN hours during which time I've emailed 3-4 times. And then when I send in a severely pissed off new ticket, the tech tells me to watch my tongue or I'll get me account terminated and tells me off for sending in multiple tickets.

Erm....hello? If they responded to the original ticket there would be no pissed off new ticket. :crazy:

In this kind of situation, I would have called them to discuss options within 5 minutes. Obviously, the datacenter has got their network to consider but full suspension of service seems a little extreme.

I would also be interested to know who this was.

FHDave
01-17-2004, 07:25 PM
HostNexus, I believe, moved from Rackspace to NAC. Not sure if it's directly with NAC or somebody else at NAC. My guess is the later.

Mrdredd
01-17-2004, 07:25 PM
Originally posted by NexDog
And no, I'm not saying who it is (unless they continue to shine in the non-communication department). Those who know who we use will figure out who it is.

XYPHEN
01-17-2004, 08:24 PM
NexDog, I thought you had your own DC by now. Oh well, guess not, your website is great btw, just a big fan of it. I understand your problem as well, was it one of the sites on your server getting DoS or the whole server?

NexDog
01-17-2004, 08:27 PM
Hey, wish I knew. Didn't say anything from a tcpdump, no traffic stats, no indication of how many packets flooding in - absolutely nothing. Just "Your server is getting DoSed so the main IP is being null routed". Then the network admin went to bed.

dogzilla
01-17-2004, 08:31 PM
If youve got a cheap dedicated server - whether its unmanaged or not - you cant expect the company to absorbe the cost of a large ddos.

When you start paying real money - generally hosts will be more willing to deal with the problem rather than just taking the easy way out and terminating the account.

FHDave
01-17-2004, 08:31 PM
That's why we love Internap. Their network admin never sleeps :)

Sorry to hear about your problem. How long were you down?

dogzilla
01-17-2004, 08:36 PM
Just to note - in my opinion the best course of action when you are on the end of a large ddos is to blackhole the IP address. 90% of the time the attack will have dropped off after 10-15 minutes.

Only in a few cases has the ddos started back up - i imagine the script kiddys dont want to overuse there remote sites and compromise them.

NexDog
01-17-2004, 08:38 PM
Originally posted by dynamicnet
Greetings:

2. If you apply the recommended sever security layers to your machines, DoS attacks can be minimized to a degree (DDoS typically is better handled by hardware devices geared towards DDoS).

Thank you. (for what? Why say thank you after each post? Thanks that we have to put up with your condescending attitude?)
I find your attitude and general presence on this board insulting. You are a self professed security expert but you continue to spout turd. Your silence in the other thread after you made a fool of yourself about Unix passwords spoke volumes. All you do is post the same lame list every time the word security pops up.

And now you come up something like this. It stifles the imagination.

There is nothing you can do on the server to prevent a serious DoS attack. The data center null-routes the IP at the router. Get it? Servers.....routers......

Thankyou.

NexDog
01-17-2004, 08:42 PM
dogzilla - the servers aren't cheap. The are on the "premium" line at the data center. Who said anything about termination? I'm talking about communication. Our combined server bill is $7+. That isn't "cheap".

Dave - 12 hours. (believe it)

wheimeng
01-17-2004, 08:44 PM
Originally posted by NexDog
This provider was supposed to be one. Spouting "managed" and all that nonsense. There's no such thing as "managed", just the provider's interpretation of it and that is "I'll say managed so I can take more cash". They actually do nothing for us really as we are light maintenance as we have a bunch of sysadmins but wouldn't it be nice to smell something of that managed in cases like this?

Thanks to WHT for letting me vent. And no, I'm not saying who it is (unless they continue to shine in the non-communication department). Those who know who we use will figure out who it is.

http://www.webhostingtalk.com/showthread.php?s=&threadid=225376&perpage=15&pagenumber=2 maybe this explains? :P

wheimeng
01-17-2004, 08:51 PM
Originally posted by NexDog
I find your attitude and general presence on this board insulting. You are a self professed security expert but you continue to spout turd. Your silence in the other thread after you made a fool of yourself about Unix passwords spoke volumes. All you do is post the same lame list every time the word security pops up.

And now you come up something like this. It stifles the imagination.

There is nothing you can do on the server to prevent a serious DoS attack. The data center null-routes the IP at the router. Get it? Servers.....routers......

Thankyou.

Chill off Laurence :)

You wouldn't like to spoil your mood just because of someone whom you can't see, would you? I find ignorance is bliss :P

BTW, maybe you should find someone that really take incharge in this rather than small potatoes techs :D

NexDog
01-17-2004, 08:55 PM
There was no-one in charge. Everyone was in bed.

wheimeng
01-17-2004, 09:12 PM
Anyway is the server back online or its still offline?

amusive.com
01-17-2004, 09:27 PM
Originally posted by NexDog
dogzilla - the servers aren't cheap. The are on the "premium" line at the data center. Who said anything about termination? I'm talking about communication. Our combined server bill is $7+. That isn't "cheap".

Dave - 12 hours. (believe it)

Jeeze, $7 is pretty cheap IMHO. ;)

NexDog
01-17-2004, 09:29 PM
Yea, came back after I sent a rather expressive email. Twelve hours down though. Hard to believe the attack lasted 12 hours and when I screamed enough, it was magically over. Like I said, no-one was monitoring it so how would they have known the attack was over?

NexDog
01-17-2004, 09:30 PM
Jim, heh, forgot a k. :stickout:

liquidweb
01-17-2004, 10:26 PM
Originally posted by NexDog
Yea, came back after I sent a rather expressive email. Twelve hours down though. Hard to believe the attack lasted 12 hours and when I screamed enough, it was magically over. Like I said, no-one was monitoring it so how would they have known the attack was over?

Lack of reliable 24/7 phone support from on site technicians in inexcusable for any actual datacenter, at which stage you could have worked through the events as they transpired.

Beyond that at $7,000/month they should have a technician who's primary priority is to manage your data and servers reliably. If they were in fact on top of the series of events as they unfolded, then it at least sounds as though they failed to communicate the situation in honesty.

Hopefully not a recurring event for you, if it is I would hope you being to consider other more reliable options. When something goes wrong such as a hardware failure or a DDOS for example, is when a provider truly shows their strength or weakness. When all is well most providers are fairly comparable.

NexDog
01-17-2004, 11:13 PM
Wow, Matthew, 3 posts in 3 years - that has to be a record. :D

I don't even expect 24/7 phone support, but I do expect 24/7 ticket support. Our bill at that data center is half that amount actually. The other half goes to EV1. And I was just about to move 2 servers from EV1 to NAC. Not sure about that anymore.

dynamicnet
01-17-2004, 11:15 PM
Greetings NexDog:

“There is nothing you can do on the server to prevent a serious DoS attack.”

If you re-read my post, you will see that I used the word “minimized.”

If you get out a dictionary, you can find out for yourself that “minimized” does not equal “prevent.”

“Like I said, no-one was monitoring it so how would they have known the attack was over?”

No monitoring?

Thank you.

NexDog
01-17-2004, 11:36 PM
Sorry, can't read what you said as you're now on my ignore list. Knock yourself out though.

XYPHEN
01-17-2004, 11:44 PM
rofl, thats just hilarious.. NexDog is the man!

coight
01-17-2004, 11:54 PM
Why didn't you call them?

linux-tech
01-18-2004, 12:14 AM
I've never had the misfortune of having to deal with crap DC's during a DOS attack, thankfully. I've seen a few, and generally iptables and APF will block them.
The few I've had the misfortune to deal with have been through Fastservers, and those guys have yet to let me down, even blocking the attacking ip's within minutes after letting them know who it was.

A DC will work with you, a RESELLER won't, because they can't. Yet another reason you want to deal directly with the DC.

FHDave
01-18-2004, 01:51 AM
Originally posted by wolfstream
A DC will work with you, a RESELLER won't, because they can't. Yet another reason you want to deal directly with the DC

Depends, which DC ... Not all network admins are equal.

12 hours of downtime? I am sorry to hear about that ... But if you are spending $7K/month, don't you think it's time for you to get your own cabinet and deal with a more reliable data center?

GWDGuy
01-18-2004, 03:19 AM
I have co-lo servers in three different DC's. Two are local here in town so I like dealing with them because I can stop in anytime and say hello (Bring pizza too... it make them work harder for you later) They are pretty good at letting me know what is happening during there downtimes, tought to get through on the phone but still they are honest of what is happening.

anyway... I also have a co-lo at Fastservers and they have been great.. it was a little rocky at first until they did that huge router/something or other upgrade but since then anytime there is a problem they answer the phone 2nd ring, post here and in their forum so their customers at least know that they are working on it.

Nothing like sitting for hours in the dark..

Not every provider is going to be the best for everyone but if you are getting bad service remember yelling solves nothing... moving to a new company tells them how you really feel. What I am saying is most companies will listen to you yell and scream and do nothing but collect your $$$. Do something that hurts them and move to a different provider.


My 2 cents.

Robert

IGobyTerry
01-18-2004, 03:46 AM
Originally posted by NexDog
Sorry, can't read what you said as you're now on my ignore list. Knock yourself out though.
C'mon man, show a bit of maturity and professionalism. There's no need for comments like that. If you have him on your ignore list, that's great - use it. Ignore him. Don't make stupid, immature comments like that.

NexDog
01-18-2004, 03:59 AM
I thought it was quite amusing. ;)

IGobyTerry
01-18-2004, 04:00 AM
Originally posted by NexDog
I thought it was quite amusing. ;)
It was funny, but I just thought it wasn't really necessary. Ya know?

liquidweb
01-18-2004, 05:23 AM
Originally posted by NexDog
Wow, Matthew, 3 posts in 3 years - that has to be a record. :D

I don't even expect 24/7 phone support, but I do expect 24/7 ticket support. Our bill at that data center is half that amount actually. The other half goes to EV1. And I was just about to move 2 servers from EV1 to NAC. Not sure about that anymore.

Indeed. I haven't previously been much beyond a lurker, my registration was quite a while ago versus the time I've spent actually monitoring threads.

FHDave
01-18-2004, 09:39 AM
NexDog,

What a privelege. 50% of Matthew's total post are just for you! :)

I find it amazing too that somebody can be that quiet :) Must be very busy down there ....

wheimeng
01-18-2004, 09:45 AM
:D :D

McRox
01-18-2004, 10:24 AM
Originally posted by NexDog
All they need to do is respond to email. Not leave me hanging for SEVEN hours during which time I've emailed 3-4 times. And then when I send in a severely pissed off new ticket, the tech tells me to watch my tongue or I'll get me account terminated and tells me off for sending in multiple tickets.

Erm....hello? If they responded to the original ticket there would be no pissed off new ticket. :crazy:

I dont think dedicated server providers that oversell their bandwidth or have a low profit margin are able to pay the bills for the traffic due to a DOS attack.

Lets say, your box connected to a switch with 20 other boxes is getting DOS-ed. I don't think the others would like to be on the same switch since it would harm them aswell.

Putting you on a different switch would be the only solution, unless your provider cant pay the bills for the dos attacks.

A DOS attack that takes longer than 36 hours means they could be losing alot more than $7000/m..

I'm not saying that they should stop the communication with you, without any news or updates.

I would understand that they null route the ips/take the box offline, however, monitoring the dos attack and communicate well with the customer is definately needed.

Btw, it might be a better idea to colo your own boxes and maintain your own small network if you are paying 7k/m :)

You will have better control over your network and boxes and it is cheaper in the long run :)

JohnCrowley
01-18-2004, 11:31 AM
Laurence,

Sorry to hear about the D(d?)os, and the lack of support and communication you received. There are DC's that will work with you on things like this, especially if you're paying top dollar.

We had a DDOS that started with 1 server, and when we worked around it for the client, they DDOS'ed our nameservers. Within 30 minutes our DC started analyzing packets, dropping bogus traffic, blocking long lists of IP's, and kept our nameservers barely above water so it didn't bring everyone down. Took 3-4 hours in total to combat, but they worked with us every step of the way.

Comunication was a little lacking, but we knew they were working on it, and were happy that they were able to overcome the DDOS. Not a fun day, but with the DC's help, it was manageable. We pay $400-$600 a month for each server, and in our past experience, it is worth it. Sure, we're paying more over the long run, but if you trust the DC, and they come through all the time, then it's peanuts compared to the headaches and downtime you could experience elsewhere.

After you cool down, talk to the overall DC manager or CEO and voice your concerns in a professional manner. Explain what you are paying, and what you expect. We have done this a few times when things didn't go as smoothly as we liked, and it helped for future problems.

Good luck.

- John C.

Nex7
01-18-2004, 11:51 AM
The entire DoS thing just disgusts me. SOMEONE is going to be paying professional network staff to spend hours clearing it up, when it was not CAUSED by a professional network staff, it was caused by some nimwit 15 year old from his mommy's computer, using a script or typing in a couple of commands in a private IRC room full of zombies that his cool 16 year old buddy showed him about; the 16 year old buddy found it on some warez site, he's as clueless as to how it works as everyone else, but hey, its more user friendly than a toaster oven, and it makes people's sites go down, so what the hell, right?

How about we create a giant posse and track down the original creators of every DDoS script/program on the planet, and teach them proper respect; with a Louisville slugger.

Sure it won't help anything, but I know I'd feel better, wouldn't you?

dynamicnet
01-18-2004, 01:40 PM
Greetings Andrew:

“How about we create a giant posse and track down the original creators of every DDoS script/program on the planet, and teach them proper respect; with a Louisville slugger.

Sure it won't help anything, but I know I'd feel better, wouldn't you?”

I would feel better if they all came to know Jesus as Lord and Savior. That would lead to a whole better world than their deaths.

With that stated, I do agree the DoS and DDoS attacks show more of a lack of values, and better things to do with one’s time.

Last year we created two network appliances; one of them is for content filtering. While they are still in beta, I noticed very late Friday evening that the content filtering appliance was being used as an anonimizer.

While the intent of the content filtering network appliance would be to place it within a network where only the network users had access, part of the testing was to see what the speed would be like for all users to get to it form the Net.

Low and behold, I found several hundred uninvited people using the device to make it look like they were coming from the device when they were browsing sites the content filtering system wasn’t blocking.

Ingenious, yes; but it also shows a lack of values and too much time.

In ending, I do partially agree with you that it would be nice to gather up all of the DoS, DDoS, and related scripts to destroy; and that the people behind them (not all children) should learn respect.

Thank you.

my_forum_id
01-18-2004, 05:58 PM
Greetings Andrew:

“How about we create a giant posse and track down the original creators of every DDoS script/program on the planet, and teach them proper respect; with a Louisville slugger.

Sure it won't help anything, but I know I'd feel better, wouldn't you?”

I would feel better if they all came to know Jesus as Lord and Savior.

Is he good with a slugger then ?

NexDog
01-18-2004, 07:26 PM
I heard he had a mean headbutt.

bizness
01-24-2004, 05:56 PM
first off, i dont think that provider took the right route be unplugging your machine. When i use to work at Dialtone internet, currently interland, we use to simply access the main routers and null route the incoming attack while the customer stays online. As for my current company, we do the same, we null route the incoming attack and thats it. We ask questions later on to why it happened...customers should come first not after. I dont think it is right for a ISP to go and unplug a machine just for that or even cancel the contract... their are other reasons why one would cancel the contract such as VERY often DOS attacks do to the client doing something against the TOS...

TheHS
01-24-2004, 07:02 PM
Which dc is this

Aboul_YouseF
01-25-2004, 03:44 PM
Guys im no pro. in this corner, but what actually happens in a DoS attack since i've never faced one. I've been purchasing dedicated servers for about 1.5 years and luckily nothing till now. But then whats the use of firewalls then??