Hi, I am doing busniess with Web Authorities. Is it down ? Even I can't get in touch with their main web server. No IP of their network is being reached. 216.177.33.0 and 216.177.51.0 This is the network I have.

Can't reach any machine.

Goni

Same with my server too. It seems they're having a serious problem in their NOC. I've sent an emergency pager to Mike as well. Hope to get it fixed back soon.

Same with my server too. It seems they're having a serious problem in their NOC.

yes, the whole NOC is down. It is down for about 30 minutes .. correct ? Cause I just checked it, first page opened and the server went down. Or is it down for long time ? I think its a recent accident.

bash# traceroute 216.177.33.125
traceroute to 216.177.33.125 (216.177.33.125), 30 hops max, 38 byte packets
1 216.40.212.1 (216.40.212.1) 0.540 ms 0.443 ms 0.310 ms
2 tayhou-223-4.ev1.net (207.218.223.4) 0.469 ms 0.360 ms 0.260 ms
3 * * *
4 * * *

traceroute to 216.177.33.125 (216.177.33.125), 30 hops max, 40 byte packets
1 rout8rm (143.169.8.254) 0.972 ms 0.814 ms 0.827 ms
2 192.168.1.254 (192.168.1.254) 1.280 ms 1.091 ms 1.066 ms
3 * * *
4 * * *

and down and down it goes..


Goni

http://www.digitallinx.com

Same here... seems to be down about 2 hours now....

:(

Have you tried to page or call Mike? I'm wondering he's aware about this incident and already working with the NOC to resolve it.

Offcourse We have paged him. I am aware of the fact that he must also be worrierd about this and will be working on it. Though many people are trying to get in contact with him thats why it is just useless to call and page him again and again.

As stated [...]I'm wondering he's aware about this incident and already working with the NOC to resolve it.[...]



Goni

Our servers are down as well- I can't reach ANY machine at the NOC, including their own box- Looks like this may be a bit serious.

Fingers crossed it won't last long-

My server seems to be back online now.

network seems to be back...

Everything came back for us about 10 minutes ago- Seems to be a bit faster too-

for me, from Germany (T-Online) the route to webauthorities.com changed ....
Formerly it went through UUnet ... now it's going via Sprint.
Formerly i had 16 hops, now i just got 9, so that has really improved.

1 62.225.255.225 (62.225.255.225) 27.321 ms 25.574 ms 25.296 ms
2 62.225.255.226 (62.225.255.226) 25.582 ms 25.585 ms *
3 Vienna-gw12.USA.net.DTAG.DE (62.156.131.190) 119.938 ms 119.485 ms 118.596 ms
4 sl-gw20-rly-2-0.sprintlink.net (160.81.255.9) 123.015 ms 119.474 ms *
5 sl-bb20-rly-3-2.sprintlink.net (144.232.14.29) 139.258 ms 186.589 ms 156.588 ms
6 sl-dr10-res-9-0.sprintlink.net (144.232.19.122) 123.741 ms 124.372 ms *
7 63.173.24.174 (63.173.24.174) 138.564 ms 137.739 ms 122.167 ms
8 208.1.115.201 (208.1.115.201) 124.472 ms 122.197 ms 151.000 ms
9 216.177.33.229 (216.177.33.229) 127.936 ms 123.305 ms 124.444 ms

Looks _really_ nice for a trace from Germany!

But maybe just the main UUnet pipe failed, so maybe that's just a backup or sth. but if, the backup is much nicer then the normal one ;-))

Greets from Germany

I'm still getting updates from the NOC. Seems like DNS on their end failed. It should be back to normal. When I find out more, I'll post a message. Figures, the one night I turn off my cell phone to charge it is the second time since we been there (over a year now) that the entire NOC went down.

NOTE TO SELF...Don't EVER turn of the cell phone. :D

Well I'm not quite sure on this but this is what I think...

For the past 2 weeks after the first time the NOC went down there was no more direct UUNET connectivity. Everything was being routed through Sprint. I was a bit bitter about this but seeing how Sprint's connectivity was pretty good I didn't say anything.


[paul@cheater paul]$ traceroute www.sprint.net
traceroute to www.sprint.net (199.0.233.22), 30 hops max, 38 byte packets
1 v55 (216.177.32.1) 29.491 ms 222.829 ms 0.787 ms
2 216.177.35.244 (216.177.35.244) 219.347 ms 0.279 ms 0.268 ms
3 500.serial1-4.gw3.tco1.alter.net (157.130.48.237) 352.753 ms 338.633 ms 348.231 ms



Now it seems as if the routing is going only though UUNET and the line seems to be a bit saturated or the router is going crazy.

Anyhow, I think that they must have switched off the BGP and are routing just through UUNET right now and before last night they were routing only through Sprint. Who knows, but I seriously doubt it was a "DNS Error" on their part.

Anyone else have thoughts on this??

-Paul

You can still use BGP-4, but use "preferencing" to send traffic to cheaper connections. Sprint is cheaper, so you would think they'd send more traffic to Sprint... Whatever they are doing up there looks a little screwy to me (no offense intended).

I take it you have space with them too Paul? For me, I'm trying to get them off of sprintlink and preference us onto QWEST and UUNET. However, I can't complain, the sprintlink speeds have been pretty good. Average 80ms using pingplotter.

While I agree there are some things to Iron out, overall I am VERY satisfied with MAE. Been there over a year and only 2 down times in that year...pretty impressive.

Originally posted by MikeA
I take it you have space with them too Paul? For me, I'm trying to get them off of sprintlink and preference us onto QWEST and UUNET.

The only way I can see them doing this for you is if you get your own ASN and IP addresses. Unless you are trying to get them to make the change for all of their customers, which they could do globally with their ASN with no problems.

Yea I have a server at MAE Dulles. My machine has been there so far for 183 days (uptime from my server and it hasn't gone down). The service has been pretty good, I wish they'd just notify the colo customers of major network upgrades in the future. The datacenter is about 5 minutes from my house so I don't mind if they slip up here and there ;-)

-Paul

Originally posted by PaulRoberts
Yea I have a server at MAE Dulles. My machine has been there so far for 183 days (uptime from my server and it hasn't gone down). The service has been pretty good, I wish they'd just notify the colo customers of major network upgrades in the future. The datacenter is about 5 minutes from my house so I don't mind if they slip up here and there ;-)

-Paul

That's pretty impressive! So do you mean that the last downtime was because of a major network upgrade? What did they upgrade? Do they still use their BGP4 router to switch to the fastest route?

Anyway, I notice that my server is now routed thru Sprintlink again, instead of UUNet. What happenned Mike?

Originally posted by jkehe


The only way I can see them doing this for you is if you get your own ASN and IP addresses. Unless you are trying to get them to make the change for all of their customers, which they could do globally with their ASN with no problems.


Greetings Jeff,

I think they can just get the ASN and ask their provider to announce and chop off a /24 and preference it to whatever they want.


elijaH:)

Originally posted by twrs

[...]
Anyway, I notice that my server is now routed thru Sprintlink again, instead of UUNet.

Seems as they switched back to normal, as the trace is going through UUnet again ....

Nice weekend for all you hosting guys out there ;-)

Greets from Germany

Originally posted by twrs
Anyway, I notice that my server is now routed thru Sprintlink again, instead of UUNet. What happenned Mike?

Remember BGP means what the router thinks is the quickest way for the packet to get to you, not what network is faster. :)

Again, they have sprintlink as one of their backbones, which is ok, in general the hops through them seem to average 80ms for me. I was told that not much traffic is going through sprintlink, but I'm still talking to them.

Kind of off topic (okay totally off topic) but I can ping www.zope.org from my server with WebAuthorities and get about 5 ms! :D

Look:PING zope.org (63.102.49.33): 56 data bytes
64 bytes from 63.102.49.33: icmp_seq=0 ttl=245 time=4.917 ms
64 bytes from 63.102.49.33: icmp_seq=1 ttl=245 time=4.856 ms
64 bytes from 63.102.49.33: icmp_seq=2 ttl=245 time=5.016 ms
64 bytes from 63.102.49.33: icmp_seq=3 ttl=245 time=4.958 ms
64 bytes from 63.102.49.33: icmp_seq=4 ttl=245 time=4.909 ms
64 bytes from 63.102.49.33: icmp_seq=5 ttl=245 time=5.164 ms
64 bytes from 63.102.49.33: icmp_seq=6 ttl=245 time=4.898 ms
64 bytes from 63.102.49.33: icmp_seq=7 ttl=245 time=5.266 ms
64 bytes from 63.102.49.33: icmp_seq=8 ttl=245 time=4.993 ms

I like it I like it! Zope is just a really cool Python based application server.

Scott

Originally posted by MikeA

Remember BGP means what the router thinks is the quickest way for the packet to get to you, not what network is faster. :)

Again, they have sprintlink as one of their backbones, which is ok, in general the hops through them seem to average 80ms for me. I was told that not much traffic is going through sprintlink, but I'm still talking to them.

I think you're right. A traceroute from my computer to my server ran through Sprintlink, but a traceroute from a location in the US ran through UUNet. I'm glad that BGP is now working well on the NOC.