I wished I could do without WebhostingTalk to contact Reflexhost.....but I think there's no other way :(

What is the point is that server I'm on has (in the last 2 days) a huge serverload, sometimes above 30 and allways above 8. Therefore the site is slow as hell. The cause of this load is one or probably more scripts. Biggest problem: the load gets higher every hour.

Of course I already mailed them a couple of times in the last 40 hours and asked them to terminate these scripts, because they are seriously damaging their business and also mine. They have't replyed to these mails and because Reflexhost can be found in these forums sometimes, I thought the best way is to post this thread and wait for a reaction.

If you have SSH access, can you try running the command 'top' and paste the output?

Hehe i had similar problems a few months ago, was the result of a DoS attack..

Hopefully Reflexhost isn't under attack.

If so, my sorrow goes to them :(

Hi Marc!

Here's the output:

5:05am up 9 days, 8:08, 1 user, load average: 35.01, 28.98, 24.64
421 processes: 398 sleeping, 18 running, 4 zombie, 1 stopped
CPU states: 1.7% user, 4.7% system, 0.4% nice, 2.6% idle
Mem: 249736K av, 240300K used, 9436K free, 311852K shrd, 5128K buff
Swap: 265032K av, 207896K used, 57136K free 20268K cached

PID USER PRI NI SIZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND
32005 nobody 18 0 3264 3264 1220 R 0 4.9 1.3 0:02 topman.cgi
31835 nobody 15 0 3216 3216 3172 S 0 4.6 1.2 0:02 topman.cgi
32162 wwwbuyi 11 0 1104 1104 656 R 0 2.5 0.4 0:02 top
32280 nobody 11 0 2260 2260 1012 R 0 2.2 0.9 0:01 topsites.cgi
29884 nobody 14 0 5372 2832 2580 D 0 2.0 1.1 0:00 httpd
32038 nobody 5 0 2956 2956 1080 S 0 2.0 1.1 0:00 topman.cgi
4 root 20 0 0 0 0 RW 0 1.9 0.0 9:28 kswapd
3935 djmmx 11 0 23216 84 36 R 0 1.9 0.0 411:59 neomail.pl
5478 djmmx 12 0 2892 80 36 R 0 1.9 0.0 412:01 neomail.pl
31698 nobody 10 0 3360 3360 1224 R 0 1.9 1.3 0:02 topman.cgi
32085 nobody 11 0 2988 2988 2268 R 0 1.7 1.1 0:00 topman.cgi
32555 nobody 13 0 6128 3484 1676 D 0 1.6 1.3 0:11 httpd
32044 nobody 19 0 2984 2984 2936 S 0 1.6 1.1 0:01 topman.cgi
32094 nobody 11 0 3332 3332 1224 R 0 1.6 1.3 0:01 topman.cgi
32188 nobody 10 0 2980 2980 2268 S 0 1.6 1.1 0:01 topman.cgi
32243 nobody 15 0 2976 2976 2932 S 0 1.6 1.1 0:00 topman.cgi
32257 nobody 10 0 3000 3000 1072 S 0 1.6 1.2 0:01 topman.cgi

Could use a reboot :D

Originally posted by Matlok
Hi Marc!

Here's the output:

5:05am up 9 days, 8:08, 1 user, load average: 35.01, 28.98, 24.64
421 processes: 398 sleeping, 18 running, 4 zombie, 1 stopped
CPU states: 1.7% user, 4.7% system, 0.4% nice, 2.6% idle
Mem: 249736K av, 240300K used, 9436K free, 311852K shrd, 5128K buff
Swap: 265032K av, 207896K used, 57136K free 20268K cached

PID USER PRI NI SIZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND
32005 nobody 18 0 3264 3264 1220 R 0 4.9 1.3 0:02 topman.cgi
31835 nobody 15 0 3216 3216 3172 S 0 4.6 1.2 0:02 topman.cgi
32162 wwwbuyi 11 0 1104 1104 656 R 0 2.5 0.4 0:02 top
32280 nobody 11 0 2260 2260 1012 R 0 2.2 0.9 0:01 topsites.cgi
29884 nobody 14 0 5372 2832 2580 D 0 2.0 1.1 0:00 httpd
32038 nobody 5 0 2956 2956 1080 S 0 2.0 1.1 0:00 topman.cgi
4 root 20 0 0 0 0 RW 0 1.9 0.0 9:28 kswapd
3935 djmmx 11 0 23216 84 36 R 0 1.9 0.0 411:59 neomail.pl
5478 djmmx 12 0 2892 80 36 R 0 1.9 0.0 412:01 neomail.pl
31698 nobody 10 0 3360 3360 1224 R 0 1.9 1.3 0:02 topman.cgi
32085 nobody 11 0 2988 2988 2268 R 0 1.7 1.1 0:00 topman.cgi
32555 nobody 13 0 6128 3484 1676 D 0 1.6 1.3 0:11 httpd
32044 nobody 19 0 2984 2984 2936 S 0 1.6 1.1 0:01 topman.cgi
32094 nobody 11 0 3332 3332 1224 R 0 1.6 1.3 0:01 topman.cgi
32188 nobody 10 0 2980 2980 2268 S 0 1.6 1.1 0:01 topman.cgi
32243 nobody 15 0 2976 2976 2932 S 0 1.6 1.1 0:00 topman.cgi
32257 nobody 10 0 3000 3000 1072 S 0 1.6 1.2 0:01 topman.cgi

Yes, it sure could. But then....It seems I can't reach Reflexhost, they're not responding to my emails :(

They contacted me and they're fixing it. Topic can be closed :)

It also looks like they need to double+ their ram. Look at the swap usage. The server should barely be touching the swap. The slowness you are experiencing is greatly caused by that swap usage, especially if they are running IDE drives (and if they're 5400rpm, I'm sorry....).

Hmmm....they still haven't solved this problem. The serverload is now 50,09 (!!!!!!) , memory used is 90,4% and swap is 0,179% .
I seriously think this server will explode in a couple of hours. Hope they made a backup of the sites.

Like I said, if they double their ram (at least) their problems should go way down or go away. If it's writing to swap THAT much, everything slows wayyyy down. 200MB in active swap usage is crazy. Even for fast SCSI disks. Perhaps they ordered more ram and are now waiting to install it? :)

Just looks like topman.cgi is causing the problem, just need to suspend the account doing that.

Originally posted by Matlok
Reflexhost!!!!! - please read this.....

I wished I could do without WebhostingTalk to contact Reflexhost.....but I think there's no other way :(

They have't replyed to these mails and because Reflexhost can be found in these forums sometimes, I thought the best way is to post this thread and wait for a reaction.

While WHT exists for various reasons, and we'd like to help when we can, please note that posting a temporary problem, such as high server load, downed server, etc., and waiting for them to see it here and react, is not what we'd like to have happen.

I realize people don't know what else to do, but to sum it up: If this is a one time problem, or one that doesn't happen often, your best bet is to contact the host and wait for a fix. If contacting the host is often impossible, or problems persist, then I'd look for another provider.

Ok. I understand. Won't do it again. My excuses for this.

I also want to say that the problems are solved and everything is back to normal.

I do believe the problem will come again ... They do need a memory upgrade, until then, the swap disks will be used heavily (200MB swap was used), and the server will be slow at times ...

cheers,
:beer:

I was having the same problem, and it was resolved...but then the server died or something, it's been down for 3-4hours now...

I think it's a ddos attack but i'm not sure...I've heard some stuff about some dude

Or, when the server runs out of memory and runs out of swap, it locks up. I'm not saying it's not a DDOS, though...

good point, but the server load had gone down to a minimum again...:)

You can have a low load and still run out of memory (a good example would be a java server running many JVMs which can take 100+ MB of memory each and use very little CPU when idle).

Is it me or has reflexhost been down for most of the day today? I couldn't reach my sites around 3 pm today using the domain addresses, I went out, just got back, and still can't reach them using the domains. However, I can reach them using their IPs. Anybdoy know if they are having a DNS problem (in addition to the other things that ail them)?

My experience with a DoS is similar to whats happening with Reflexhost, the attacker would attack ever 2 hours or so, then stop and re-attack.

It was a pain, thats why you should always set a filter up.

The problem with reflexHost was that one account holder was running neomail.pl (twice) for over 7 hours which each process ate up about 2.0 to 8.0 of the cpu, now since that account holder was running it twice you would have to double that. Another account had a eggdrop running which wasn't eating much of the cpu but it would be about 1.0 to 2.8 of the cpu every jump. Then there was the big resource hogger which was the accoutn that was ruinning topman.cgi and topsites.cgi. That account had it running 18+ times all at the sametime the script either wouildn't kill itself or it was being run way to many times by different visitors for the site. With these 3 sites joined you can imagine the resource that was being taken up.

Now with them being down for what reason? I don't know, I wish I knew. My site is now down and I am hoping it won't be down for long.

Ive been speaking to one of the 3 people who owned reflex host and he says he has resigned. Ive had conversations with him on icq for a few months and i dont think any of them have enough linux knowledge to know how to deal with these kind of issues. If im wrong then I apologise to Reflex Host.

wont it be easier just killing off the pids thats doing topman.cgi?

To clarify:

I resigned from my position at ReflexHost on Thursday, due to a number of reasons. I cannot discuss these in detail, but in brief it was because of a major disagreement with one of the directors.

Regarding the Linux expertise, there is the knowledge remaining in ReflexHost to sort the problem out, but due to financial difficulties and remaining rifts in the company, I believe ReflexHost are going to terminate business. I therefore recommend that you look elsewhere for hosting.


Matthew Russell

The remaining Reflexhost "directors" must address this post immediately. In particular, if you intend to shut Reflexhost down, at a minimum please confirm that you will refund all payments made for service that your customers will not receive.

Everyone will recieve a refund according to the TOS.

I already got my refund, sorry to hear that their shutting down though.

??? How hard is it to go to command line:
locate topman.cgi
and then either disable the script or account, or even limit that account's amount of resources available to it?? It really doesnt take any linux knowledge to do that.. If these guys dont know something that simple, then they should find something else to do with their lives instead of hosting.

The reasons for the late disabling of that script was not lack of knowledge. It was a lack of communication on our part.

Ah ok, nevermind my post then....

that topman.cgi needs to be hunt down like bin laden :D

rofl ;)

Originally posted by muppie
that topman.cgi needs to be hunt down like bin laden :D