i read the problems about vdi. Is splashost based at VDI? because they have been down for over a hour now?


---------------------------
no biggy!

SplashHost is with pwebtech/NAC ... My server at Site5/NAC is up, so the problem must be unique to SplashHost.

cheers,
:beer:

Yes we are currently probles with the server yet again. So far we have replaced the hardware and the operating system. I believe it must be something a user is doing to the server, im writing a script at this very moment to log every process that runs on the server to try and track it down.

Does the user gaing root access or he/she just simply causes the server to be down by running resource hog processes or??? Wish you all the best ...

cheers,
:bere:

Hi Alan,

When dya think the server will be up again?
I really hope it is going to be soon because I am about to explode here with the number of clients knocking on my door.
But I know you must be feeling very stressed too.
We'll pull through this, Alan.



Originally posted by SplashHost.com
Yes we are currently probles with the server yet again. So far we have replaced the hardware and the operating system. I believe it must be something a user is doing to the server, im writing a script at this very moment to log every process that runs on the server to try and track it down.

I think as resellers we should keep these sort of discussion among us.

However I do understand why everything is being posted here, Alan, should really (imho) try to keep us more informed (a mailing list?). Especially when the down-time is long ( > 1/2 hour).

This is gonna cost me money though :( (I'm going to have to give some credits).

But I'm waiting till the end of this month, to see if it gets better. I've been with Alan for a while and there have NEVER been any problems untill the past 2 weeks. I still believe he's on top of things and it will work out....

Originally posted by royharyanto
Hi Alan,

When dya think the server will be up again?
I really hope it is going to be soon because I am about to explode here with the number of clients dnocking on my door.
But I know you must be feeling very stressed too.
We'll pull through this, Alan.

perharps Alan (and any host) should setup a separate mailling list/forum on different server or if possible, different server on different network? We are doing exactly like this and it has been working great and our clients like it; VDI was down and our customers can still contact us ...

cheers,
:beer:

I agree, any host should keep their site off the machine they use for reselling. 'Cos let's face it, what's the chance that their machine and the resellers machine go down? (well, if the connection is okay, that is)... 25%... hehe.... four options: both, reseller, host, none, so that's 25% :)

Alan: Set-up a dedicated machine for splashhost.com only. This is what we have done. If another machine goes down, your clients still have access to support. If the splashhost machine goes down, your client's sites are still up.

Originally posted by dektong
perharps Alan (and any host) should setup a separate mailling list/forum on different server or if possible, different server on different network? We are doing exactly like this and it has been working great and our clients like it; VDI was down and our customers can still contact us ...

Good advice. We do the same.

I have full confidence that Alan will see this problem to a successful conclusion. I strongly recommend that SplashHost customers persevere through this temporary disruption. This kind of problem could happen to any host at any time.

Based on some of the hosting nightmares out there, It makes sense to sit tight with someone who, on the whole has a good track record.

We run our own windows based machines, which is a slice of hell sometimes. Our current sysadmin is very good. However we have had some rough patches. But because we stuck with this guy, we now have a good business relationship, and we are treated well - problems are usually resolved fast. However, since basically anyone can write asp, we get bad scripts on occasion. Usually this only requires a reboot, however from time to time the whole damned system gets screwed. This has nothing to do with the sysadmin, period.

Given Alan's track record, I figure that once this problem is resolved, it should be smooth sailing. So I wouldn't be too eager to bug out, just yet.

'Tis the nature of shared hosting......

No reply from Alan by email and he is not logged on to ICQ. Does anyone have a update on this situation?

Hi Alan,

I dont know if this is a coincidence, but I was trying to submit a trouble ticket via your PerlDesk as a customer of mine was having problem. After clicking submit..... your server went down.

As you know, when you tried to pipe my email address through my Perldesk, it brought the server down because the script ended up talking to itself by sending mails from suport to support.

Could PerlDesk be related to this this problem?

Also, as you know, I agree with some of the comments above, SplashHost.com really should be on a seperate server so at least customers can be kept in touch with.

Originally posted by UneedSpace
After clicking submit..... your server went down.

Ahh, so it's your fault! *just kidding*

Just a note to Alan. Keep your chin up buddy. It's never a good time when a server goes pear shaped and you're trying to work out what it is. We had an 'odd box' for a while that started to get progressively worse till it just dropped off the net. No errors could be found anywhere, and we spent a large amount of time rebooting the services on that puppy to keep it alive. In the end the last thing we saw was load roar up over 20 in just over 10 minutes then dead air.

Took a new CPU to get it back online again. Currently it been up for over 2 months without reboot and hasn't skipped a beat.

I'd suggest a long nap once that one of yours is back in the swing of things (and a few cool 'beverages' if you're of legal drinking age :D )

Greg Moore

He everyone,

I would like to thank you all for be so understanding and patient in this situation.

The Splash Host server has been crashing almost every day for the last 3 weeks, we have replaced the hardware on the server. That made no difference. We then replace the operating system and that still made no difference. The server wasnt actually freezing, but in the time it takes to refresh top the load was going from 0.3 to 1000. A couple of the times the high load continued after the server was restarted resulting in some long outages.

Yesterday the same thing happened to my other main server. We believe to now know what the cause of this is, many other hosts would deny that this has ever happened to them, but honesty being my policy i will let you know that 2 of my servers were compromised yesterday. I believe the main splash host server was probably compromised before yesterday which would explain what has been happening on that server.

Jason from pwebtech has been working to get the servers back online as soon as possible, but this has been delayed by cpanel licensing servers being down.

To ensure this doesnt happen again i will be investing in hardware firewalls and hiring a security expert to keep my servers secure. A small price to pay compared to the cost of the last few weeks.

Hopefully Splash Host and my bank balance will make a full recovery from this. ;)

I hope no other hosts ever here have to go through what im going through.

So sorry to hear that, but I'm sure your honesty and the reputation you seem to have built up will ensure your company & bank balance bounce back in no time :)

Some people complain about problems, but I think quite often it is the way a problem is resolved that says more about a company than the problem itself!

Anyway, your honesty is refreshing and much appreciated!

By the way, many admins who are working with both Linuxes and FreeBSD say that FreeBSD is better from security point of view.

Splashhost is an excellent Host in my opinion, although there were some minor problems which can happen to any Host, Alan has already mentioned steps to rectify it. I have a VDS account, Dedicated Server pending, 3 resellers account( one of them is Splashhost)

I would highly recommend Splashhost even though the past 3 weeks was rather bumpy like a yo-yo but the most important is.... Alan always does his best and that really is something which not many people has that kind of optimistic character.

Thanks Cactus

Sergio, FreeBSD is more secure, but i dont think whm works on it which is the basis of my business.

We then replace the operating system and that still made no difference.

That's not entirely true.... the new OS install has freetype support enabled and now my counter images display properly! :) (Silver lining in every cloud.)

To any potential SplashHost customers out there -- Alan has always been very responsive and up front with his clients. This is the first problem I've encountered while hosting with him and I'm sure it will be resolved as soon as humanly possible. Things are usually rock-solid and reliable. Malicious user aside, there wouldn't be a problem now.

The Splash host server has bee compromised again, we have taken the other server at pegasus offline as well until we can find out who/how this is being done.

Alan,

I understand these things happen and I appreciate your honesty. A suggestion, contact all your resellers and reset their passwords. Have them do the same for their clients. Then check for rootkits, etc.

Some people just suck.... so if you want to start a mob to track down whoever did this... well, call me :)

We have already checked for root kits etc.. I personally read through 100+ bash history files on one of the servers to find nothing. I cant get in the servers from here just now, the techs from pegasus are working on them.

Originally posted by SplashHost.com
Thanks Cactus

Sergio, FreeBSD is more secure, but i dont think whm works on it which is the basis of my business.

Take a look into H-Sphere as an option. It's one of the best (not look) CP out there that will work with FreeBSD, Linux, as well as Windows. HostGUI (though not available yet) also supports FreeBSD although you may need to wait until it's a proven stable control panel ...

cheers,
:beer:

Ehm, seems everything is back up.... care to share how the server was hacked? (after you fixed it of course :) )

Originally posted by SplashHost.com
We have already checked for root kits etc.. I personally read through 100+ bash history files on one of the servers to find nothing. I cant get in the servers from here just now, the techs from pegasus are working on them.

Hey guys I am trying my very best to control my emotion now but all I wanna say to the person who has been compromising Splash Host's server

" I'll fry your #### if I ever get my hands on you "

Some people just have too much time that they do not know what to do with it.

It's now 6 AM in my timezone. I have been refreshing this page every now and then for the past 20 hours. Have not slept the whole day yesterday. Going to call every one of my clients to explain if they are still willing to listen. Have 2 functions to attend later in the afternoon up to evening. I hope I can maintain my sanity.

Before I lose it I wanna thank Alan for updating me about whats going on. I hope this kind of thing will not happen again and I am positive Alan will try his very best to install security features that kicks every hackers' ass away from the server.

I was wondering if anyone is in worse situation than me.

I know how you feel.... I've decided to give my clients refunds, which sorta sucks... but it's what I feel is right.

Originally posted by royharyanto
" I'll fry your #### if I ever get my hands on you "

Pegasus have said that there is someone trying to kill the server, and there is nothing more they can do.

Ehm, what exactly do you mean Alan?

Originally posted by SplashHost.com
Pegasus have said that there is someone trying to kill the server, and there is nothing more they can do.

Ehm, what exactly do you mean Alan?

At the very least, are we talking DDOS or "other?"

They mean im on my own to fix this, they cant help any more.

A number of self-described Linux geeks have offered here and on the SplashHost forums to help you/us out of this. I would urge you to take them up on their offers right away.

-- Jon

That's a very disappointing answer from your managed server provider (was it pwebtech?).... I realise they can't work on it forever, but a comprised machine is serious stuff... not only for your machine and clients, but others as well... (think: DOS attack).

Sigh... this sucks.

Originally posted by SplashHost.com
They mean im on my to fix this, they cant help any more.

The server is up now. but no database connectivity.either mysql daemons are not started or it is using a port which is closed.any comments?

I received an email from PayPAL:


PayPAL: INVALID
An invalid response was returned from PayPal. This means that someone has discovered confirm.php and is trying to execute it in order to create accounts without paying.

User's IP: 24.234.17.120


I did some checking and the results showed the culprit's info:

Address lookup
canonical name cm120.17.234.24.lvcm.com.
aliases
addresses 24.234.17.120


Domain Whois record
Querying whois.internic.net with "dom lvcm.com"...

Whois Server Version 1.3

Domain names in the .com, .net, and .org domains can now be registered
with many different competing registrars. Go to http://www.internic.net
for detailed information.

Domain Name: LVCM.COM
Registrar: NETWORK SOLUTIONS, INC.
Whois Server: whois.networksolutions.com
Referral URL: http://www.networksolutions.com
Name Server: PRIME-NEWS.LVCABLEMODEM.COM
Name Server: PRIME-BE1.LVCABLEMODEM.COM
Updated Date: 12-nov-2001


>>> Last update of whois database: Thu, 7 Feb 2002 17:11:08 EST <<<

The Registry database contains ONLY .COM, .NET, .ORG, .EDU domains and
Registrars.


Querying whois.networksolutions.com with "dom lvcm.com"...

The Data in Network Solutions' WHOIS database is provided by Network
Solutions for information purposes, and to assist persons in obtaining
information about or related to a domain name registration record.
Network Solutions does not guarantee its accuracy. By submitting a
WHOIS query, you agree that you will use this Data only for lawful
purposes and that, under no circumstances will you use this Data to:
(1) allow, enable, or otherwise support the transmission of mass
unsolicited, commercial advertising or solicitations via e-mail
(spam); or (2) enable high volume, automated, electronic processes
that apply to Network Solutions (or its systems). Network Solutions
reserves the right to modify these terms at any time. By submitting
this query, you agree to abide by this policy.


Registrant:
Prime Cable - Las Vegas (LVCM-DOM)
121 South Martin L. King Boulevard
Las Vegas, NV 89106
US

Domain Name: LVCM.COM

Administrative Contact, Technical Contact, Billing Contact:
Fountain, John (JF4071) John.Fountain@COX.COM
121 S. Martin Luther King Blvd.
Las Vegas, NV 89106
702-384-8084 (FAX) (702) 383-7048

Record last updated on 20-Dec-2001.
Record expires on 28-Oct-2005.
Record created on 27-Oct-1997.
Database last updated on 7-Feb-2002 08:41:00 EST.

Domain servers in listed order:

PRIME-BE1.LVCABLEMODEM.COM 24.234.0.5
PRIME-NEWS.LVCABLEMODEM.COM 24.234.0.7


Network Whois record
Querying whois.arin.net with "24.234.17.120"...

Community Cable TV (NETBLK-PRIMECABLELV) PRIMECABLELV
24.234.0.0 - 24.234.255.255
Community Cable TV (NETBLK-EXPRESS-LASVEGAS) EXPRESS-LASVEGAS
24.234.1.0 - 24.234.95.255

To single out one record, look it up with "!xxx", where xxx is the
handle, shown in parenthesis following the name, which comes first.

The ARIN Registration Services Host contains ONLY Internet
Network Information: Networks, ASN's, and related POC's.
Please use the whois server at rs.internic.net for DOMAIN related
Information and whois.nic.mil for NIPRNET Information.

Querying whois.arin.net with "!NETBLK-EXPRESS-LASVEGAS"...

Community Cable TV (NETBLK-EXPRESS-LASVEGAS)
121 S. Martin L. King Blvd
Las Vegas, NV 89106
US

Netname: EXPRESS-LASVEGAS
Netblock: 24.234.1.0 - 24.234.95.255

Coordinator:
Fountain, John (JF4071-ARIN) John.Fountain@cox.com
702-384-8084x481 (FAX) (702) 383-7048

Record last updated on 02-Mar-2000.
Database last updated on 6-Feb-2002 19:56:46 EDT.

The ARIN Registration Services Host contains ONLY Internet
Network Information: Networks, ASN's, and related POC's.
Please use the whois server at rs.internic.net for DOMAIN related
Information and whois.nic.mil for NIPRNET Information.

DNS records
name class type data time to live
cm120.17.234.24.lvcm.com IN A 24.234.17.120 3599s (59m 59s)
lvcm.com IN MX preference: 1
exchange: inbound-mail.lvcm.net
60s (1m)
lvcm.com IN MX preference: 10
exchange: inbound-mail.lvcablemodem.com
60s (1m)
lvcm.com IN MX preference: 100
exchange: smtp.lvcablemodem.com
60s (1m)
17.234.24.lvcm.com IN SOA server: ns1.lvcablemodem.com
email: administrator@lvcablemodem.com
serial: 12
refresh: 3600
retry: 600
expire: 604800
minimum ttl: 3600
3600s (1h)
120.17.234.24.in-addr.arpa IN PTR cm120.17.234.24.lvcm.com 3600s (1h)
17.234.24.in-addr.arpa IN NS ns1.lvcablemodem.com 3600s (1h)
17.234.24.in-addr.arpa IN NS ns2.lvcablemodem.com 3600s (1h)
17.234.24.in-addr.arpa IN SOA server: ns1.lvcablemodem.com
email: administrator@lvcablemodem.com
serial: 12
refresh: 3600
retry: 600
expire: 604800
minimum ttl: 3600
3600s (1h)

Service scan
FTP - 21 Error: Connection refused
SMTP - 25 Error: Connection refused
HTTP - 80 Error: Connection refused
POP3 - 110 Error: Connection refused
NNTP - 119 Error: Connection refused

Traceroute
Tracing route to cm120.17.234.24.lvcm.com [24.234.17.120]

hop rtt rtt rtt ip address fully qualified domain name
1 0 0 0 216.46.228.241 port-216-3073265-dal16509b-drtn.devices.datareturn.net
2 0 0 0 64.29.192.237 port-64-1949933-zzt0prespect.devices.datareturn.net
3 0 0 0 64.29.192.226 port-64-1949922-zzt0prespect.devices.datareturn.net
4 0 0 0 209.246.152.201 gige3-0-101.ipcolo2.dallas1.level3.net
5 0 0 0 209.244.15.101 gigabitethernet11-0.core2.Dallas1.Level3.net
6 0 0 0 209.247.10.109 so-4-1-0.mp2.dallas1.level3.net
7 30 30 30 64.159.0.249 so-2-0-0.mp2.losangeles1.level3.net
8 30 30 40 64.159.1.205 gige8-2.hsipaccess1.losangeles1.level3.net
9 30 40 40 166.90.145.14 unknown.level3.net
10 40 30 40 24.234.1.233 1000m.e2-8.bi8.cr.lvcm.com
11 40 40 41 24.234.1.201 1000m.e2-1.bi8.he.lvcm.com
12 60 70 50 24.234.17.120 cm120.17.234.24.lvcm.com

Trace complete

-- end --
URL for this output

The person is trying to setup an account on Splashhost's server where I am a reseller. Anyway I have earlier(1 week ago) disabled PHPManager(link to 2checkout) and also direct 2checkout sign-up when the server was showing erratic performance 2 weeks ago which Alan(Splashhost) informed resellers by email. It seem someone is dead set in causing problems or targeting Splashhost's server(s) for some reasons unknown.

I hope Alan catches the culprit soon...

cactus, did you forward this to Alan?

alan already started the mysql server a few minutes back, right after i mentioned it to him

I'd forward that, along with any time information from PayPal, to the technical contact listed. It looks someone on a Cox Cable network cable modem is the culprit....hopefully they can track him down and nail his ass to the wall...with big nails. :)

Can they be big RUSTY nails? :)

Originally posted by drose25
I'd forward that, along with any time information from PayPal, to the technical contact listed. It looks someone on a Cox Cable network cable modem is the culprit....hopefully they can track him down and nail his ass to the wall...with big nails. :)

I believe ive had a customer whos hostname had lvcm.com i can remember seeing it a long time ago in human click. Is there away i can search through e-mails in outlook to see if that comes up?

Originally posted by dhabets
cactus, did you forward this to Alan?

No, I didn't informed Alan as I was not sure whether it's related to Alan's problem. Now that things are getting very serious on Splashhost's server(s) I thought maybe as resellers, whatever I info we get , it's only right to pass it on to Splashhost. The email just came in a few hours ago so I hope Alan reads this or I may have to inform him by Mail. I hope every reseller of Splashhost takes extra care until the problem is solved or the culprit(s) eliminated totally from Splashhost's server(s)

Try running 'last |grep <ip>' or 'last |grep cm' for the cm120.17.234.24.lvcm.com hostname. See what pops up.


Course, that's only if the culprit is on your machine (a user of yours). If not, check your log files, and the log files of that person whose confirm.php was being used.

Originally posted by SplashHost.com
I believe ive had a customer whos hostname had lvcm.com i can remember seeing it a long time ago in human click. Is there away i can search through e-mails in outlook to see if that comes up?

setup a filter/rule for lvcm.com. should work fine.

I have found out who that customer is, his ip number came up on my newest server ftp logs. He was originally on splash host server and asked to be moved to the new server. I dont believe this can be him though, ive know him for ages before i even started splash host

Originally posted by SplashHost.com
I have found out who that customer is, his ip number came up on my newest server ftp logs.

Alan, what does this customer say about the confirm.php and the paypal warning though? even though he's not the person causing trouble, the paypal warning is pretty clear.

WITCH hunt!.... :D

It bears mentioning that hacking servers subjects the hacker to federal criminal liability -- including jail time -- and severe civil penalties/damages. As to civil damages, although I rarely take cases on a contingent basis, I'd be inclined to make an exception for this one. As to criminal enforcement, its easy enough to file a report with the FBI, see http://www.nipc.gov/incident/incident.htm .

Here's hoping the culprit is identified soon.

What kind of attack is being made against Superfastserver?

How?

Originally posted by dhabets
That's a very disappointing answer from your managed server provider (was it pwebtech?)....

Who says it's a managed server? In fact, I don't think Pweb offers a managed server ... Managed server would be much more expensive ...

cheers,
:beer:

dektong,

if it's not then I have to say that Pwebtech (that's who alan is with, right?) have been exceptional. Either way though, a compromised server is not just troublesome for the owner of that server, but also for other people. again, think of a dos attack.

oh, beer! :stickout

I checked pwebtech's page and yep, doesn't seem to sell managed servers... allright :) I'm an idiot.

later!



Originally posted by dektong


Who says it's a managed server? In fact, I don't think Pweb offers a managed server ... Managed server would be much more expensive ...

cheers,
:beer:

Im afraid the splash host server will be offline for most of the night, we are resorting to switching on the services one by one until we can find what crashes it, apache being the last one to be switched on.

Well sometimes things can go wrong that you are on the brink of going nuts. When te going gets tough the tough gets going.

The culprit will get his ass fried.

I`ve now hit the whisky and am contemplating suicide :(

As a new starter, I have obviously hit a bad period and customers are taking out contracts of $500,000 on my life and im wondering now what the weather is like in the antartic? :D Gawd, I didnt even realise I was worth $500,000 till I signed up with Splash! You MUST be good if the contracts were only worth $100 when I was with DonHost :D

Unfortunately for myself, not only have I had to refund my customers, but I have had to set up a new account to move them to when the server is back up and I can download thier sites.

Alan, my heart cries out for you and all your other customers who have stuck by you through thick and thin. I personally would like to get hold of this t**t that has attacked the server and ram a few gigawatts up his u know where.

Your reputation is incredible here and your support for me so far has been exempilary, what a star! :)

I hope, and am sure you will get through this, and even though I may have to move my customers, I hope to keep my account going as long as possible for new customers.

I for one have NEVER seen so much support on a forum and you deserve it all. Here`s wishing we can be all back online soon... hic ;) keep your chin up mate!

Now where did I put that other bottle o whisky and the sleepin pills :D

Hey you need space,
Im in the same situation as you. So do everyone.
I lost quite a sum of money but the worst part is actually losing the absolute confidence my clientele. They used to think we are rock solid guys who knows how to handle everything.
Now they might think were just some sissy bull****ter.
We have to lose in order to gain.
Ive refunded money for most of my clients.
Hoping that they know were not the type who run away from problem when it arise.

I look like a panda now, which is result from lack of sleep.My Nokia 8850 has been ringing like mad. I think it needs some rest just like I do. I decided to switch it off for the moment.
Might be going to imbibe some Vodka soon.Hopefully the nightmare can end soon ,then I can rebuild the business that I have spent so much time to establish.


Originally posted by UneedSpace
I`ve now hit the whisky and am contemplating suicide :(

As a new starter, I have obviously hit a bad period and customers are taking out contracts of $500,000 on my life and im wondering now what the weather is like in the antartic? :D Gawd, I didnt even realise I was worth $500,000 till I signed up with Splash! You MUST be good if the contracts were only worth $100 when I was with DonHost :D

Unfortunately for myself, not only have I had to refund my customers, but I have had to set up a new account to move them to when the server is back up and I can download thier sites.

Alan, my heart cries out for you and all your other customers who have stuck by you through thick and thin. I personally would like to get hold of this t**t that has attacked the server and ram a few gigawatts up his u know where.

Your reputation is incredible here and your support for me so far has been exempilary, what a star! :)

I hope, and am sure you will get through this, and even though I may have to move my customers, I hope to keep my account going as long as possible for new customers.

I for one have NEVER seen so much support on a forum and you deserve it all. Here`s wishing we can be all back online soon... hic ;) keep your chin up mate!

Now where did I put that other bottle o whisky and the sleepin pills :D

guys, i know what you're all feeling right now. i'm a reseller too and these incidents have affected my business as well. since we're all part of the splashhost community of clients and resellers, let's help alan nail this b**tard.

Yeah lets nail him and put him behind bars once and for all.

well, i've been monitoring the situation for sometime now. indeed this hacker should "HACKED". they now found the problem and is working on it. tell you guys, i can't believe it. it was a FILIPINO who discovered the problem and is helping to fix it now.

Originally posted by sydel
tell you guys, i can't believe it. it was a FILIPINO who
discovered the problem and is helping to fix it now.

Meaning what? :mad:

Alan,
I'm sorry that you're having to deal with this. What a pain. When HostGUi is released I think we will be going woth FreeBSD and Kiwi thanks for the advice that people should get a server for the site itself as well as clients site. I think that is a good idea so they can reach support. However they always have the toll free number if they write it down. Anyways Alan hope things go well and are back up and running normal soon.

Originally posted by dhabets


Meaning what?

Nothing really. I'm just impressed with the Filipino. :)

Who is Filipino? :blush:

Can you tell me what this problem is seeing as you think you have it resolved?

Originally posted by sydel


Nothing really. I'm just impressed with the Filipino. :)

Oh, okay :) .... servers that act up get to me :cool:

FILIPINO is an individual who originates from Philipine( part of Asia) . Filipinos are quite adept in IT just like Indians.

Alan,

This thing has been up against you for some time now. I hope it get's resolved sooner. As always, we're by your side.

Cheers to all :beer:

The Server is working righ now, but perl does not work, apache is showing the code of my scripts, path of the server blah blah blah :eek:

Yes ive disabled cgi and the server ahsnt crashed for 3 hours.

How do we know if the hacker is not gonna come back at us again?

Hi Alan,

the url below has a tool to test CGI security. I recommend taking a look at "Whisker" It requires no compilation or preparation other than simply unpacking the archive(tar zxf whisker.tar.gz)

Whisker has an extensive library of hacks that it will run against your web server and CGIs. Included in the default script database are more than 500 attacks on Apache.

I have not used it but only read it from one of my books on Linux security so I hope it can be of help to restore your ailing server(s)

As CGI is very important and disabling it will only have grave consequence and disadvantages to compete. If it's temporarly, then it's okay but my guess is not to only pinpoint the source that is causing the problem(CGI as suspected) but to eliminated totally and protect the server against future abuse by hackers/runaway CGI scripts on the servers.

url: http://www.wiretrip.net/rfp/p/doc.asp?id=21&iface=2

NOTE AUTHOR's COMMENTS:

I recommend reading through the Whisker documentation to develop you own CGI testing regimen. Whisker can perform a variety of attacks, including brute-force file and username/password detection. Used as a preventative measure against your server, Whisker is an excellent tool. In the hands of an attacker attempting to find vulnerabilities in a web server, it can be equally effective. It's just a matter of who uses it first.

Alan please note:
Search for whisker.pl or whisker.cgi on your servers may help just in case.

Hope this helps

Regards

:(
I just close all my clients account so that it losen the server a bit
hope Alan will fix the problem soon..
but...

ndc: operation failed: verify failure (failed to verify signature)
ndc: reload command failure: verify failure

:(

nvm abt that Alan you have my fullest support.


rgds,
Neo Gim Tiong

Ok, the server has now ran for 11 hours with cgi disabled and hasnt crashed.

I believe its a poorly written or deliberate cgi script doing this. Combined with this bug in 2.4.x kernals

As I reviewed the archives of late December, I found that the per-user limit support in the 2.4 series kernels is broken. With the limit support broken, any user -- privileged or not -- has the potential to suck up all of the machines resources, effectively causing an intramural DoS (Denial of Service) attack. They could do this accidentally, and it would cause a great deal of grief for any system administrator.

Were going to try downgrading the kernal to 2.2.19

Thats good news :)
By the way cron tabs are still disabled ?
Good luck Alan and thanx for keeping us up to date!

Originally posted by royharyanto

FILIPINO is an individual who originates from Philipine( part of Asia) . Filipinos are quite adept in IT just like Indians.



:mad: This sounds just stupid... an individual is good at IT or anything else for that matter... NOT a group of people. (or ethnicity).. I'm sorry, but statements like the one quoted are just dumb.

Originally posted by dhabets


Alan, what does this customer say about the confirm.php and the paypal warning though? even though he's not the person causing trouble, the paypal warning is pretty clear.

WITCH hunt!.... :D

Hey guys! That IP is mine, its funny that I even read this post. I have just been keeping myself up to date. Thats wierd how you got that email or whatever it was cactus. I was messing around with phpManager last night with the billing functions etc. Hmm

Originally posted by guru_ck





Hey guys! That IP is mine, its funny that I even read this post. I have just been keeping myself up to date. Thats wierd how you got that email or whatever it was cactus. I was messing around with phpManager last night with the billing functions etc. Hmm



It's him, get em!!! :D

Btw, guru_ck, I have phpManager as well, how do you like it so far? Do you use the automated setup? It seemed sorta flakey at times...

what is phpManager ?

Originally posted by dhabets

It's him, get em!!! :D

Btw, guru_ck, I have phpManager as well, how do you like it so far? Do you use the automated setup? It seemed sorta flakey at times...


Hehe! I just bought it the other day and at first I had an installation problem. So I posted on there boards and they responded right away! The program does what it should for the low price that I paid for it (20$). I haven't tried the automated setup yet but yesterday when I was messing around with it I created an account and then deleted it and it deleted the username/site that was in WHM. HAHA OPPS!

-Chris

Originally posted by m0h
what is phpManager ?

http://phpmanager.taysoft.com

Maybe guru_ck should run a security check on his PC to make sure a backdoor trojan or similar nasty hasn't made it's way on his machine.

Originally posted by drose25
Maybe guru_ck should run a security check on his PC to make sure a backdoor trojan or similar nasty hasn't made it's way on his machine.

Norton Anti Virus is always on ;) but im probabley going to format soon anyway... xp is getting too slow.

Originally posted by drose25
Maybe guru_ck should run a security check on his PC to make sure a backdoor trojan or similar nasty hasn't made it's way on his machine.

why do you say that?

Hey I am talking about the general population u fool!
Haven't you studied stats in school?

Originally posted by dhabets




:mad: This sounds just stupid... an individual is good at IT or anything else for that matter... NOT a group of people. (or ethnicity).. I'm sorry, but statements like the one quoted are just dumb.

There was questionable activity attributed to his IP address. Someone could have been using his machine remotely without his knowledge using something like BackOrifice or a similar trojan/backdoor type program.

My ip was made public and posted on this board :crap:

Originally posted by royharyanto
Hey I am talking about the general population u fool!

Haven't you studied stats in school?





Yes, I have. I actually have a degree in CS and Math. Anyhow, the way it is worded is that this particular attribute is common for a population, which is just not true.

Filipinos are quite adept in IT just like Indians.

Read it again and you see what I mean. That's the last that I'm saying about this 'cos it's getting too much off topic.

Originally posted by drose25
There was questionable activity attributed to his IP address. Someone could have been using his machine remotely without his knowledge using something like BackOrifice or a similar trojan/backdoor type program.

That's true. Thanks for explaining.

Originally posted by guru_ck

My ip was made public and posted on this board :crap:



Does your cable connection come with a fixed ip? My dsl connection resets the ip address every time I power down the modem. I don't know, but maybe that will work for your cable modem?

I still dont think the sentence is wrong.

Maybe we just have different perception.
Or do you think it is more appropriate if I say
"General population of Philippines has higher IT capabilities and
General population of India has higher IT capabilities than general population of most countries."


Originally posted by dhabets


Read it again and you see what I mean. That's the last that I'm saying about this 'cos it's getting too much off topic.

Originally posted by dhabets

Does your cable connection come with a fixed ip? My dsl connection resets the ip address every time I power down the modem. I don't know, but maybe that will work for your cable modem?

Cox claims its dynamic but its been the same for 5 months. Even if I do unplug my modem and plug it back in its the same. I don't care though.

Cable modems tend to have static IPs.
DSL family modems are the one that tend to change their IPs

Originally posted by guru_ck


Cox claims its dynamic but its been the same for 5 months. Even if I do unplug my modem and plug it back in its the same. I don't care though.

Originally posted by royharyanto


I still dont think the sentence is wrong.







Maybe we just have different perception.



Or do you think it is more appropriate if I say



"General population of Philippines has higher IT capabilities and



General population of India has higher IT capabilities than general population of most countries."

















Yep, that sounds much better :)



Though I don't think it's true.... anyhow, the first wording sounded a bit racey... that's all....

Yeah I do love to generalise.
I always go like
"you gals are always ......"

Anyway I heard somewhere that every Indian kids have to go through programming classes. Its a compulsory subject in India.
Imagine!

Originally posted by dhabets






Yep, that sounds much better :)



Though I don't think it's true.... anyhow, the first wording sounded a bit racey... that's all....

I access via cable modem also (Adelphia) but have a dynamic IP. However, it tends to stay the same IP for several days or weeks until it changes...

Funny how fast the topic changes.

Originally posted by guru-cik
Hey guys! That IP is mine, its funny that I even read this post. I have just been keeping myself up to date. Thats wierd how you got that email or whatever it was cactus. I was messing around with phpManager last night with the billing functions etc. Hmm

I am using phpManager, so I don't know whether you were at my site or somewhere else testing the program/software as you mentioned.

I received the email from PayPal which was generated via phpManager that was diabled when Alan inform resellers regarding their server problem/upgrading.

Anyway, since it's a genuine case and you have stated and confirmed it, then it's nothing to worry about. This helps clear the mystery as we are all trying to help Alan in any way we can, so that he can find out what is causing the servers to crash erratically. I think he has found what was causing the servers to crash and let's hope that everything is back to normal soon....

Regards

Is splashhost again? I was just checking my mail and uploading some files a few minutes ago then "kaboom!" (or is that over-reacting), the server is down again. What a coincidence. I hope it's just minor and that it would be fixed sooner. :crap:

It seems to be down again. I was editing some php config files in a SSH session and testing the results in my browser when it died.... I managed to get an uptime command to finally report and the load was over 800. Sounds like a runaway script or process.

Hopefully Alan has been able to narrow it down some and will get t resolved soon. My clients are getting a little impatient with me. :D

The problem is still persisting.

Originally posted by SplashHost.com
The problem is still persisting.

Geez. This problem is one good example for consistency and persistence. :)

I hope it gets resolved really sooner. I just got my first phone call from a weary client today.

:stickout

Originally posted by royharyanto
Yeah I do love to generalise.
I always go like
"you gals are always ......"

Anyway I heard somewhere that every Indian kids have to go through programming classes. Its a compulsory subject in India.
Imagine!



Yeah, I think it's a good idea in a way... but in other ways I'm thinking that it's better to have a balance between science and arts.... damn... I'm getting way too off topic...

on topic: it seems that splashhost is still down. but I have moved all my accounts elsewhere for now.... we'll see....

Originally posted by sydel


Geez. This problem is one good example for consistency and persistence. :)

I hope it gets resolved really sooner. I just got my first phone call from a weary client today.

:stickout

You don't email your clients when you know the server is down? Hmmm, maybe that's what I'm doing wrong! :D

Originally posted by dhabets
You don't email your clients when you know the server is down? Hmmm, maybe that's what I'm doing wrong! :D

I wish I could but how? They can't receive emails. One thing wrong that I've done is that I didn't get alternative email ads from them in the beginning. :blush:

Originally posted by sydel


I wish I could but how? They can't receive emails. One thing wrong that I've done is that I didn't get alternative email ads from them in the beginning. :blush:

When they sign up don't they give you an email address? Oh, I guess you already answered that... I think I need to sleep.... Anyhow, I require one, 'cos usually if you have an internet connection then you also have an email address! :) (btw, I don't like hotmail or other free email addresses)...

Originally posted by dhabets
When they sign up don't they give you an email address?
Yes they do. That's the problem with me. I tend to junk email ads that are not being used.

Alan,

Is it possible to keep the emails running while you're working with the rest of the server? Most of my clients are complaining only about the e-mail (and that's one of the reasons why they discover that the server is down - they can't check).

Please....:bawling: Please.....

thanks

Sorry, just have to add my opinion...

This doesn't make sense. Larger percentage of people in India and Philipines have NEVER touch a computer, compared to those in most western countries and even other asian countries like japan, hk (china), singapore etc.

You'd just see more IT people from India and China simply because these 2 countries, in combination have nearly HALF of world population, and these people with IT skills are higher in demand in countries with high IT growth (buck lack IT-skilled workers) like in Europe and USA.

I'd think that you'd attribute Filipinos due to "love bug" incident which anybody in the world with too much time on their hands could've unleashed.



Originally posted by royharyanto
I still dont think the sentence is wrong.

Maybe we just have different perception.
Or do you think it is more appropriate if I say
"General population of Philippines has higher IT capabilities and
General population of India has higher IT capabilities than general population of most countries."

Originally posted by Cognigens
Sorry, just have to add my opinion...

This doesn't make sense. Larger percentage of people in India and Philipines have NEVER touch a computer, compared to those in most western countries and even other asian countries like japan, hk (china), singapore etc.

You'd just see more IT people from India and China simply because these 2 countries, in combination have nearly HALF of world population, and these people with IT skills are higher in demand in countries with high IT growth (buck lack IT-skilled workers) like in Europe and USA.

I'd think that you'd attribute Filipinos due to "love bug" incident which anybody in the world with too much time on their hands could've unleashed.


Disclaimer: I just came back from the bars
I somehow agree with you Cognigens, however my initial objections where with the generalizations being made... yes, you're right about the numbers... I think the amount of techies (that'd be us) is fairly equal in ANY society... and I'm fairly sensitive to these generalizations... okay, me must sleep now... beer good though :)

Being an Indian IT guy for the last 13 years and having worked on projects with very diverse IT teams including Americans, British, Japanese, Chinese, Malaysians, Indonesians, Philippines, Fijians, Australians, Russians, Croatians, Germans, Vietnamese, Greeks, Italians, South Americans and ofcourse Indians, and having lived in a few countries I can safely say one thing...

Countries with a high population and somewhat hard living conditions [like India, China, South America or South East Asia] have a tendency to bring out the survival instincts of our intellect including a certain native cunning [very similar to the sharp senses of a jungle person] and countries with very high standards of living like Australia or Scandinavia dulcify your native cunning [Some of my Indian friends' kids in Australia cant do basic math without a calculator..]

And if you go to the root of all analytical and lateral thinking skills it is a certain conniving and convoluted thought process that solves problems [logic is all about the left brain but it's the intution from the right side that triggers the left]

And software is mostly about this skill at the root , primordial thought level

I know this is arguable, and I dont have data to substantiate it but if some anthropologist can take it up and do a PhD on it, it can be proven right or wrong!

As for India, we have over 1 billion people with about 10% of them speaking English [yeah thats 100 million, making India the second largest English speaking country in the world after US !] and maybe 1-3% of these use computers... still a large number and more than the population of many countries!

So its a combination of both numbers and environment

Cheers
Balaji

So...what exactly do these posts have to do with the topic of Splashhost?

J/K

Originally posted by alchiba


Good advice. We do the same.

I have full confidence that Alan will see this problem to a successful conclusion. I strongly recommend that SplashHost customers persevere through this temporary disruption. This kind of problem could happen to any host at any time.

Well said :D

There's no perfect host but a host whether will do their best... counts :)

Regards,
Choon

The cgi on my website does not work. Anyone having the same problem? I could access the homepage but it does not mean much since all of other pages on my website running cgi.

If it's happened to everyone then hopefully it get fixed soon.

Originally posted by dhabets


Yeah, I think it's a good idea in a way... but in other ways I'm thinking that it's better to have a balance between science and arts.... damn... I'm getting way too off topic...

on topic: it seems that splashhost is still down. but I have moved all my accounts elsewhere for now.... we'll see....

But how do you moved them? (I mean databases, pop e-mails)

I may have stopped the crashing now, will need to wait 24 hours to see if i have or not though.

Originally posted by sergio


But how do you moved them? (I mean databases, pop e-mails)

You're out of luck with pop emails, however you wouldn't be receiving any email with the mailserver being down.

I move site files by making a tarball and gzipping it. Databases are done ... manually by exporting them :(

Important site, which are stored redudantly, are just rsync-ed daily between accounts (it only copies changes).

Note, the splashhost server seems to behave now and Alan has had GREAT performance and service for a long time. So don't panic, but just know how to move things should it be necessary.

http://www.shmooze.net/~markjr/mysqlsync/

this syncs mysql databases :)

You know what they say, misery loves company....well, my sites have been up and down for weeks, my customers are complaining their tails off and I look at www.splashhost.com and it's up.

I guess I'd feel better if Alan and I were in the same boat, but right now it seems like I'm sinking alone.

shanda, what do you mean?

Originally posted by SplashHost.com
shanda, what do you mean?

I believe her (?) website http://cprhosting.com/ isn't working.

Originally posted by Kiwi


I believe her (?) website http://cprhosting.com/ isn't working.

Same here

That could be because its her reseller account that was doing this to the server.

could be but I'm sure it's not. If so, remove the account but don't make everyone else suffer because of it. Also, use my e-mail address to say, hey Shanda, stop what you're doing over there.

Actually, I'm surprised I entertained your accusation, as I know it's not me. Spend another sleepless night looking for the solution Alan, I'm not the culprit.

In addition, it's taken one post from me on this forum to help you figure out a problem that's been plaguing your servers for three weeks? Maybe you should put me on payroll.

The suspend on my entire account was really unnecessary, especially without any warning. You have my e-mail address, why not use it?

I just need my files and the files of my customers, and believe me, I will be permanently out of your hair.

Anything I supposedly did was not intentional. Maybe if you would have said something it wouldn't of had to go down like this.

:angry: :angry: :angry: :angry: :angry: :angry: :angry: :angry:

I agree

:angry:

shanda this is a malicious attack on me, as far as im concerned ive got every right to suspend your whole reseller account. Normally i would just suspend the one account but in this case i could not take the risk that it was the actual reseller doing this. There was a reason i had not e-mailed you, some times in these situations i find if i dont contact the account owner then they will realise what has happened to their account and move without mentioning it knowing they have been caught. If they are not doing it deleberately they are normally quick to be asking whats wrong with their site.

Also shanda you will notice that the problem was already resolved before you posted.

So is it over or what?

Malicious attack? Alan, i don't even know you. That being said, I am your customer. Remember that word, CUSTOMER. You've completely taken my business down without any reason or explanation as to why.

You sent an e-mail out Thu, Feb 07 (4 days ago) saying that superfastserver and extrafastserver was compromised. Why would I contact you to say that my site was down when you acknowledged there was a problem in the e-mail? I trusted you would get things up and running in a few days.

Then yesterday, I just happened upon this forum and posted my original message. You then turned around and attacked me saying that I was the root of your problems. And I was like, wtf? I'm sitting here thinking that my pages would be up eventually, and this guy's got me down permanently.

So, Alan, if you think I acted maliciously, oh well. Just give me my files.

"So, Alan, if you think I acted maliciously, oh well. Just give me my files."

Surely this is admitting guilt?

I completely understand both sides of the story and I would have to side with Alan on this one.

If you look at the logistics of it, 1 $xx.xx account is causing the server (unethically) to crash....and crash.....and crash.....and crash again....resulting in 2-3 hardware replacements

As a hosting company, why should he risk thousands of dollars in revenue a day for 1 $xx.xx account?

Sure, there's a point where customer service is a factor, but that just goes to a certain point, when a repetitive action is happening on the part of a customer unknowingly or knowingly causing these actions, drastic actions must be taken to employee the safety and satisfaction of other customers

Every day we are forced to kick off customers that we consider as High Resource, One site was taking in over 75,000 page views a night, causing a server to overload and die, resulting in us loosing quite a few customers over a weeks time.

Rather then risk more customers walking away, we suspended the site, amazingly the server crashes stopped....

either way, hope you find a new host that you will be satisfied with and hope this very long thread with splash being up in waves can end :)

Shanda ive switched your reseller account back on accept for the offending account.

In the cgi-bin of cprdemo was a fork bomb that had been disguised as a formmail script. That was the cause of the local DOS attacks

Admission of guilt? No. But if he found out that something that was going on with one of my accounts was the problem, then for that I will apologize. It was not intentional, I promise.

How can I admit to something when I still don't know what in the heck the problem?

Realize, ALL of my sites were/are down too.

Alan,

I set up cprdemo in the same way you had it set up on your site. Sorry it caused problems. Glad that you found/fixed the problem. Sorry to the other customers of Splash Host who had to suffer.

As a host, I just ask that in the future, you at least send an "account suspended" e-mail to the culprit.

I too would have to side with Alaskanwolf and Alan on this one..
I've been in situations that it was a better choice to suspend the account that was having the problems, even if it was a reseller. Remember shanda, splashhost was shooting in the dark on who exactly was causing the problem. It takes many frustrating hours to determine a problem like that one, especially when dealing with numerous servers and accounts.

Bottom line, Splash Host had every reason to put your account on hold. Let me refresh your memory about the Terms and Conditions on their site:

"Splash Host reserves the right to deactivate and remove any web site from its servers which contains material that at Splash Host's sole discretion is deemed to be unacceptable, undesirable or may damage the reputation of Splash Host."

We understand Shanda that you are the customer. The customer is always right... except when the customer is taking down your business. That is where you need to draw the line, which is what Splash Host has done.

Hopefully this will give you a different perspective.
laterz.

Not saying the customer is always right. In fact, that is rarely the case. But, upon deactivation, could I get a little notey-note saying that I'm the cause of the mayhem? Customer service? F-that, common courtesy. My site's been up and down for weeks. Once the problem was found, how hard would it have been for him to type a few lines in an e-mail and hit send?

Instead, he posts on the forum (first time I'd heard anything about it) that I was the cause of his problems. Be for real, I don't think anyone else would have appreciated that either.

I know you guys/gals go through a lot with hackers and the like. In fact, deactivation was a good move. But deactivation without any contact is just unprofessional.

You guys, I don't think she is upset because her account was suspended. I think she's upset because it was suspended and no notice was given. Alan's quote:


There was a reason i had not e-mailed you, some times in these situations i find if i dont contact the account owner then they will realise what has happened to their account and move without mentioning it knowing they have been caught. If they are not doing it deleberately they are normally quick to be asking whats wrong with their site.
That's just not right. The customer shouldn't be assumed to be "guilty." They are your customer! True, you had a hell of a time with your servers, true her account (as you claim) was causing the problem, and true, it should have been (and was) suspended. The only problem was that the customer was not notified of the suspension; she merely assumed the server wasn't back up yet, per your email stating the servers were having problems. Had you mailed another email saying everything was back to normal, she might have noticed something fishy with her account. It's a shame she had to come to WHT to find out what had happened to her account...

If I was in her position, I'd be fuming. If my provider was having big problems with the network and then cut my servers off for being the culprits and assumed I was doing something illegal or something -- without contacting me -- I'd rip those servers right out and do something legally. You just don't do that to someone's business.

ok, i was wrong to not send her an e-mail.

Friends, friends, friends!

Can we all close this thread, shake hands and go home and have a beer please???

Cheers
Balaji

Originally posted by dhabets


I move site files by making a tarball and gzipping it. Databases are done ... manually by exporting them :(

Important site, which are stored redudantly, are just rsync-ed daily between accounts (it only copies changes).


But how you are doing rsync?

Originally posted by MotleyFool
Friends, friends, friends!

Can we all close this thread, shake hands and go home and have a beer please???

Cheers
Balaji :beer:


e-drinks? virtual beer? either way, it's on me!

Now everyone is in a good mood have a look at this cheery little story. Make sure you have your sound turned up as the audio quality is not that good.

http://home.kimo.com.tw/netspooky/kikia/

Originally posted by shanda
:beer:


e-drinks? virtual beer? either way, it's on me!

Bravo shanda!

That's the spirit ! Although I prefer vodka or red wine to a beer ...;) I think I would love beer as much for this special occasion!

:beer:

Cheers
Balaji

I'm glad the situation has worked out.. I'm sure everyone has learned a thing or two from this thread- another good thing about WHT..

Cheers! :beer:

Originally posted by sergio


But how you are doing rsync?

check the rsync man pages.
you can run in a script and use a crontab to run it every night.

p.s. it's best to pm me. I don't always keep track of these threads.

Good work guys. Cheer !!!!!!!

TrungNgo