Hi, Every one,

My store(www.samszone.com) loads really slow recently with 6000 visitor and 40000 pages views a day. I talked to my host and they told me the codes needs to be cleared because each request on my home pages and categoire pages opens a lot of connections to mysql db. (I use oscommerce)

Any experts on oscommerce? what kind of traffic can oscommerce handle? Is it possible to clear up the code and make it easy to handle higher traffic? anybody has the samiliar traffic? Can anybody recommend a shopping cart system that is able to easily handle 20000 visitors and 200000 page views a day? I am expecting that traffic in half a year.

HELP!

Kevin

I'm very familiar with a couple of solutions that can readily handle that kind of volume with ease.

I would have to know some additional specifications to be able to give you more specifics but it is definitely possible to support and sustain both your current and projected volume.

One thing that you may want to consider is to have a static-based shopping cart as opposed to dynamically, database-driven pages. This way your page loads are substantially faster and the store back office system can just rebuild the static pages when you add or change products.

What platform and server setup are you using? Linux, Windows? And what payment gateway are you hooked up with currently? Also if you don't mind me asking... what kind of transaction volume are you running at from those 6,000 visitors/day? (feel free to not answer this if it is too intrusive - that is not my intent, I am only thinking in terms of various load factors & logistics)

Hi, I am use Linux, authorize.net, sorry for the last question.

what shopping cart or solution would you suggest? can you send me an email regarding your contact info?

Connections to the DB are one of the most expensive (resource wise) action that takes place in a scripting language, check to see if you have conneciton pooling on. It will be in catalog/includes/configure.php and should say
define('USE_PCONNECT', 'true');

If you are already using pooling, you might want to think about migrating to a more scalable platform such as Java (servlet/jsp not applet), one of the issues around php is its lack of scalability.

Other trees to bark up would be the MySql setup on the host server, there may be too many "other" users. Or the disk drives may be older, and a newer drive with more ram (on the Hard Drive not the Server though that never hurts) may speed up the data retrival, in turn speeding up the page rendering.:cartman:

You might want to check out http://oscommerce.snappyserver.com/ - they are very familiar with osCommerce & might be able to help you out & give you a few suggestions as well. I have used them in the past to refer clients to for osCommerce & they have been extremely helpful

Originally posted by khashs
Connections to the DB are one of the most expensive (resource wise) action that takes place in a scripting language, check to see if you have conneciton pooling on. It will be in catalog/includes/configure.php and should say
define('USE_PCONNECT', 'true');

If you are already using pooling, you might want to think about migrating to a more scalable platform such as Java (servlet/jsp not applet), one of the issues around php is its lack of scalability.

Other trees to bark up would be the MySql setup on the host server, there may be too many "other" users. Or the disk drives may be older, and a newer drive with more ram (on the Hard Drive not the Server though that never hurts) may speed up the data retrival, in turn speeding up the page rendering.:cartman:

All these points are very good. I can add few. If he projects 200000 pages/day, he must have a dedicated host with a database on a separate host. And I agree, he must run away from the PHP - Java is a much better choice.

Peter.

Originally posted by pnorilsk
All these points are very good. I can add few. If he projects 200000 pages/day, he must have a dedicated host with a database on a separate host. And I agree, he must run away from the PHP - Java is a much better choice.

Peter.

Well,

I will agree with the first part of this statement - however...

"And I agree, he must run away from the PHP - Java is a much better choice."

This just is not accurate (though - traditionally for this type of load Java has been the preferred route)

What I would recommend is to migrate your application to 2 dedicated servers... osCommerce can easily handle the loads you are talking about - however, if you are in a shared environment - you are doomed...

You need 1 web server and 1 juiced up DB server to handle mySQL

The DB server should have as much RAM as possible and as much processor as you can afford. This statement would be true, no matter what the platform...

With this many visitors to a storefront, you should be able to generate enough income to support the appropriate environment for your application. Expecting any DB driven application, especially a storefront to handle this load in a shared environment is not reasonable, regardless of whether its written in php or java...

Hope this helps....

Originally posted by Gargoyle
Well,

I will agree with the first part of this statement - however...

"And I agree, he must run away from the PHP - Java is a much better choice."

This just is not accurate (though - traditionally for this type of load Java has been the preferred route)

What I would recommend is to migrate your application to 2 dedicated servers... osCommerce can easily handle the loads you are talking about - however, if you are in a shared environment - you are doomed...

You need 1 web server and 1 juiced up DB server to handle mySQL

The DB server should have as much RAM as possible and as much processor as you can afford. This statement would be true, no matter what the platform...

With this many visitors to a storefront, you should be able to generate enough income to support the appropriate environment for your application. Expecting any DB driven application, especially a storefront to handle this load in a shared environment is not reasonable, regardless of whether its written in php or java...

Hope this helps....

Well, you didn't give me another choice. I would like to examine the next your statement "though - traditionally for this type of load Java has been the preferred route". And why is this so? Let me redirect you to the numerous studies, reports, actual life examples, such as Wall Street, Fortune 500, etc. Why all of them choose to use Java (in some instance .NET) based solutions, but not PHP?
I do not ask for any retractions, but to characterize my statements as inaccurate is not "accurate" at least.

Peter.

And I agree, he must run away from the PHP - Java is a much better choice.

I dont believe he needs to "run" away from PHP - This is just bad advise - start from scratch? - especially when their current application will meet their requirements


This just is not accurate (though - traditionally for this type of load Java has been the preferred route)

And why is this so? Let me redirect you to the numerous studies, reports, actual life examples, such as Wall Street, Fortune 500, etc. Why all of them choose to use Java (in some instance .NET) based solutions, but not PHP?

I think we are saying the exact same thing - notice my comment -
traditionally for this type of load Java HAS BEEN the preferred route

However - I must repeat what Ive said above - Just because java is prominant in legacy systems does not mean that recommending someone switch from PHP to Java is the correct advise - they are seeing growth and resultant performance issues on a presumabley shared server - theres many things that can be done to improve performance without ripping and tearing - this just isnt sound advise - simple cost analysis would show this -

Hi, thanks for all the advise.

I am with a dedicated server already with P IV 2.66 and 512 ram. here is what my host says after investigaion.

This page needs to be recoded. This server is querying the mysql database
for every image on every page, this is overloading mysql. The html code on the
main page is also creating 37+ connections per user accessing the site, a new
session is being created for every image clicked on.

Originally posted by catchmeifuca
The html code on the
main page is also creating 37+ connections per user accessing the site, a new
session is being created for every image clicked on.

Yes, this would be a serious issue !!

Sessions utilize resources - having a session per user is one thing (and expected) - having a session per image is a nightmare and needs to be addressed...

Im a little confused how this happened - do you have any custom coding in your osC -

I did customize it. but I don't think I touched that part. Are you familiar with oscommerce code and can you fix it?

I am familiar with osCommerce - but, let me ask you a few questions ....

Do you have persistent connections on? (if you do, turn them off - and ask your hosts to kill any that are "open" - if used properly they can increase the efficiency of your site - but, more then likely they would cause you problems)

I took a look at your site and you have FAR too many images per page - can you reduce them?

Have you taken a look at your image headers and ensured that they can be cached by your visitors?

There are many things you could do to improve performance...
(first thing to do is get more RAM - 512 may not enough for 4000 uniques/day - any idea what your average # simultaneous users is?)

I simply have a hard time believing that osC is creating a session for each image (unless something is wrong with persistent connections and you simply have many many persistents open without them being properly re-allocated - and your host is not interpreting this properly)

- If persistent connections are off then I am certain the problem is elsewhere....

Originally posted by Gargoyle
I think we are saying the exact same thing - notice my comment -
traditionally for this type of load Java HAS BEEN the preferred route

However - I must repeat what Ive said above - Just because java is prominant in legacy systems does not mean that recommending someone switch from PHP to Java is the correct advise - they are seeing growth and resultant performance issues on a presumabley shared server - theres many things that can be done to improve performance without ripping and tearing - this just isnt sound advise - simple cost analysis would show this -

One should be true or "for this type of load Java HAS BEEN the preferred route", or "Just because java is prominant in legacy systems does not mean that recommending someone switch from PHP to Java is the correct advise", but not both. Java typically is not referred as "legacy", COBOL and PHP do.

I cannot characterize system without extra info, but if he choked with 40000 pages, what he can expect from 200000 pages load.

At some point you should accept notion that PHP cannot be used with heavy loaded transaction oriented systems. We found it long, long time back. Good luck.

Peter.

Originally posted by pnorilsk
Java typically is not referred as "legacy", COBOL and PHP do.


?????



I cannot characterize system without extra info, but if he choked with 40000 pages, what he can expect from 200000 pages load.


He isnt choking because of number of page loads - something is going on, and I havent been able to isolate yet (with the info given) - more then likely its a combination of several things - Ive seen a websphere system choke at half this load, and its in no way indicative of webspheres ability to handle much much larger loads (and yes, websphere is java based)


At some point you should accept notion that PHP cannot be used with heavy loaded transaction oriented systems. We found it long, long time back. Good luck.


There is absolutely no basis for this statement - especially not now - previous versions of php admittedly were not as robust - but, I dont think this statement is valid anymore.. You may want to revisit this issue..

Originally posted by Gargoyle
[B]
He isnt choking because of number of page loads - something is going on, and I havent been able to isolate yet (with the info given) - more then likely its a combination of several things - Ive seen a websphere system choke at half this load, and its in no way indicative of webspheres ability to handle much much larger loads (and yes, websphere is java based)/B]

So, he is not choking because of load, but you are not able to isolate the cause (may be load?) and btw WebSphere choked even with the smaller load (based on what you saw).

You cannot do that.

Here is a deal. Please, use logical arguments, don't say thing unless you know it for sure; otherwise there is no sense to continue this dialog.

I apologize to forum for this outburst but we are dealing here with a special case. And we cannot afford to have it stay without an answer.

Peter.

I think we are having a communication problem here - I do not believe we are saying different things - just looking at things from different perspectives...

I apologize to forum for this outburst but we are dealing here with a special case. And we cannot afford to have it stay without an answer.

I dont feel you need to apologize, as I dont see an outburst here - just 2 people discussing a situation... There is no special case here - and frankly, Im not sure I understand what you mean...

Here is a deal. Please, use logical arguments, don't say thing unless you know it for sure; otherwise there is no sense to continue this dialog.

I have used logical arguements - however - being logical does not mean I will agree with all your statements or opinions...

With the loads described by the original poster - I dont feel that osCommerce is the problem, or the language that osCommerce is written in..

My only point is as follows - even the most robust applications can function below standards if not implemented and structured properly -

If you feel the original posters problem IS osCommerce and IS the fact that osCommerce cannot handle the proposed loads - Great !!

I feel the loads described are within the capabilities of osCommerce and the poster is better off looking at optimizing that solution vs ripping it apart and starting from scratch...

Ultimately, this is a decision for the original poster to do their research on and decide what is best/makes most sense for them -

Originally posted by pnorilsk
One should be true or "for this type of load Java HAS BEEN the preferred route", or "Just because java is prominant in legacy systems does not mean that recommending someone switch from PHP to Java is the correct advise", but not both. Java typically is not referred as "legacy", COBOL and PHP do.

I cannot characterize system without extra info, but if he choked with 40000 pages, what he can expect from 200000 pages load.

At some point you should accept notion that PHP cannot be used with heavy loaded transaction oriented systems. We found it long, long time back. Good luck.

Peter.

Actually, PHP was designed to be one of the most efficient languages. In fact, JAVA from experience and performance tests, proves to be one of the slowest executing applications, of course in some cases that is not always true but for the requirements here, I believe it certainly is.

I am familiar with both PHP, JAVA and osCommerce and I would have to say, the suggestions to divide the database and php, side of things into two servers is likely the most effective solution.

Even if it was not, the reason for osCommerce/PHP not being fast enough, would only be a limitation of how osCommerce has been coded, not that anything was done wrong, but in theory.

Those are my thoughts, but your suggestions are valid in some contexts, especially in more demanding and complex situations.

In the end, I don't think it wise to automatically suggest which programming language is the best for the requirements, because I think the business model comes first, and the programming language is certainly, in traditional Systems Analysis is not the first thing we should look at :)

Those are my thoughts, and they were not intended to downplay your suggestions.

Originally posted by stivan
Actually, PHP was designed to be one of the most efficient languages. In fact, JAVA from experience and performance tests, proves to be one of the slowest executing applications, of course in some cases that is not always true but for the requirements here, I believe it certainly is.



Please, give us specific examples of performance tests or correct your statement.

The most publicized independent study (I know) is a work of "The Middleware Company" where J2EE (Java) application was tested against the similar application written in C# on .NET. They found no significant difference in the performance between these two environments for this specific application. I would be happy to provide a copy of this report on request. In this context you would not argue that any scripting language such as PHP is "faster" than low level language it's written - PHP vs C or its cousin (C#).

There are many more reason to use Java in commercial web-centric applications . Here is how one guy put it together. I don't think I can do it better:

"The installation base is the primary reason I'm using PHP for a project now; it's intended for use by novices who most likely are in a virtual hosting environment without access to a servlet container. But whenever I'm working on it I find myself longing for a lot of the features of Java.\n\n Simply put, Java is a real programming language and PHP isn't. While you can code in a non-OO way with Java, anyone that uses it much has the whole concept drummed into their heads at an early stage. So even a rotten app written in Java is generally much more maintainable than one written in PHP. The language is bigger, it's more flexible, and it gives you much finer-grained control over what's going on than does PHP. Want a GUI? No problem. Database connection pooling? Ditto. A multi-threaded app? Objects that persist between server calls? To use XML and XSL? You get the idea.\n\n And there are so many fantastic open-source resources for Java. Jakarta awes me, and so does a lot of the stuff on sourceforge. I'm learning Ant, Hibernate and the JSTL right now--it's like being a kid in a candy store. By contrast, what's available for PHP is a joke. Sure, Metabase and Smarty are both good projects, but I have a hard time thinking of any other decent PHP libraries (and that includes what's in PEAR). And those examples aren't anywhere near the functionality and maturity of Eclipse or Cocoon, just to name two.\n\n My thought: if you want to hack out a webapp in a weekend, definitely go with PHP. If you're building anything you're going to be stuck maintaining for a while, or even rebuilding a Java app that completely stinks, salvage as much of the code as you can and stay with Java. You'll be much happier in the long run.\n\n"

Peter.

Does anyone know of a system that gets 200k+ pv and is running on php & mysql?

I would suggest that catchmeifucan has or will soon outgrow osCommerce. If he throws enough money and time at it, I am certain that he can make php / mysql / osc work at his expected loads.

As Chris Rock said "Just cause it can be done doesn't mean it should be done"

O.K, thanks a lot for your suggestions. but you guys really make me confused. So Andrew believes that php/oscommerce should be able to handle my expected traffic if I configure them well, but Peter thinks that I should switch to JAVA based appliactions.

Let me provide more information, probably with these information you can help me isolate the cause.

1. persistant connection: initially it was off, then I followed the advice to turn it on, I didn't think I have significant improvement. Now I turned it off again after reading Andrew's post.(Should I turn it on or off?)

2. "Have you taken a look at your image headers and ensured that they can be cached by your visitors?"
I don't know, can you let me know how to check?

3. my host says "The html code on the
main page is also creating 37+ connections per user accessing the site, a new
session is being created for every image clicked on."
He gave this conclusion based on runing "netstat" on my server, he says runing such a command shows that a user has 30-40 time_waits

here is part of the output
--------------------------------------------------------------------------------------------------
tcp 0 0 132.67-18-68.rever:http adsl-67-119-13-41.:4749 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50544 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http svcr-209-74-8-235.:3528 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 67-51-129-212.dsl:60551 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50608 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http cs242737-116.austi:3475 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 65.119.159.27:1898 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50487 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 96.syracuse-09-10r:4831 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-205-153-97.wtr-:1708 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http host217-42-5-32.ra:2390 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-67-119-13-41.:4746 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50551 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http bzq-132-62.red.bez:3576 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-116-212-26.cpe.:7139 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50615 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http bzq-132-62.red.bez:4984 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http cs242737-116.austi:3474 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 65.119.159.27:1899 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50486 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 96.syracuse-09-10r:4830 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-205-153-97.wtr-:1709 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-67-119-13-41.:4747 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http host217-42-5-32.ra:2391 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50550 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http bzq-132-62.red.bez:3577 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http svcr-209-74-8-235.:3534 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 67-51-129-212.dsl:60545 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-116-212-26.cpe.:7138 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50614 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http pcp01256702pcs.:hylafax TIME_WAIT
tcp 0 0 132.67-18-68.rever:http bzq-132-62.red.bez:4985 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http cs242737-116.austi:3473 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 65.119.159.27:1896 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50485 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 96.syracuse-09-10r:4829 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-205-153-97.wtr-:1710 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http d-216-195-151-84.g:1371 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http host217-42-5-32.ra:2388 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-67-119-13-41.:4744 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50549 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http bzq-132-62.red.bez:3578 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http svcr-209-74-8-235.:3533 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 67-51-129-212.dsl:60546 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-116-212-26.cpe.:7137 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50613 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http bzq-132-62.red.bez:4986 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http cs242737-116.austi:3472 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 65.119.159.27:1897 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50484 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 96.syracuse-09-10r:4828 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-205-153-97.wtr-:1711 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http d-216-195-151-84.g:1370 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-67-119-13-41.:4745 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http host217-42-5-32.ra:2389 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50548 TIME_WAIT
tcp 0 267 132.67-18-68.rever:http bzq-132-62.red.bez:3579 FIN_WAIT1
tcp 0 0 132.67-18-68.rever:http 67-51-129-212.dsl:60547 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-116-212-26.cpe.:7136 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50612 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 0-2pool17-135.nas8:4855 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http bzq-132-62.red.bez:4987 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http cs242737-116.austi:3487 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 65.119.159.27:1894 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50491 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http host217-42-5-32.ra:2394 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-67-119-13-41.:4742 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 96.syracuse-09-10r:4819 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-205-153-97.wtr-:1696 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50555 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-116-212-26.cpe.:7151 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50619 FIN_WAIT2
tcp 0 0 132.67-18-68.rever:http 0-2pool17-135.nas8:4856 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http cs242737-116.austi:3486 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 65.119.159.27:1895 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50490 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-67-119-13-41.:4743 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http host217-42-5-32.ra:2395 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 96.syracuse-09-10r:4818 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-205-153-97.wtr-:1697 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50554 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http svcr-209-74-8-235.:3522 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-116-212-26.cpe.:7150 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50618 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 65.119.159.27:1892 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50489 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http host217-42-5-32.ra:2392 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 96.syracuse-09-10r:4817 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-205-153-97.wtr-:1698 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50553 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http svcr-209-74-8-235.:3521 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-116-212-26.cpe.:7149 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50617 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http cs242737-116.austi:3484 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 65.119.159.27:1893 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50488 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-67-119-13-41.:4741 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http host217-42-5-32.ra:2393 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-205-153-97.wtr-:1699 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 96.syracuse-09-10r:4816 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http bzq-132-62.red.bez:3575 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50552 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-116-212-26.cpe.:7148 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50616 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http bzq-132-62.red.bez:3447 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http bzq-132-62.red.bez:4983 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http cs242737-116.austi:3483 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 65.119.159.27:1890 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50495 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http host217-42-5-32.ra:2398 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 96.syracuse-09-10r:4823 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-205-153-97.wtr-:1700 TIME_WAIT
tcp 0 11508 132.67-18-68.revers:ssh 69-169-57-174.anhm:2448 ESTABLISHED
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50559 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http svcr-209-74-8-235.:3527 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 67-51-129-212.dsl:60552 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-116-212-26.cpe.:7147 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50623 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http cs242737-116.austi:3482 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 65.119.159.27:1891 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50494 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http host217-42-5-32.ra:2399 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-205-153-97.wtr-:l2tp TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50558 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 67-51-129-212.dsl:60553 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-116-212-26.cpe.:7146 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http cache-rm06.proxy.:60744 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50622 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http cs242737-116.austi:3481 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 65.119.159.27:1888 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50493 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http host217-42-5-32.ra:2396 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-67-119-13-41.:4736 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-205-153-97.wtr-:1702 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50557 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-116-212-26.cpe.:7145 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50621 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 65.119.159.27:1889 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50492 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-67-119-13-41.:4737 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http host217-42-5-32.ra:2397 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 96.syracuse-09-10r:4820 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-205-153-97.wtr-:1703 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50556 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http svcr-209-74-8-235.:3524 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http 24-116-212-26.cpe.:7144 TIME_WAIT
tcp 0 0 132.67-18-68.rever:http adsl-0-136-217.pn:50620 TIME_WAIT
-------------------------------------------------------------------------------------------------

As for memory usage, here is the output when I run a "free" on my server
-----------------------------------------------------------------------------------

total used free shared buffers cached
Mem: 495776 427788 67988 0 37556 159660
-/+ buffers/cache: 230572 265204
Swap: 1012052 130544 881508
-----------------------------------------------------------------------------------

But this is not my peak time. My hosts says the cpu usage and memory is O.K.

here is the output when I run "top" on my server.
------------------------------------------------------------------------------------
13:06:36 up 16 days, 12:59, 1 user, load average: 0.64, 0.84, 1.20
107 processes: 102 sleeping, 5 running, 0 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 39.9% 0.0% 11.0% 0.0% 0.1% 0.0% 48.7%
cpu00 39.9% 0.0% 10.3% 0.0% 0.3% 0.1% 49.1%
cpu01 39.9% 0.0% 11.7% 0.0% 0.0% 0.0% 48.3%
Mem: 495776k av, 432444k used, 63332k free, 0k shrd, 38228k buff
220240k actv, 33676k in_d, 6592k in_c
Swap: 1012052k av, 130544k used, 881508k free 160076k cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
20411 mysql 21 0 27388 9640 1344 R 12.8 1.9 0:01 1 mysqld
20412 mysql 20 0 27388 9640 1344 R 5.0 1.9 0:00 0 mysqld
20414 mysql 19 0 27388 9640 1344 R 2.4 1.9 0:00 0 mysqld
13267 nobody 15 0 7936 7348 2368 S 2.0 1.4 19:54 0 httpd
20296 nobody 16 0 6548 6124 2752 S 1.6 1.2 0:00 1 httpd
13271 nobody 15 0 7368 6888 2164 S 1.5 1.3 19:14 0 httpd
13266 nobody 16 0 7836 7272 2492 S 1.2 1.4 19:47 1 httpd
13295 nobody 15 0 7420 6932 2156 S 1.2 1.3 20:03 0 httpd
20415 mysql 20 0 27388 9640 1344 R 0.9 1.9 0:00 1 mysqld
20293 nobody 16 0 6104 5652 2380 S 0.6 1.1 0:00 1 httpd
20413 nobody 20 0 4784 4252 1752 S 0.4 0.8 0:00 1 httpd
13277 nobody 15 0 7628 7148 2408 S 0.1 1.4 19:40 0 httpd
1 root 15 0 468 440 420 S 0.0 0.0 0:14 1 init
------------------------------------------------------------------------------------

As for average # simultaneous users, oscommerce "who's online" shows around 100-150.

What are the setting for the following apache directives

KeepAlive
MaxKeepAliveRequests
KeepAliveTimeout

you can find them in the apache dir /conf/httpd.conf

Originally posted by khashs
Does anyone know of a system that gets 200k+ pv and is running on php & mysql?

I would suggest that catchmeifucan has or will soon outgrow osCommerce. If he throws enough money and time at it, I am certain that he can make php / mysql / osc work at his expected loads.



Yes, actually - a division of Yahoo uses php/mysql - 20 - 40 front end servers serving many many many more DB requests then this shop could ever produce - I honestly do not see any reason to believe that this poster will outgrow osCommerce...

http://jeremy.zawodny.com/blog/archives/000173.html#000173

Here is a contribution to osC specifically designed for larger sites that will minimize the # of DB quries and increase performance

http://www.oscommerce.com/community/contributions,1862

mysql requires optimizing, as any DB does to handle large loads - but, there is absolutely no proof that php/mysql is not as scalable as any other option...

Originally posted by catchmeifuca
O.K, thanks a lot for your suggestions. but you guys really make me confused. So Andrew believes that php/oscommerce should be able to handle my expected traffic if I configure them well, but Peter thinks that I should switch to JAVA based appliactions.

Hi Catchmeifuca,

You have a really difficult decision - Your site is growing (congratulations by the way) and youre obviously making money from it.

The trick now is to determine how you deal with capacity and performance issues - This is typical of any business experiencing growth (ie you are def not the first to face this - regardless of platform)

If you choose to start from scratch with a custom development - make sure you select someone that will be around to support you for the long haul - because, if they disappear, you are out of luck - and finding someone else to step in and work on their coding will not be easy....regardless of what you may hear...

If you choose to "fix" your current install - I would recommend getting 2 servers (1 strictly to operate as a DB server) and choose a company that can optimize and manage your environment. This will not be cheap - but, it is available - and since youre dealing with osCommerce - youre not really realying on 1 person or 1 company - as that expertise exists...


I would recommend doing a thorough cost analysis of both options as well as thorough research on osCommerce sites - particularly larger ones - and determine what the currently established "upper" limits are - I dont thin you will approach these "limits" any time soon - but - it is advisable to completely research this before commiting to any direction...

Hope this helps....

Very interesting to watch all of this, traffic on our OSC store is growing rapidly, and its interesting to watch peoples points of view on it. I've never our store reacting with this session issue with the images but after this I may investigate it lol.

catchmeifuca,

I think I need to summarize what I said. Here is major points,
- Java is much better choice for high load transaction bounded applications
- transition from PHP to Java is not easy or should not be recommended as long as load is manageable
- advantages of Java is far above and beyond of scalability issue. Many Java based solutions don't need any programming at all, web sites can be built and maintain without need to write a single line of Java code.
- Java based technologies and frameworks allow to offer so many new services and solutions to customer and clients. And majority of them are fee, based on the Open Source Software. Please, check Jakarta projects and compare with PHP.
- the TOC (total ownership cost) of Java (OSS) is far less than PHP based solutions
- Java based solutions will bridge and/or eliminate a horrendous gap between "haves" (JPMorgan, CVS/pharmacy, GE, etc) and "have nots" (SME/SOHO)
- I will recommend to use Java based solutions on all new projects with few exceptions.

But your immediate problem could be (should be) mitigated with approach in this paper (agree with Gargoyle), pay attention to meaning of "master" in this paper.

http://jeremy.zawodny.com/blog/arch...173.html#000173

Peter.

I usually dont respond to these.

But before you folks discuss the pros and cons of Java and PHP (which bear absolutely no relevance on this),

Do you have CACHING turned on ?

Are persistent connections off?


40,000 hits a day are not that much at all.
If you have two dedicated servers, 1 db server, 1 web server, you should be able to handle this load without a problem.

FYI Java was never the preferred language for legacy systems, in fact it didnt even exist in those days. Its only now that its starting to be more accepted in the mainframe world.

Originally posted by m8solutions
Do you have CACHING turned on ?

Are persistent connections off?


please read previous posts in this thread - these questions were already asked



40,000 hits a day are not that much at all.
If you have two dedicated servers, 1 db server, 1 web server, you should be able to handle this load without a problem.


As per previous posts - we are in agreement


FYI Java was never the preferred language for legacy systems, in fact it didnt even exist in those days.

Definition of Legacy - "data or hardware that existed prior to a certain time"

I guess it depends on what time frame you are speaking to - Legacy can be/and often now is - looked at as present install base - I will be more careful to clarify completely in the future...

Nope,

Caching... osCOmmerce has a caching feature...

IS that turned on?

Mark

Originally posted by m8solutions
Nope,

Nope what?


Caching... osCOmmerce has a caching feature...

IS that turned on?


Though caching was previously mentioned - osCommerce caching feature specifically was not - fair enough - but...

Regardless of whether its turned on or not - this is not the root of the posters problems (though it could be a contributing factor)

One thing that stands out from the stats posted is the number of sleeping processes - although not at an alarming point - it is high enough to raise some flags - especially since the poster is running a dedicated server with just one application... This would indicate that an installed osC hack or some sort of cgi script is causing some havoc...

Theres other hints as well -

There is nothing wrong with the sleeing processes..

I'm beginning to wonder if you know what you are referring to...

The calls to the db are made because they need to be made.
The osCommerce cache features help eliminate the need for these constant calls to the db. hence the term 'cached'

Turn on the cache..
add some more ram - 512 is not nearly enough
and you should be fine.

I have work to do.

Good luck.

Originally posted by m8solutions
[B]There is nothing wrong with the sleeing processes...


I'm beginning to wonder if you know what you are referring to...


LOL



I have work to do.


nice to know

hope new releases of oscommerce address this issue!

Well not to step in your game, boys, but I concur on the RAM issue. He's at 75% utilization during a non-peak time... the first thing I would do is max out the RAM on the box. If nothing else it will buy some time while diagnosing the other issues.

:D Bailey

Originally posted by bithost(NET)
Well not to step in your game, boys, but I concur on the RAM issue. He's at 75% utilization during a non-peak time... the first thing I would do is max out the RAM on the box. If nothing else it will buy some time while diagnosing the other issues.

:D Bailey


Hi Baily,

No problem at all - actually - Im hoping you can help out..

I spent around 30 mins on the phone with Kevin (the original poster) - obvioulsy, the first thing we discussed was increasing RAM - before the poster makes a decision to separate the DB server - thought it would be a good idea to try and optimize the current environment then plan for any future changes...as increased load demands it....

However, there are underlieing issues here -

The technicians at his DC (the planet) are seeing the same thing - and frankly - I have no idea what the root cause is (either do they)

Heres whats happening -

- Alot of hanging DB connections
- Alot of sleeping processes
- These symptoms re-appear (fairly quickly) after a reboot - even under light load

Obviously these points above are affecting application performance...

Original Poster (Kevin) has done some mod r/w work in order to SEO osC - Ive done a bit of research here and am seeing opposing information - some feel this shouldnt affect performance at all, others feel it will, particularly at heavy loads...

I checked out the site today - and it seems to be running better - (but, it could just be a period of light load)...

We covered the basics with him - RAM, turning off persistent connections, cacheing, reducing the size of some of his images, etc...

Frankly - we have never worked with an osC site of this size (and judging by what Ive read in this thread - there arent many people that have) - I have referred Kevin to another organization that may have more experience in this area..

Anyone have any insight ?

Hi, everybody, here are more information regarding this issue.

I had an admin who seems to be more experienced in Linux and oscommerce to look into this issue. He checked my error log and solved part of the errors. But I still have a lot of sleeping processes and a lot of requests in queue.

if you go to my site and click

http://www.samszone.com/government_purchase.htm

this is a static page which load instantly while my other pages are very slow. and MySql uses 20% of cpu time at all times.

My oscommerce store is based on Linda's SEC and I added quite a few adds on such as affiliate program and visitor tracking. I decided to remove them all, hopefully it's gonna be better.

Below are some ticket details, those who are interested in please take a look and see if you could have some ideas.

Ticket #1
Ticket Details:
Queue:Technical Support
Status:OPEN
Summary:website loading slow
Details:Recently my website was loading really slow. many of my affiliates reported
it took 20-30 seconds to load my home page. and I did tested loading on my website
at different times and locations and found it was true.

I am running an oscommerce based store, through research on the forum, I believe
oscommerce should be able to handle the traffic I have.

do you have any expert on oscommerce system there? I would appreciate that if some
one can look into my system thoroughly (both software and hardware) and give me
suggestions on how to improve my responds time to my visitors.

thanks

Kevin


------------------------------------------
(mcounts-06/01/04-06:20):
Can you post a traceroute to and from one of the affected locations so we can
diagnose this?
--------------------------------------
(c16859delt-06/01/04-16:43):
how to do that?

------------------------------------------
(blackj-06/01/04-17:22):
>From one of the locations that is reporting that the site is loading slow, go into
the Windows command prompt. From there type in:
tracert <IP number of server>

The resulting display is the trace route from your location to the server in
question. Copy and paste this information into a reply to this ticket and submit it
back to us.
--------------------------------------
(c16859delt-06/02/04-13:49):
in windows command prompt, how am I able to copy and paste? can I write the output
in a file?


------------------------------------------
(dmurphy-06/02/04-14:16):
To copy and paste right click the DOS screen and select mark. Hightlight the text
you which to copy hit ctrl+c then right click. Got to wordpad or notepad and paste
the text in. Alternativly you can run tracert ip > C:pathtosavefile.txt


--------------------------------------
(c16859delt-06/02/04-19:13):
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:Documents and Settingskevin>tracert 67.18.68.130

Tracing route to 130.67-18-68.reverse.theplanet.com [67.18.68.130]
over a maximum of 30 hops:

1 1 ms 1 ms 1 ms 192.168.0.1
2 25 ms 24 ms 27 ms bdsl.66.13.55.121.gte.net [66.13.55.121]
3 21 ms 29 ms 28 ms a-6-0-102.lsanca1-aa1.bbnplanet.net [4.24.44.33]

4 25 ms 23 ms 21 ms p3-0.lsanca2-cr2.bbnplanet.net [4.24.206.1]
5 21 ms 21 ms 28 ms p0-0.lsanca2-br1.bbnplanet.net [4.25.112.2]
6 28 ms 28 ms 28 ms so-5-2-0.bbr1.LosAngeles1.Level3.net [64.159.4.2
5]
7 61 ms 56 ms 62 ms so-0-0-0.bbr1.Dallas1.Level3.net [64.159.3.149]

8 53 ms 54 ms 53 ms so-6-0-0.edge1.Dallas1.Level3.net [209.244.15.16
2]
9 58 ms 69 ms 73 ms unknown.Level3.net [63.209.48.110]
10 63 ms 66 ms 53 ms sm-asr2-2-v1.dllstx2.theplanet.com [12.96.160.26
]
11 53 ms 58 ms 67 ms gig0-2.sm-car5-3.theplanet.com [69.41.250.74]
12 58 ms 61 ms 63 ms 130.67-18-68.reverse.theplanet.com [67.18.68.130
]

Trace complete.

C:Documents and Settingskevin>
------------------------------------------
(mmontgomery-06/02/04-20:46):
The SQL database for samszone is what's killing your machine... it's consistently at
20% cpu load, just for mysql...

It's a 300MB Database, and I'm assuming the site is issuing very large queries... I
would suggest tuning mysql for this large database, as this is something that isn't
done without a charge.

Please let us know if we can help.

Thanks.
--------------------------------------
(c16859delt-06/03/04-02:02):
do you have an expert there for oscommerce? I would like to discuss with him in more
detail.
------------------------------------------
(jhodge-06/03/04-02:08):
If you update this ticket with whatever specific questions you would like to
discuss, we can try and answer them to your satisfaction. Please let us know.
--------------------------------------
(c16859delt-06/03/04-21:22):
Hello,
One thing I have noticed is that to backup a 2 mb file via mysql it is taking over 5
minutes to do so. There must be a configuration error in the my.cnf or in the
php.ini, causing this. I have increased the max memory in php.ini to 64m however
not much is going on with that. I have another site on another host, where the sql
file is 18 meg in size and it takes approximately 40 seconds to perform the backup.
if you would like to compare, i can walk you through both sites, the one here and
the other one on a server at another company in florida.
also, there seems to be an exorbitant amount of processes running on the system, the
load average stays fairly high.
once we can get these issues resolved, then there will be no problem.
------------------------------------------
(zwilliams-06/04/04-05:52):
How are you trying to backup the mysql database? I just dumped the
samszone_testsite to a file in the root directory, and it took only a few seconds to
do so. That database appears to be a few megabytes.

Also, to transfer a file of this size through phpmyadmin, you'll need to increase
the max file size, which defaults to 2M.
--------------------------------------
(c16859delt-06/04/04-08:15):
it was being done via oscommerce mysqldump, which seems to be timing out. the sql
db size of that is 374mb.
i changed the max memory to 64, which does not seem to have much of an effect, the
load average is very high all the time. there has to be a way to decrease this, i
believe there are configuration errors in the httpd.conf and php.ini which make the
server slow to a crawl.
------------------------------------------
(mmontgomery-06/04/04-20:50):
Doing some more testing on this now...
Resolution:

Please do not reply to this email. If you have additional information, please
update the ticket using the link to the support system provided. If the support
ticket was closed, and the issue was not resolved to your satisfaction, please feel
free to open another ticket and reference this ticket number.

Thank you,
The Planet
------------------------------------------------

ticket #2

Ticket Details:
Queue:Technical Support
Status:OPEN
Summary:SSLSessionCashe Error in the error_log
Details:Hello,

I have been reviewing the error_log, since middle of May, it is 189 MB in size.
latest recurring error is:
mod_ssl: Cannot open SSLSessionCache DBM file '/usr/local/apache/logs/ssl_scache'
for writing (store) (System error follows)
system: No such file or directory (errno: 2)


I created the ssl_scache file by doing a 'touch ssl_scache' and then tried different
rights, and ownership /groups.

We could use some assistance.
------------------------------------------
(mmontgomery-06/03/04-20:50):
[Thu Jun 3 20:49:43 2004] [notice] Apache/1.3.29 (Unix) mod_auth_passthrough/1.8
mod_log_bytes/1.2 mod_bwlimited/1.4 PHP/4.3.3 FrontPage/5.0.2.2634 mod_ssl/2.8.16
OpenSSL/0.9.7a configured -- resuming normal operations
[Thu Jun 3 20:49:43 2004] [notice] suEXEC mechanism enabled (wrapper:
/usr/local/apache/bin/suexec)
[Thu Jun 3 20:49:43 2004] [notice] Accept mutex: sysvsem (Default: sysvsem)


This appears to be fine now after restarting apache, and lessing it create it.
--------------------------------------
(c16859delt-06/04/04-11:54):
still geting an error:
mod_ssl: SSL handshake interrupteb by system [Hint: Stop button pressed in
browser?!] (System error follows)
[error] System: Connection reset by peer (errno: 104)

however this error displayed in the rror log when I went to a page on the site,
while I was monitoring the log.

thus there is a serious misconfiguration in the server, which makes it run very slow.
------------------------------------------
(mmontgomery-06/04/04-22:16):
This is going to take some more investigation, and as such, I'm moving this to level
two.

------------------------------------------

an email from the admin

My recommendations are, if you are going to remain with the current host,
then the site needs to be revamped, RedHat needs to be configured properly.

Overall the site needs to be upgraded to the current versions of the
contributions, to eliminate bugs.
The error log for the site needs to be monitered, so all errors can be
eliminated. Your web designer should be made aware of the errors listed in
the log so they can be attended to.

The major errors dealing with mod_ssl sessioncache have now been resolved,
this one single problem generated 3,600 errors per hour and has been
happening for the length of the error log from what I could determine. It
was resolved by dealing with tech support, notifying them of the errors in
the error log and they reconfigured that portion of the operating system.

I have posted other issues which tech support needs to deal with.

We can discuss further assistance if you desire.

My overall preference would be to have you move from theplanet.com to
another server, whereby you do not use the control panel, as that also takes
lots of overhead and as you see, theplanet does charge you additional for
using the cpanel.

If you wish to discuss this aspect of moving to another server we can do so,
recommend if you decide to do this, that at least two weeks time for proper
configuration to allow the site to be tested fully.
----------------------------------------------------------------------------------

Right now I still don't have a clue on what's going on, since it is a mixed issue of hardware, operating system, software issue. It seems I had some serious misconfiguration on the server since I have a lot of sleeping processes and requests in queue. Also I belive bad coding is a serious issue too.

Basically if noboby is experienced with oscommerce store that has more than 10,000 visitors a day. I am considering switching to an another shopping cart system that can handle at least 20,000 visitors a day. I think a shopping cart that can generate static html pages are better than a dynamic system. if you guys can recommend me such a shopping cart. (the idealed system would use a database to manage products, but all the pages are generated as HTML pages.

I can tell you I had a somewhat similar experience with osCommerce. I tried importing a client's database into osCommerce. It ended up having probably about 1000 categories, including nested subcategories.

For whatever reason, this slowed osCommerce to a crawl. I couldn't find any logs at the time, but I am quite confident the problem in that case is that, osCommerce by nature, calls the function to load up all the categories...........and since there are so many and some are nested. Eg category 0 is a sub of 1 and subcategory 2 is a sub of 1 etc...... it slowed it to a crawl.

I should also add that there were thousands of products. Maybe osCommerce is, dare I say, not coded efficiently to handle large volumes of queries because just a quick browse through some of the code, showed me that functions that shouldn't be called consantly, were being called, creating excess database queries.

I posted the question in an osCommerce forum and no one was able to help.

However, I have seen fairly large stores on-line. I also believe the server that was being used, was certainly misconfigured, as I can attest php and mysql were not configured properly.

Perhaps updating all the components such as mysql, php etc...... could solve your problem. If not, maybe do a complete os reload and start from scratch.

Best wishes :)

May I know which is the fairly large stores on line?

Originally posted by stivan
I can tell you I had a somewhat similar experience with osCommerce. I tried importing a client's database into osCommerce. It ended up having probably about 1000 categories, including nested subcategories.

For whatever reason, this slowed osCommerce to a crawl. I couldn't find any logs at the time, but I am quite confident the problem in that case is that, osCommerce by nature, calls the function to load up all the categories...........and since there are so many and some are nested. Eg category 0 is a sub of 1 and subcategory 2 is a sub of 1 etc...... it slowed it to a crawl.

I should also add that there were thousands of products. Maybe osCommerce is, dare I say, not coded efficiently to handle large volumes of queries because just a quick browse through some of the code, showed me that functions that shouldn't be called consantly, were being called, creating excess database queries.

I posted the question in an osCommerce forum and no one was able to help.

However, I have seen fairly large stores on-line. I also believe the server that was being used, was certainly misconfigured, as I can attest php and mysql were not configured properly.

Perhaps updating all the components such as mysql, php etc...... could solve your problem. If not, maybe do a complete os reload and start from scratch.

Best wishes :)

I don't know how long ago you did that, but I had a similar problem with a store. I found that caching the category list and turning off product counts (so categories are not displayed like this: "category_name (# products)") sped it up A LOT.

If you ever do look into developing a custom solution, I would highly recommend you look into PERL. Its pre-existing libraries is unmatched by any other language (cpan.org) and it is used by the likes of etoys.com and amazon.com. When considering a language, maturity and programmer base are probably the most important things. PERL is very mature and many people know how to program in it, so finding good programmers will not be a problem (unlike with Java). In addition, it is very similar to PHP (one time when I had WAY too much time, I converted the osCommerce admin area to PERL in about a weeks time of casual work).

Hi Mark,

Thanks for the tip about osCommerce. I'll remember that for next time :).

As for PERL, I disagree after my experience with it. Not that the language is the problem here, but PERL is much slower than PHP.

PERL is also more prone to cause crashes. And more importantly, PERL progarms are horrible in my opininon because they have lots of standard libraries but every system I've dealt with, was a pain because most tend to use specialized libraries which can not always be found or easily tracked down.

I would certainly stay away from it for many reasons.

I believe PERL is on the way out. I'd much rather use JSP or CFM before PERL and I don't like JSP or anything with a J at all :)

Sorry to rain on your parade. But those have been my professional experiences with PERL. Not to say that it isn't powerful or can't be used for good purposes, I just don't think it is a wise idea to use it.

I just came across this thread and thought I would jump in and ask a couple of questions...

Okay, the Server is a P4 2.66... What kind of hard disks are you using and how many? Are they IDE drives?

I know its been stated, but, yes, the memory definitely needs to be upgraded.

Are there any other sites being hosted on this Server? If so, how many?

What version of osCommerce are you using? (Snapshot date)

Are you storing your sessions in MySQL? If so, that could be the reason for so many MySQL connections.

Do you find the site slow only when browsing the category levels, or is it also slow when going through checkout or viewing a specific product item?


For a site running osCommerce with as much traffic as described should be on its own Server running Dual Processors, 15K SCSI drives, and 2GB of RAM. That should help.

Overall, osCommerce should be able to handle the load no problem.

-R

Hi, Randy, Andrew and all,

Thanks for all the help.

I guess I have an IDE drive.

my oscommerce is based on Linda's SEC and then I added quite a few adds on.

there a 5-6 sites on the server, but this store occupies 95% of the traffic.

if I don't store session into MySQL, is it safe?

I trucated three tables, affiliate_clickthrough, visitors, using_tracking, these tables overall has more than 1,000,000 records, after that, the site is loading a lof faster.

So I would suggest never install these contributions. I soldem use them and they use a lot of resouces. after trucating these tables, MySQL drops from more than 25% of CPU time to less than 3%.

Is 1,000,000 record large? I still don't know whether MySQL is able to handle that many record or some kind of misconfiguration in my MySQL settings.

Randy, what is the largest oscommerce store you've seen? what is the maximam concurrent users allowed for oscommerce? 500, 1000, or more?

I guess I have an IDE drive.
That means that your drive is probably either 5200 or 7200 rpm. They aren't the fastest drives. Moving to high-speed 15K rpm drives with RAID5 would help speed up your MySQL queries, considering queries are disk intensive and you are talking about alot of users making many queries at any given point in time.

my oscommerce is based on Linda's SEC and then I added quite a few adds on.
What version of Linda's SEC install are you using? v2.2? Or something older? Open /catalog/includes/application_top.php file and let me know the date at the top of the file.

if I don't store session into MySQL, is it safe? Yes, using TMP file/directory-based sessions is safe.

I trucated three tables, affiliate_clickthrough, visitors, using_tracking, these tables overall has more than 1,000,000 records, after that, the site is loading a lof faster.

So I would suggest never install these contributions. I soldem use them and they use a lot of resouces. after trucating these tables, MySQL drops from more than 25% of CPU time to less than 3%.
Since there are no limits to the number of records stored for any of those contributions, these tables continue to grow. Did you keep a dump of the records you removed? The Visitors and User Tracking records are for your viewing, so its up to you on how far back you keep these records. But, if you have active affiliates, make sure you are not removing any records that may effect them getting paid.

Both the Visitors and User Tracking contributions are okay if you don't have access to your site logs. Otherwise, it basically gives the same info.

Is 1,000,000 record large? I still don't know whether MySQL is able to handle that many record or some kind of misconfiguration in my MySQL settings.
Yes, its large, but MySQL can handle it, but it really depends on the "horsepower" it has to work with. By changing the hard disk to 15K SCSI drives in a RAID5 configuration, adding dual processors and adding as much memory as you can afford, you will find that MySQL will be very happy.

Randy, what is the largest oscommerce store you've seen? what is the maximam concurrent users allowed for oscommerce? 500, 1000, or more?
What do you mean be the largest? For example, I am currently building an online store for an ATV Parts shop with osCommerce. They have over 10,000 parts available.

Consider this, many popular forums, including this one, are PHP/MySQL based. Just imagine how many records are currently being stored for WHT? No problem.

osCommerce itself has no User limits. It's the web server config that sets those limits.

Here's my take...

If your store is that busy, you must be making money from it. If its generating a large amount of revenue, you should be able to afford to make some upgrades.

Remember, yes, osCommerce is a very robust solution, but it is also open-source. Meaning you really don't have anybody specific that can be responsible for it if it breaks or you get in over your head (Which works out great for me!). Presently, your only re-course should something happen is to post questions on the osCommerce forums and attempt to fix it yourself, or find someone like me to fix it for you. Keep asking yourself how mission-critical your store is, and how much money you would be losing if it were to go down for a period of time.

(Moderators - if any of the above statements are considered self-promotion and are deemed inappropriate, feel free to remove. It was not intended in that manner.)

Please let me know if you have any further questions.

-R

Originally posted by catchmeifuca
Hi, Randy, Andrew and all,

Thanks for all the help.

I guess I have an IDE drive.

my oscommerce is based on Linda's SEC and then I added quite a few adds on.

there a 5-6 sites on the server, but this store occupies 95% of the traffic.

if I don't store session into MySQL, is it safe?

I trucated three tables, affiliate_clickthrough, visitors, using_tracking, these tables overall has more than 1,000,000 records, after that, the site is loading a lof faster.

So I would suggest never install these contributions. I soldem use them and they use a lot of resouces. after trucating these tables, MySQL drops from more than 25% of CPU time to less than 3%.

Is 1,000,000 record large? I still don't know whether MySQL is able to handle that many record or some kind of misconfiguration in my MySQL settings.

Randy, what is the largest oscommerce store you've seen? what is the maximam concurrent users allowed for oscommerce? 500, 1000, or more?

I don't know if this has already been answered, but did you make sure that all those tables had proper indexes on them?

Consider setting up one of the tiny web servers (e.g. thttpd or cherokee) and use it to serve images. They run in a tight select() loop and can handle a LOT while using few resources.