Web Hosting Talk







View Full Version : How to optimize apache for image hosting? 1 Million+hits :(


Hands-on Mark
08-17-2005, 05:17 PM
View:
http://img101.imageshack.us/img101/4153/explode1do.gif
note that on those days I am getting, 700,000 + hits, and my apache is failing every 15 minutes or so because of the requests I am getting.

Anyway how to fix this? anyone know how to?

Server is a celeron, unless layeredtech can get me that server. :(

sshepherd
08-17-2005, 05:31 PM
someone with experience is going to need to know a heck of a lot more about your setup and hosting scripts being used before they can give any specific help.

The only obvious advice I can offer is get rid of apache, get lighttp or one of the other web servers that are optimized for static file hosting. They can often outperform apache 5-10x, handle double the connections if not more.

Hands-on Mark
08-17-2005, 05:40 PM
Well, server is a celeron like I said, with a gig of ram,
apache 1.3.33
No mysql is involved
I'll look into lighttp thanks :)

sshepherd
08-17-2005, 05:45 PM
When you do some research on lighttp you'll find info/link to a number of similiar web servers optimized for static hosting. Litespeed is another but the higher end versions cost $$$.

If there's no mysql, it sounds like you're purely facing a limitation in the amount of traffic your apache installation can handle. Have you tried optimizing it, turning settings like keepalives off, removing unneeded modules?

Hands-on Mark
08-17-2005, 06:00 PM
Nope. I've never had to deal with this ammount of traffic before.

edit: anyone know what imageshack uses?

2detailed
08-17-2005, 07:17 PM
Apache/2.0.52 (Unix)

At www.imageshack.us -- not sure if thats what there image servers use. Have you thought about creating your own image serving daemon?

hiryuu
08-18-2005, 12:54 AM
That hardly seems productive. There are plenty of razor thin web servers around (lighttpd, thttpd, litespeed, boa, tux), and his chances of reinventing the wheel better are rather slim (regardless of skill level). Eventually the bottlenecks (disk, memory IO, network connection) will be bottlenecks no matter how efficient you make the shoveling.

gogocode
08-18-2005, 06:12 AM
As others have said, if you don't need everything Apache provides, choose a light weight alternative, such as khttpd ( http://www.fenrus.demon.nl/ ) which is integrated into the linux kernel itself.

PerfTuner
08-18-2005, 05:38 PM
Apache2 can do surprisingly well when serving static content, providing your OS kernel isn't too old.
highperformance-std.conf file from it's distribution has a good example of settings.
700 thousand hits per day wouldn't be any problem even if your celeron box was 5 year old and had 64 megs of RAM. :)

azizny
08-18-2005, 11:41 PM
FYI: Imageshack uses more than 1 server for its image hosting service :)..

Peace,

Hands-on Mark
08-19-2005, 04:52 AM
Originally posted by azizny
FYI: Imageshack uses more than 1 server for its image hosting service :)..

Peace,

They look like they are using over a hundred, I may be wrong, they are uploading now to 114.imageshack.us and 120.imageshack.us

azizny
08-19-2005, 10:36 AM
Originally posted by Mark L
They look like they are using over a hundred, I may be wrong, they are uploading now to 114.imageshack.us and 120.imageshack.us

Those numbers are sub-domains and have nothing to do with the number of servers.... and no, they aren't using that much...

Peace,

riverpast
08-19-2005, 03:26 PM
Originally posted by azizny
Those numbers are sub-domains and have nothing to do with the number of servers.... and no, they aren't using that much...

Peace,

Each of their subdomain has its own IP address, and may be using one server, or maybe multiple subdomains are located on the same server. We don't know.

Their subdomain is up to 360+ now.