I ask my host if they had mod_gzip installed, and if not, would they please install it, and here is the answer I received...

"mod_gzip is not installed and will not be. Too much CPU overhead for a shared hosting environment."

Is this true? Does mod_gzip really cause too much CPU overhead for shared hosting?

Depending on the load of the server yes. You figure its going to compress every file 100k and smaller for thousands if not hundreds of thousands of page views a day.... so yes it could easily double the load on a server althou I'd say normally you'd look @ a 25% increase, but these are rough estimates as I've never played with it in a controled test enviroment

Well it can hit the cpu hard if in use alot but if its done right it can realy reduce bandwidth....

This CPU usage is a valid reason but the server's must be doing some heavy loading.

Mod_gzip can run in php without it being installed on the server mind.

People seem to get too confused about this when they think about compression and how it serves up the files. It takes CPU time to serve up the larger files, the compression ratio is faster on CPU time to compress and serve up these files types, than it is on CPU time to serve up a larger file. This is not a valid excuse and if it's set up properly it will not cause any disadvantages at all. It will actually remove load from your CPU when serving up the files. If you are paranoid or uneducated about compression and how it all works and how files are served up, please research it, otherwise you will miss out on a great module by assuming it must equate to using more CPU to compress the file.

Originally posted by 2host.com
People seem to get too confused about this when they think about compression and how it serves up the files. It takes CPU time to serve up the larger files, the compression ratio is faster on CPU time to compress and serve up these files types, than it is on CPU time to serve up a larger file. This is not a valid excuse and if it's set up properly it will not cause any disadvantages at all. It will actually remove load from your CPU when serving up the files. If you are paranoid or uneducated about compression and how it all works and how files are served up, please research it, otherwise you will miss out on a great module by assuming it must equate to using more CPU to compress the file.
Thanks Robert. Do you have any documentation from a reputable source that I can send to my host that supports your statements that mod_gzip uses less CPU power, not more?

Originally posted by chrisb

Thanks Robert. Do you have any documentation from a reputable source that I can send to my host that supports your statements that mod_gzip uses less CPU power, not more?

Well if they have knowledge of it, it will stand to reason. However I believe that the mod_gzip site itself actually has some information about this. They mention it on their site, but people seem to want to claim it's just an empty, unproved claim. But they also have links. Further web site searches about this topic will find more compelling support. However keep in mind that there's a lot of paranoid people claiming things in opposition too (and think it's just too good to be true and there has to be something that will fault).

Yes I would like to see that document also. I installed it one of my servers after reading all the great things on it here and saw load increases of 25 to 40% right away. I have decided to not install it on my other servers. While it does lower the bandwidth and speed the download it also seems to cause a larger strain on the server from what I have seen.

Now I will admit I am not that knowledgable of it so I may not have it optimized. If there is someone that can show me some place that details how to do that I would love to see it.

I have so problem with mod_gzip and Tomcat (specially if I run tomcat through mod_jk), seems like they don't like each other, so, I had to remove mod_gzip :(.

Is it possible to load mod_gzip in my own directories without root access to test it out?

Well, if you want to get fancy you could compile your own copy of apache... you should have enough user privelages to do so, you'd just have to run it on an empty port.

Also as mentioned as above with php you can run it gzip compressed without mod_gzip installed.

Originally posted by chrisb
Is it possible to load mod_gzip in my own directories without root access to test it out?

No, at least as far as I know you cannot.

Hey, Deb, just wondering if futurequest has mod_gzip turned on???

I must have security too high on our machines, or too many specialized modules...definitely something. Even the mod_gzip developers couldnt get it to work on our machines. They finally gave up. I dunno either. I really wish they could have gotten it to work. I was really interested in seeing how much bw it saved us, then switch to mod_hs and see if that saved us even more.

Too bad. Mod_hs (newer and supposedly far better version of mod_gzip) is $1500 per CPU (that's $3000 per machine for us). I decided that at this time I wasnt interested in spending that much if I cant even compare it to mod_gzip in a live environment to see for myself how much better it is. It was interesting that they could get mod_hs to work, but not mod_gzip. I didnt try it though..

Like robert said, the longer the file transfer, the more that gzip will save you on cpu, because the cpu will be utilized more at the beginning but it will be utilized for a shorter period of time overall.

At some exponential point, things will equal out or actually improve.

For very short transfers, you will just have higher cpu at the beginning, and not get the exponential savings of having it run for a shorter time.

There are basically two things that determine the length of the transfer. The speed of the transfer and the size of the file.

mod_gzip should be set up to only compress files over a certain size.

You can't control the speed, but you would see more savings compressing files for your 56Kb customers than your 1.2Mb customers. Most 56Kb modems are already using compression though, so somewhere down the line, your modem vistors aren't equal to your non-modem 56Kb visitors.

For most instances, you will probably only want to compress text files, and not binaries that will already be compressed, if they're being streamed from a web site, anyway.

You should also compile mod_gzip on your machine, and not use any precompiled binaries.

It's understandable that your host would not want to install mod_gzip, as ultimately the bandwidth bill is being passed onto you, the customer. Typically "special" configurations in a shared environment aren't going to be available. That's shared hosting for you. Also, you should expect to pay an hourly rate for the special configuration and any support required if they actually enabled it for you.

Unfortunately the calculation that compression and transmission of a file is easier on the CPU than the transmission of a larger file seems to be somewhat theoretically based - like Monte, in use we found mod_gzip to put a higher load upon the CPU, and it seemed to cause Apache to crash on a more frequent basis.

We have used mod_gzip on a server serving 200-250K pageviews a day and apache never crashes. Of couorse, with 2GB of memory on the server. I plotted the CPU load with MRTG and my recollection was mod_gzip does not consume too much CPU (according to the graph). It concumes some, but not in the range of 25% or so ... probably few percent. But again, this was just my recollection since I no longer have the MRTG plots of the CPU Load with and without mod_gzip. I guess, somebody can test it out to plot the CPU Load with MRTG and turn on/off Mod_gzip for several days ...

I can't imagine why it's actually caused more of a load than reduced it, let alone to be noticeable if that was the case. How did you set it up and configure it? Mark mentioned it's probably not worth having it compress binary files and that's 100% correct. The compression ratio on a binary file would be so minimal and it would not serve any purpose. You only want to compress files that are text (HTML, PHP, CGI scripts and their output actually, text), not compiled files, etc.

You can also set a limit on how large a file needs to be before it will compress it, just as well as up to what file size you'd want to. It will be faster (trust me on this), unless I've just been lucky for years of running it? I've run this on different versions of FreeBSD, Linux Redhat, Debian and Slackware, with all different versions of Apache and using all sorts of modules along with it and I've never experienced a problem.

I have seen people configure it poorly though many times in the past, where they'd have huge amounts of work files in the /tmp directory or worse. Also, another option your clients can have, is to have their files that would be compressed, to be gzipped when they are stored. This would be annoying for files that you modify a lot, but for some content that you don't modify much or ever, you can have it already gzipped and it will just serve up that file without compressing it.

There's obviously some potential problems that can be created when using it, if you don't use it for the purpose it's meant for (like if you were trying to compress everything, including large binary files, image files or whatnot), and you have to implement it for certain file types too, not just certain file extensions, so you don't blindly compress what a client could have named a file to make it appear it's text when it's binary, for example -- an that might have been some of the problem that some people faced.

I didn't do the Apache compile that included it, it was a few months back and I'm not sure how it was configured. I think perhaps it was poorly configured and we didn't play around with it enough to make good use of it.

I'll take your recommendation and with our next Apache recompile, I'll include it and see what results it brings.

Matt

Do you have a test server you can use it on? Just in case you do have any problems, and even though it's nothing that should take a long time of complex tasks or anything (it's pretty simple after a few minutes of reading), it will allow you to test out what you need to before trying it with clients and risking the server possibly crashing.

I don't know of any URL's off hand that I could point you to (I wish I did) that can explain some ways to use it so you can avoid the issues such as a client potentially naming some large (or even within the directive specifications large) warez file to .txt or something and have it chew on that actual binary file, for example. If I can think of some site that might better explain this, I'll post a link.

Originally posted by bitserve
It's understandable that your host would not want to install mod_gzip, as ultimately the bandwidth bill is being passed onto you, the customer. Typically "special" configurations in a shared environment aren't going to be available. That's shared hosting for you. Also, you should expect to pay an hourly rate for the special configuration and any support required if they actually enabled it for you.

I'm very suspicious that the main reason most hosts have for not running mod_gzip is because they are not interested in saving the customer bandwidth charges.
However, if they have experienced problems with it, they may have a valid reason; and I'd prefer they not run it, rather than take the risk that it would cause problems.

I wasn't expecting a special configuration. If I could find proof to show to my host that it doesn't cause more CPU usage, then I was going to kindly suggest they install it server-wide. I don't think a host should charge you for that.

However, with the differing views about it here, I don't feel comfortable recommending it, until I am more convinced.

FHDave: it's good to hear that mod_gzip has never crashed your Apache. However, you have not been in the hosting business but a few months; and that's a short time. My host has been in business for several years, that's why I kept thinking that maybe they know something about mod_gzip that younger hosts have not experienced.

Yes - we can test it on a nonclient box - i'll add it to that ever growing to do list ;)

Originally posted by chrisb


I'm very suspicious that the main reason most hosts have for not running mod_gzip is because they are not interested in saving the customer bandwidth charges.
However, if they have experienced problems with it, they may have a valid reason; and I'd prefer they not run it, rather than take the risk that it would cause problems.

I wasn't expecting a special configuration. If I could find proof to show to my host that it doesn't cause more CPU usage, then I was going to kindly suggest they install it server-wide. I don't think a host should charge you for that.

However, with the differing views about it here, I don't feel comfortable recommending it, until I am more convinced.

...



And you have the right attitude too. Being fair and curious. Trying to find the truth about it, not because they are lying, but to suggest it actually isn't a problem. That's surely not an easy task for a client to tell his web host and have them consider anything in most cases. After all, they say that it's going to cause problems. I wonder if they tried to use it at some point and it did cause problems. If so, if it was truly a problem or just configured poorly. Also if they tested it a very long time ago when it might have actually had some problems when it first came out.

I'm not sure how far you're going to get with them, and I don't know that you're going to find only positive comments about anything. I can't think of anything that people use that someone else hasn't complained about having problems with using, or saying it's difficult to set up, or insecure (i.e., Sendmail _used_ to be, but so did ProFTP, and so did Apache, and so did older kernels, and so did SSH, and on and on) and there will always be people that truly had problems, didn't understand it or just don't like the idea. It seems that this did actually create problems for people here in this thread, while others of us have experienced no problems. I'm not sure how you can find information that will convince you it's wise to suggest to them given those variables.

This module is not the be all, end all solution for everything anyway and it might not make that much of a difference. How many of their clients have the majority of their bandwidth generated by text files? That's to say, how many people generate enough bandwidth from text, HTML, CGI, PHP and other similar files to really cut down that much on their bandwidth to make this of enough of an advantage for the provider or the other clients to care about or consider? I'm not sure given that most bandwidth is due to files like images, zipped files, mp3s, and other such things, in addition with not being comfortable trying to suggest that perhaps they are wrong and there's nothing wrong when they seem to think there is something wrong (and maybe there is, for them) and that they should use it. I'd suggest you continue to investigate it, but don't spend too much time or worry and consider how it would really help your site.

Robert

Did you tried mod_gzip with mod_jk? I'm in mod_gzip and Tomcat mailing list, and I find a lot of people with problems using both modules.

Indeed, I couldn't find one person using both modules without problems.

Hey, Deb, just wondering if futurequest has mod_gzip turned on??? <edited to correct> things like vBulletin's gzip ability will work with us..however mod_gzip itself is not a part of our Apache offering </edited>

Originally posted by voxtreme-matt
Unfortunately the calculation that compression and transmission of a file is easier on the CPU than the transmission of a larger file seems to be somewhat theoretically based - like Monte, in use we found mod_gzip to put a higher load upon the CPU, and it seemed to cause Apache to crash on a more frequent basis.

I can only presume that this would be because sending a file over the network requires very little CPU activity, whereas compressing data requires lots of CPU activity. Proportional to the size of the data, I mean. Everyone seems to be overlooking this.

Kevin

and it seemed to cause Apache to crash on a more frequent basis.

Very true! MOD_GZIP has caused so many crashes on our providers server.
:angry:

fhdave has been doing hosting for a bit longer than 'a couple months'. i think you should listen and listen well to what he is saying - he knows whereof he speaks =]

I don't see how it can cause a load because it surely reduces it here. I run a site which servers over 10 million page views monthly (probably more but I've not checked analog in a good while) and I use mod_gzip on it extensivly for 56kers and below.

Without it, the loading would be far horrible as it's extensivly based on php and almost everyting is backed to mySQL.

Anyways, I can't buy that it would double your cpu load or anything because I've never seen it. If I still shopped for 'shared hosting' -- which I dont, because I have my own servers I wouldn't even consider a host who don't have mod_gzip installed.

Just my opionon.

Looking at it from another point-of-view...

With the costs coming down for better MB, RAM and Network connection, using a P4 2.x instead of P3 1.x, with RAM and Network connection being relative, I can see where in most cases "mod_gzip" will become outdated. A faster CPU and more or better RAM should alleviate the problems "mod_gzip" was used for. Not to say there won't be situations where gzip could or should be used, but for the most part, we will be able to get by -- just fine, without it. Probably save a few hairs along the way as well with not having the potential gzip problems.

P3 1.7 with 512 MB SDRAM and gzip -or- P4 2.4 with 512 DDR RAM and no gzip

With everything else being equal, I know which Server I would choose for Hosting. ;)

mod_gzip is installed on all our servers and we never had problems...
Daily statistics for average server - 1.500.000 requests to apache, ~6-7Gb traffic, load ~2-3, sites/server ~360, Dual P3 1Ghz, 1Gb-1.5Gb RAM

If you have mod_gzip on the server, i recommend you to disable it for SSL sites and move /tmp to big partition ( /home/tmp )

I'm surprised no one mentioned this, or knows this, but your typical apache install will serve compressed content on the fly too. If you're serving static HTML, just gzip the .html file and bingo! Saves you some bandwidth.

IE.

gzip index.html
now it's index.html.gz

look at the difference in file size, then pull it up in your browser.

mod_gzip should be used for dynamic sites, etc... however, i'd be wary of using it in conjuction with SSL. I have heard it causes problems, so it's best avoided :)

Here is a rough idea for what you'll save with text-based data:

host# du -sh test.html
77M test.html
host# gzip test.html
host# du -sh test.html.gz
576K test.html.gz


77 megabytes to nearly half a meg.

More realistic:

host# du -sh beheb10.txt
416K beheb10.txt
host# gzip beheb10.txt
host# du -sh beheb10.txt.gz
144K beheb10.txt.gz


Amazing, no?

Even a big and crowded free shared hosting like Lycos offer gzipping on the fly for all html/texte content :o some host are just dumb

anyone who doesn't use mod_gzip is stupid. People like pages to load 5 minutes ago... nobody wants to wait. Your host is stupid ... is that company run by teens?