Is it possible to make each Httpd thread take up less ram?

last pid: 19434; load averages: 0.11, 0.07, 0.05 up 3+18:18:49 16:15:15
220 processes: 1 running, 219 sleeping
CPU states: 0.0% user, 0.0% nice, 1.2% system, 0.2% interrupt, 98.6% idle
Mem: 264M Active, 1472M Inact, 161M Wired, 104M Cache, 214M Buf, 9904K Free
Swap: 2048M Total, 276K Used, 2048M Free

PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND
229 bind 2 0 99716K 98516K poll 1 35:31 0.00% 0.00% named
7029 root 2 0 8864K 5528K select 1 0:41 0.00% 0.00% httpd
7044 apache 2 0 8984K 5732K sbwait 1 0:30 0.00% 0.00% httpd
7038 apache 2 0 8936K 5676K sbwait 0 0:30 0.00% 0.00% httpd
7047 apache 2 0 9316K 6184K sbwait 0 0:30 0.00% 0.00% httpd
7048 apache 18 0 8936K 5684K lockf 0 0:30 0.00% 0.00% httpd
7050 apache 2 0 8936K 5680K sbwait 1 0:30 0.00% 0.00% httpd
7036 apache 2 0 8936K 5680K sbwait 0 0:30 0.00% 0.00% httpd
7049 apache 2 0 8936K 5676K sbwait 0 0:30 0.00% 0.00% httpd
7058 apache 2 0 8964K 5708K sbwait 0 0:30 0.00% 0.00% httpd
7057 apache 2 0 8936K 5680K sbwait 1 0:30 0.00% 0.00% httpd
7041 apache 18 0 8936K 5676K lockf 0 0:30 0.00% 0.00% httpd
7037 apache 2 0 8936K 5680K sbwait 1 0:29 0.00% 0.00% httpd
7040 apache 2 0 8936K 5676K sbwait 1 0:29 0.00% 0.00% httpd
7045 apache 18 0 8936K 5680K lockf 0 0:29 0.00% 0.00% httpd

yes, you could try mod_l33t from http://www.l33t.ca/freecrud/

Originally posted by roly
yes, you could try mod_l33t from http://www.l33t.ca/freecrud/

Thanks, have you tried it yourself?

no, i have not

Turn off the modules you dont need....other then that nothing I know of.

In particular, remove mod_perl and mod_php. Apache processes should be no more than 1MB each once you trim out the junk.

This is slightly OT to your question, but what about bind? It takes 100MB of your RAM. Do you have many zones running on this machine?

I'm guessing that BIND's bloatedness is a result of caching recursive queries (note the time count; there is no way it could have spent that long answering nonrecursive queries).

Well, that, along with the fact that it is BIND, of course.

mod_perl and mod_php

How do I know if any of my scripts need them or not? What are they for?

you can run both as CGI, althrough I know that cperciva know more than I, I thought that run perl and php as module would be more resources efective.

We will have to wait for cperciva to get back =) He's genius

Yeah, ok, I'm awake now, what was the question?

Oh, mod_perl and mod_php again, right. Ok...

Perl and PHP interpreters take a certain amount of memory. This amount varies, of course, but expect it to be at least a MB each. In addition to this, significant memory is spent on "caching" scripts or modules once they are used. The net result is that each Apache process can climb to 10-20MB.

If you run perl and php scripts as normal cgi scripts, apache will fork off a process to exec the script. Instead of keeping several copies of the perl and php interpreters constantly in memory, the appropriate interpreter is loaded only when necessary. It gets loaded, it runs, it exits, and the memory is once again free.

Of course there is a trade off, in this case execution time. Using mod_perl and mod_php has the major benefit of eliminating the need for process forking; consequently, if you're only ever running "hello world" scripts, mod_php and mod_perl will be faster... until you run out of memory.

Given that memory is now far more often a limiting factor than cpu time, forking perl and php scripts is in most cases a much better solution than carrying the interpreters around "in your back pocket" all the time.

(There are also security and accounting issues -- mod_perl and mod_php run with apache's priviledges, while without them you can run scripts through suexec -- but that's a different matter.)

Ok, do I just edit my httpd.conf and remove the lines? How do I know what's safe to remove?

# Example:
# LoadModule foo_module libexec/mod_foo.so
LoadModule vhost_alias_module libexec/mod_vhost_alias.so
LoadModule env_module libexec/mod_env.so
LoadModule define_module libexec/mod_define.so
LoadModule config_log_module libexec/mod_log_config.so
LoadModule mime_magic_module libexec/mod_mime_magic.so
LoadModule mime_module libexec/mod_mime.so
LoadModule negotiation_module libexec/mod_negotiation.so
LoadModule status_module libexec/mod_status.so
LoadModule info_module libexec/mod_info.so
LoadModule includes_module libexec/mod_include.so
LoadModule autoindex_module libexec/mod_autoindex.so
LoadModule dir_module libexec/mod_dir.so
LoadModule cgi_module libexec/mod_cgi.so
LoadModule asis_module libexec/mod_asis.so
LoadModule imap_module libexec/mod_imap.so
LoadModule action_module libexec/mod_actions.so
LoadModule speling_module libexec/mod_speling.so
LoadModule userdir_module libexec/mod_userdir.so
LoadModule alias_module libexec/mod_alias.so
LoadModule rewrite_module libexec/mod_rewrite.so
LoadModule access_module libexec/mod_access.so
LoadModule auth_module libexec/mod_auth.so
LoadModule anon_auth_module libexec/mod_auth_anon.so
LoadModule dbm_auth_module libexec/mod_auth_dbm.so
LoadModule digest_module libexec/mod_digest.so
LoadModule proxy_module libexec/libproxy.so
LoadModule cern_meta_module libexec/mod_cern_meta.so
LoadModule expires_module libexec/mod_expires.so
LoadModule headers_module libexec/mod_headers.so
LoadModule usertrack_module libexec/mod_usertrack.so
LoadModule unique_id_module libexec/mod_unique_id.so
LoadModule setenvif_module libexec/mod_setenvif.so
<IfDefine HAVE_SSL>
LoadModule ssl_module libexec/libssl.so
</IfDefine>
<IfDefine FRONTPAGE>
LoadModule frontpage_module libexec/mod_frontpage.so
</IfDefine>
LoadModule throttle_module libexec/mod_throttle.so
LoadModule php4_module libexec/libphp4.so


# Reconstruction of the complete module list from all available modules
# (static and shared ones) to achieve correct module execution order.
# [WHENEVER YOU CHANGE THE LOADMODULE SECTION ABOVE UPDATE THIS, TOO]
ClearModuleList
AddModule mod_vhost_alias.c
AddModule mod_env.c
AddModule mod_define.c
AddModule mod_log_config.c
AddModule mod_mime_magic.c
AddModule mod_mime.c
AddModule mod_negotiation.c
AddModule mod_status.c
AddModule mod_info.c
AddModule mod_include.c
AddModule mod_autoindex.c
AddModule mod_dir.c
AddModule mod_cgi.c
AddModule mod_asis.c
AddModule mod_imap.c
AddModule mod_actions.c
AddModule mod_speling.c
AddModule mod_userdir.c
AddModule mod_alias.c
AddModule mod_rewrite.c
AddModule mod_access.c
AddModule mod_auth.c
AddModule mod_auth_anon.c
AddModule mod_auth_dbm.c
AddModule mod_digest.c
AddModule mod_proxy.c
AddModule mod_cern_meta.c
AddModule mod_expires.c
AddModule mod_headers.c
AddModule mod_usertrack.c
AddModule mod_unique_id.c
AddModule mod_so.c
AddModule mod_setenvif.c
<IfDefine HAVE_SSL>
AddModule mod_ssl.c
</IfDefine>
<IfDefine FRONTPAGE>
AddModule mod_frontpage.c
</IfDefine>
AddModule mod_php4.c
AddModule mod_throttle.c
AddModule mod_perl.c

To remove mod_perl, mod_php:

Comment out (by placing a hash mark (#) in front of the lines:

# LoadModule php4_module libexec/libphp4.so

&

# AddModule mod_php4.c
# AddModule mod_perl.c

Hmmm.. there should have been a

LoadModule perl_module modules/libperl.so in there too. If you have it, comment that out too.

hi :rolleyes: my httpd.conf looks quite similar concerning the modules.... are there other "obsolete" modules which can be removed? thanks for this!

Something else to keep in mind, that I don't see anybody else mentioning...

If you're running on Linux, anyway, those httpd processes aren't really using that much memory each. Linux uses "shared copy-on-write" processes, which means that (in a nutshell) only the differences between each forked copy of a process are kept in memory. (That's oversimplifying, but it works for the purposes of discussion.) So, the main apache code, PHP interpreter, etc., is only stored in memory once, even though the usage is reported as part of the size for each process.

If you look at the actual memory usage of your machine, it's likely to be well below the sum of all the process sizes shown in a listing.

http://httpd.apache.org/docs/mod/index-bytype.html

Here is a list of modules included in Apache 1.3 that are potential candidates for removal. You could go through this, and remove the ones that you aren't using. In particular,

mod_autoindex **This is often considered a security whole anyway***

mod_asis
mod_setenvif
mod_vhost_alias **UNLESS THIS IS HOW YOU DO VHOSTING**

Any of the Access Control Methods you don't use:
mod_access
mod_auth
mod_auth_anon
mod_auth_dbm
mod_auth_digest
mod_digest

mod_include ***If you don't allow SSI***

mod_status ***if you don't use it for reporting/diagnostics***
mod_info ***some consider this a security hole anyway***

mod_imap ***users would be forced to use client side imaps (most do anyway)***
mod_proxy ***if apache isn't a proxy server***
mod_mmap_static ***neat module, but you're probably not using it**

mod_example **if its there, kill it. It's just for developers to use as a model**


Definitely kill any of the obsolete ones if they are there, but they probably aren't:
mod_browser
mod_cookies
mod_dld
mod_log_common


Not to mention any third party modules listed that you aren't using. I would strongly recommend that if you see a module that you don't think that you use, check to see what it is at the apache module repository:


http://modules.apache.org/search

Jake
-- I think I really like vanilla--

Can anyone else confirm that what Travis is saying is true? Any links where I can read more about this? It's not that I doubt you Travis, it's just that I haven't read that the process memory usage info wasn't accurate. Great to learn that though... that's really exciting!

Jake
--now considering adding another cool module or two to his test box at home--

There's some brief additional information about this feature here:

http://www.faqs.org/faqs/linux/info-sheet/

Search for "shared" to find it quickly.

As an anecdotal example, I admin a dedicated server that runs a very busy forum. It uses persistent MySQL connections, and typically has about 150-250 MySQL processes running. The size of each process is reported as about 80 megs because of the increased buffer and cache sizes I'm running this installation with. The machine has one gig of memory. Even with only 150 processes at 80 gigs each, that's about 12 gigs of memory required! Obviously, it's not really using this much.

Only what's different between each process consumes memory. (Data pages, etc.) Everything else (code segments, common data, etc.) is shared between the processes.

Originally posted by Travis
There's some brief additional information about this feature here:

http://www.faqs.org/faqs/linux/info-sheet/

Search for "shared" to find it quickly.

As an anecdotal example, I admin a dedicated server that runs a very busy forum. It uses persistent MySQL connections, and typically has about 150-250 MySQL processes running. The size of each process is reported as about 80 megs because of the increased buffer and cache sizes I'm running this installation with. The machine has one gig of memory. Even with only 150 processes at 80 gigs each, that's about 12 gigs of memory required! Obviously, it's not really using this much.

Only what's different between each process consumes memory. (Data pages, etc.) Everything else (code segments, common data, etc.) is shared between the processes.


I semi good example of something like this is the free command:


root@grep [~]# free -m
total used free shared buffers cached
Mem: 500 485 14 0 38 162
-/+ buffers/cache: 283 216
Swap: 517 127 390


Free says I have 14mb free ram, but without the buffering and cache, I have 216mb free....

Like I said, not the greatest example, but might help clear things up...

Well, that's a different behavior you're describing. Linux will use most of free RAM as for caching and buffering. So, if you have 150MB free, Linux might allocate 140MB for caching. While you only have 10MB "free", in reality you have 150MB free, and 140MB has been temporarily allocated for cache. Linux will release any portion of the 140MB necessary when applications request more memory.

last pid: 650; load averages: 0.10, 0.04, 0.01 up 0+00:02:25 16:54:38
251 processes: 1 running, 250 sleeping
CPU states: 0.4% user, 0.0% nice, 2.7% system, 0.0% interrupt, 96.9% idle
Mem: 69M Active, 23M Inact, 42M Wired, 8504K Cache, 18M Buf, 1869M Free
Swap: 2048M Total, 2048M Free


I think I messed up my apache. Why does it say 23M Inact?

For the best performance, you would like to have as little memory as possible marked free. Most *nixes will cache any functions to speed them up next time it is run, so your server will run much faster with less memory just sitting there, but being used effectively.