Here I'am again stressing you :stickout

first of all, I'm new in WHT but in this short time you (users+operators) have helped me soo much that I'd like to say "Thanks! Thanks a lot for all your comments and for your time!"

Stop beeing sentimental ;)

Well, since quite a lot of time I'm comparing tons of hosts, and during this time I've seen which companies are really good in support, and which are not, which have happy customers (i saw it here), and which are unhappy.

But I did not find the right one for me, yet. :bawling:

This is because (as you proably know) my site makes about 30gigs of traffic, and I'm not sure if the hosts will allow me a cpu/memory usage as I need.

The last thing I want is to be kicked out coz of usage policies ;)

Due to the fact I'm a newbie... I'm asking you, if there is a command where I can see exactly HOW the memory usage and CPU usage of my website is.

I've a virtual account at iserver.net, running freebsd... but is there a command which will give me an average info about the usage (I'm not talking about the statistics/bandwith) of cpu and memory?

I'd like to be able to make my "pre-sales inquierys" in a more inteligent way... than I did now, by giving them the exact usage of my site :)

Thanks again :)

P.S. should this command be "top", is there a way to get an average info during the f.eg. last 24 hours? Can I post my 'top' output in here, and hear your comments? cheers ;)

The only ones i know are for the entire server im sure there is a way to get certain sites cpu but.

If you type top or uptime that will show you for the entire server.

is it the same for ALL hosting companies? will UP and TOP always say the loading of the ENTIRE server? *lol* it would be cool, coz so all the loading is not my fault ;))

I don't know about real linux boxes but when i type top on any cobalt account with telnet it shows the entire server i just did it on my virtual account with Ghoulnet im quiet shocked by the cpu level.

Useful side note - you can read the manuals for 'top' and another useful command, 'ps', right from the command line. Try typing

man top

or..

man ps

HINT: Cursors to scroll, 'q' to quit

FYI the reason you get to see the load across the entire server is because all this command does is read the information out from /proc. Try:

cat /proc/cpuinfo
cat /proc/meminfo
cat /proc/loadavg

To see the info in it's raw format...

Note - don't play around in /proc if you are logged in as root ;o)

Also, I don't know if this helps (it might be a little too technical, not possible with iServer or both!) but I believe you get your own private Apache webserver at iServer. If this is the case then it might be worth looking at the following URL to see if you can 'tune' Apache to your website's requirements. Webserver tuning can yield excellent results when done properly...

http://www.apache.org/docs-2.0/misc/perf-tuning.html

HTH

thanks complexmind... here is what i did find via telnet:

myusername: {77} % pwd
/proc
myusername: {78} % ls
45965 46521 46755 46780 46858 46966 47040 47133 47143 curproc
46006 46568 46757 46794 46861 47020 47106 47141 68176
46302 46592 46771 46842 46863 47038 47117 47142 96449
myusername: {79} %

but when trying something with cat i do always get:
No such file or directory. when i try to open it with 'vi' it seems to be empty :(

thanx for the url... but it is really tooo complex for me ;)

First I was thinking.. I'm a newbie and I don't know commands.... but now I'm really wondering that there is no easy tool available to measure the cpu/memory usage of a virtual server :( how do he hosting companies do?

Is there really no command available? :confused:

Looking at the contents of your /proc, those contents are probably down to the way iServer's systems work. On a 'normal' system you would have got a more 'useful' listing...

No 'cat' command? That surprises me - sorry I don't have much experience with iServer otherwise I might be able to help more.

Basically, unless you have the time and the programming skills to write your own server statistics then I doubt there is a low-cost option available to you...

FYI the hosting companies handle it in two ways:

1) They have a method of limiting your usage (which also means they have access to the statistics, but they're not likely to share ;)

or

2) They don't have any real monitoring in place and wait for their servers to be hammered, before reacting to the problem and kicking you off or getting you to upgrade...

Whilst the iServer system is great for the features it provides, it's not that scalable at the virtual server level since (as far as I am aware) their virtual servers are hosted on individual machines without the ability to cluster or load-balance the available server resources.

With 'traditional' ISPs' systems (ie with clustered servers) this is not so much of an issue since the load can be accomodated much more easily. This is why the vast majority of ISPs charge for bandwidth and not resource usage...

The only ISP's I know of that can provide guarantees are expensive, since they use 'Virtual Machines' as a way of providing % of CPU and % of RAM to an individual customer. This means far lower customer:server density and increased pricing as a result...

The tools you need are expensive, and until there is an open source solution or somebody brings such a product to the mass market (read: low price) this is unlikely to change.

HTH

mark, thanks a lot for all the info you gave me.

myusername: {108} % ls
Mail cronjob etc pub tmp www
bin dev ftp sbin usr
compat eggdrop perl.core shlib var

pwd -> usr/home/myusername

perhaps i did look into the wrong directory... I went down to the servers root... hemm... *lol* I've spent 10 minutes for browsing through some of those directories... without finding /proc

Could you please suggest me in which one I shall search?

Sorry for my huge ignorance :) Thanks!

Just an FYI, FreeBSD machines don't have those cpuinfo and memory files in /proc. Linux machines do. That's one of the very few things I miss about Linux. In FreeBSD, only the process id's are listed in /proc.

You have to do much more digging in FreeBSD to find out about the machine than you do in Linux. I like that for security reasons, but for programming reasons it's a real pain in the rump.

Hope that clears up some /proc confusion.

I wouldn't bother with this. Whatever readings you come up with (assuming they have some accuracy) apply to that particular server you are hosted on. The amount of CPU time and ram and load etc... this depends on many things which are beyond your control. The type and speed of the CPU, amount and type of Ram, the kind of harddisk, number of sites on the server, etc... many things you cann't control and are not guaranteed to be duplicated in other hosts.

If you want to give a prospective host a clear idea about your site, a disk space and bandwidth used numbers would be great help. Also if you use any CGI programs or PHP scripts etc.. let them know what they are and how heavily they are used.

There's just a chance this might work... try "ls -l /var/acct/" and "/usr/sbin/sa -im". If process accounting is enabled -- and that is a big IF -- and the security is feable enough that you can access those files -- which is probably more likely, however depressing that is -- then you'll get to see how much cpu time, memory, and disk I/O each user has consumed. Anyway, you might as well try it.

thanks... the first command did not work... but the second did! here is the output i get while I'm running the heaviest script (Links2flat by gossamer, 14000 entries), but it really looks like this command is an average indication... or am I wrong?


myusername 49305 66.89cpu 138540tio 83335649k*sec


I do not understand this output... even using 'man' isn't helping me that much. Could you help please?

man:
avio Average number of I/O operations per execution
cp Sum of user and system time, in minutes
cpu Same as cp
k CPU-time averaged core usage, in 1k units
k*sec CPU storage integral, in 1k-core seconds


BTW... the techspecs of the hardware of this server are:
- 550 MHz Intel Pentium III CPU
- 512 MB 50 ns ECC Viking memory
- 3 10,000 RPM Seagate HDDs:
- Dual 36 GB user volume with RAID 1 mirroring
- 36 GB 24-hour user volume backup
- 2 "Ultra SCSI 2" disk I/O channels
- FreeBSD

Originally posted by tortellini
thanks... the first command did not work... but the second did!

Ok, someone misconfigured their server. Why did they o-r *that* directory if they weren't also going to o-x it?


myusername 49305 66.89cpu 138540tio 83335649k*sec


What sa -im does is list, per user (-m), the total number of processes executed (49305), cpu time used (66.89 minutes), number of I/O operations (138540), and "cpu storage integral" (83335649 kilobyte-seconds) used since the logfile was last rotated.

That last part is why I wanted to see the contents of that directory, so that I could say whether that 67 minutes of cpu time was all between the beginning of the day and now, or if it was starting from whenever your account was created.

Now to work that out, probably the easiest solution would be to "grep periodic /etc/crontab", "grep accounting /etc/*.conf", and "grep accounting /etc/defaults/*.conf". Also run "/usr/sbin/sa -im" and "date" again. (If the logs are being rotated daily we need to know how far through the day it is when we are looking at them).

An alternative would be to execute "/usr/sbin/sa -im" every 6 hours or so for a day; if the logs are being rotated daily you'll be able to see a sharp drop in those numbers; if the logs aren't being rotated daily you'll be able to see how much cpu time you're using from the rate at which the values are increasing.

Originally posted by Technics
The only ones i know are for the entire server im sure there is a way to get certain sites cpu but.

If you type top or uptime that will show you for the entire server.

I didn't notice this comment until now, sorry about the late reply. While uptime will indeed normally display the load average for the entire server, load average is an ugly metric and tells you nothing about which programs are using the cpu time. However, top and ps on many systems will only display processes "owned" by the calling user; on FreeBSD, for example, there is a sysctl variable which controls this (and there are very good reasons of security why people should not be able to see each others processes).

myusername % grep periodic /etc/crontab
59 0 * * * root periodic daily
30 3 * * 6 root periodic weekly
30 5 1 * * root periodic monthly

myusername % grep accounting /etc/*.conf
/etc/login.conf:# This file controls resource limits, accounting limits and
/etc/login.conf:## - no time accounting, restricted to access via dialin lines
/etc/login.conf:## Example standard accounting entries for subscriber levels
/etc/rc.conf:accounting_enable="YES"

myusername % grep accounting /etc/defaults/*.conf
/etc/defaults/periodic.conf:# 310.accounting
/etc/defaults/periodic.conf:daily_accounting_enable="YES"# Rotate acct files
/etc/defaults/periodic.conf:daily_accounting_compress="NO"# Gzip rotated files
/etc/defaults/periodic.conf:# 200.accounting
/etc/defaults/periodic.conf:monthly_accounting_enable="YES"# Login accounting
/etc/defaults/rc.conf:accounting_enable="NO" # Turn on process accounting (or NO).

myusername % grep accounting /etc/*.conf
/etc/login.conf:# This file controls resource limits, accounting limits and
/etc/login.conf:## - no time accounting, restricted to access via dialin lines
/etc/login.conf:## Example standard accounting entries for subscriber levels
/etc/rc.conf:accounting_enable="YES"


thank you so much for yor help! :D I rotate the logfiles of my site "manually" the last time was about 12 hours ago (but the last 12 hours where the heaviest during all the day). I will run the sa command every 6 hours, without rotate the files... and then post the results in here. Thank you very, very much :)

Originally posted by tortellini
myusername % grep periodic /etc/crontab
59 0 * * * root periodic daily

myusername % grep accounting /etc/*.conf
/etc/rc.conf:accounting_enable="YES"

myusername % grep accounting /etc/defaults/*.conf
/etc/defaults/periodic.conf:daily_accounting_enable="YES"# Rotate acct files


(I've pulled out the important lines above). This says that process accounting is enabled, the process logs are being rotated, and said rotation is being done at 00:59 every day.

thank you so much for yor help! :D I rotate the logfiles of my site "manually" the last time was about 12 hours ago (but the last 12 hours where the heaviest during all the day). I will run the sa command every 6 hours, without rotate the files... and then post the results in here.

When you are rotating your logfile is irrelevant, the question (now resolved) was when the server is rotating *its* logfiles. If you can just run "/usr/sbin/sa -im" and "date" once now -- and while you're at it please run "/usr/sbin/sa -im /var/account/acct.0", we'll be able to say quite accurately how much cpu time you're using. (There's no need to run sa every 6 hours as I had earlier suggested, that was merely an alternative method of working out when the logfiles were rotated.)

myusername % date
Thu Jun 7 01:36:54 MDT 2001

myusername {81} % /usr/sbin/sa -im
...
myusername 1290 1.96cpu 7174tio 2914171k*sec
...

myusername % /usr/sbin/sa -im /var/account/acct.0
...
myusername 49890 79.62cpu 144534tio 85215385k*sec
...


Thanks again :) so what do you say... will everybody kick me as soon as I move to another server ;)? Is my use in average for a virtual? :D

Short note: I build all the pages with gossamer-thread's Links2 flat once daily. I can feel that I slow down the entire server for about 15 to 20 minutes (the time needed to build). During the day there are some other cgi's called quite often, f.eg. the privacy mail script (a mod of Links to be able to use it as a classified ads site)

Originally posted by tortellini
Thanks again :) so what do you say... will everybody kick me as soon as I move to another server ;)? Is my use in average for a virtual? :D

Yesterday you used almost 80 minutes of CPU time. So far today -- that is, in the first 38 minutes of the day -- you've already used 2 minutes and there is no reason to suppose that will not reach the same level of about 80 minutes by the end of the day.

Although this doesn't tell us what sort of processor is being used here (80 minutes on a 486 is far less work than 80 minutes on a P4-1700!), assuming it is something modern (you could check this by running "grep CPU /var/run/dmesg.boot") you're looking at a pretty major cpu load there. Based on a 12-hour day (that is, peak = average*2) and the fact that if cpu load due to cgis ever exceeds about 50% the performance impact will be significant, no more than 4 sites like yours should be hosted on the same server. I'd expect to pay a surcharge of about $50-100/month for such high levels of cpu usage.

thanks for your help... this is the confirmation to what I was scared of ;)

PU: Pentium III/Pentium III Xeon/Celeron (547.18-MHz 686-class CPU)

and as told before the hardware is:
- 550 MHz Intel Pentium III CPU
- 512 MB 50 ns ECC Viking memory
- 3 10,000 RPM Seagate HDDs:
- Dual 36 GB user volume with RAID 1 mirroring
- 36 GB 24-hour user volume backup
- 2 "Ultra SCSI 2" disk I/O channels
- FreeBSD

Thanks again for your help, I will give this info when I send emails to the sales office of the hostingcompanies. Should I not be able to find somebody disposed to host me as a virtual user, what should the minimal techspecs of a dedicated be?

If you tell hosting companies that you've been using 80 minutes/day on a 550MHz processor that should be all they need to know; your I/O and memory usage are quite moderate in comparison to the cpu usage.

I'd guess that you'll have trouble finding virtual hosting companies which will give you a reasonable deal here; virtual hosting companies are generally setup around the concept of providing web servers, not providing cpu servers, and they tend to not like the idea of any sort of high-cpu-usage sites.

Specs for a dedicated server? More or less anything. A Celeron 366 with 256MB of RAM should easily be enough. Even a Raq (shudder) would probably suffice. This might well end up being the cheaper route even if you can find hosts willing to accept your unusally large cpu burden.

Well, it sounds good... celeron+ram....

I admit that I'm running an eggdrop BOT on this vserver.... how much would it reduce the CPU usage, if I'd leave the bot away (aprox.)? Thank you so much and sorry for all my questions!

Originally posted by tortellini
I admit that I'm running an eggdrop BOT on this vserver.... how much would it reduce the CPU usage, if I'd leave the bot away (aprox.)?

Well I know basically nothing about irc, but I've asked a friend who does and he tells me that eggdrops generally use quite insignificant amounts of cpu time (under a minute per day). You could check by running "ps -u" which will show you (among other things) when the process was started and how much cpu time it has used.

thanks a lot... this WAS my last question. You did really help me A LOT.... hemm... should you come to my parts... there will always be a lot of fresh tortellini and chianti for you *lol*

t h a n k s

Another comment: Not many hosts will like that you are using eggdrop or any IRC.:rolleyes:

tortellini... STOP STOP STOP

your smilies count is maxing out the WHT server !!

:D

Web browsers will only fetch each image once, actually, so it doesn't matter how many times he smiles... only how many different ways he smiles. ;)

hemmm just do not charge me for bandwith please ;)

Didn't you know that bandwidth is unlimited and therefore there is no need to charge for it?
:D

I've another question :D sorry for this...

How can I see the average amount of memory (ram+swap), I'm using during the day? I don't know if this is already visible with the commands you have suggested me before. Could you please help me again?

Cheers!! :stickout

Eat more spaghetti.
Forgetting about hosting headaches,
hmmm Food Yummm.....

Thanks for the tip. :D