Results 1 to 30 of 30

Thread: VPS freezes

  1. #1

    VPS freezes

    Ok, so my VPS is freezing quite a lot of times nowadays. It seems to be going from bad to worse.

    It has about 20 sites on it, not much traffic etc, with 700 Mb RAM.

    About 2 or 3 times a day it becomes unresponsive. No sites load, ssh doesn't work, ftp, mail... During that time ping and traceroutes work fine.

    How can I track the issue that is causing this? what should I look for in logs?

  2. #2
    Join Date
    Oct 2007
    Posts
    1,902
    How much resourced are you allowed to use on that VPS by web hosting provider? Maybe you need more that you have?
    ServerPoint.com - a true hosting company offering online presence solutions since 1998.
    >>Web Hosting, colocation, dedicated servers and virtual private dedicated servers.
    >>>>Wholly owned multi homed network, servers and facilities.

  3. #3
    Tasks: 95 total, 1 running, 94 sleeping, 0 stopped, 0 zombie
    Cpu(s): 0.0% us, 0.1% sy, 0.0% ni, 80.2% id, 19.8% wa, 0.0% hi, 0.0% si
    Mem: 786432k total, 214628k used, 571804k free, 0k buffers

    this is what top reads, with load average: 0.48, 0.75, 0.68

    Also, when the server becomes unresponsive, the load average goes higher than 4

  4. #4
    Join Date
    May 2007
    Location
    Chicago
    Posts
    322
    Is there anything suspicious in the system logs during the time when it's unresponsive? If not, you might try something like sar (see http://linuxreviews.org/man/sar/) to gather more information during the problem times.

    Also, is there any regularity to when it happens? If so, it might be a recurring process either on your VPS or on someone elses on the same physical machine that is sucking up all of the resources when it runs.

  5. #5
    Quote Originally Posted by eu_host View Post
    How can I track the issue that is causing this? what should I look for in logs?
    Like I said in my first post, I am unsure, as I do not know exactly what to look for. There doesn't seem to be any regularity as to what causes this. Or at least I do not see it.

    About the sar command, how can I make it run and store information in a specific file even if I log out of ssh?
    Last edited by eu_host; 03-12-2008 at 10:34 AM.

  6. #6
    What VPS is it? Xen, OpenVZ, Virtuozzo?
    FluidVM :: Manage Xen PV, Xen HVM and OpenVZ
    Multi-Hypervisor, Extensible Virtualization Platform :: Website :: User Forums ::
    Simple Browser based UI-Simple User VPS Control Panel!

  7. #7
    the VPS is on virtuozzo

  8. #8
    Please see if anything is failing in the "failcnt" column in:

    /proc/user_beancounters

    It will give you a fairly good idea on what is wrong.
    FluidVM :: Manage Xen PV, Xen HVM and OpenVZ
    Multi-Hypervisor, Extensible Virtualization Platform :: Website :: User Forums ::
    Simple Browser based UI-Simple User VPS Control Panel!

  9. #9
    all 0 in fialcnt column.

  10. #10
    Hmm.. There seems to be no problem with the limits. AFAIK, the base machine kernel never resets the failcnt for any VPS during its uptime. You probably need to look into the regular logs.
    FluidVM :: Manage Xen PV, Xen HVM and OpenVZ
    Multi-Hypervisor, Extensible Virtualization Platform :: Website :: User Forums ::
    Simple Browser based UI-Simple User VPS Control Panel!

  11. #11
    Join Date
    Oct 2006
    Location
    New York, NY
    Posts
    1,034
    Quote Originally Posted by eu_host View Post
    Like I said in my first post, I am unsure, as I do not know exactly what to look for. There doesn't seem to be any regularity as to what causes this. Or at least I do not see it.
    I apologize if you have already gone down this route, but from the way your describing the problem it sounds like there could be an issue with the physical node itself.

    Even if your on an unmanaged/self-managed VPS plan, I would still go ahead and contact your provider and describe this problem to them. They have access to the actual physical node log files and should be able to determine what might be causing this. Especially if the root cause is another VPS on the node.

  12. #12
    Join Date
    Feb 2005
    Location
    Australia
    Posts
    5,842
    Agreed. This is what jumps out at me:
    Quote Originally Posted by eu_host View Post
    Tasks: 95 total, 1 running, 94 sleeping, 0 stopped, 0 zombie
    Cpu(s): 0.0% us, 0.1% sy, 0.0% ni, 80.2% id, 19.8% wa, 0.0% hi, 0.0% si
    Mem: 786432k total, 214628k used, 571804k free, 0k buffers
    Watch that i/o wait figure during the high-load periods - disk i/o is where the isolation of a vps from its neighbours often breaks down.
    Chris

    "Some problems are so complex that you have to be highly intelligent and well informed just to be undecided about them." - Laurence J. Peter

  13. #13
    Join Date
    Sep 2005
    Location
    Canada
    Posts
    645
    Do you mean freeze or just slow down? A total freeze could indicate a network problem.
    VPSVille.com
    Toronto, London, Dallas, Los Angeles
    Quality VPS hosting on Premium bandwidth

  14. #14
    Join Date
    Mar 2005
    Location
    Labrador, Canada
    Posts
    951
    I agree with foobic.

    - Significant io wait, and
    - Significant load with Cpu(s): 0.0% us, 0.1% sy

    Are indicating an issue on the host node.

    If the provider gives "equal share" cpu another VPS could be hammering the cpu. Otherwise, another VPS could be hammering disk access.

    Only your provider can tell for sure. Work with them. If there's a pattern (e.g., 4:00pm daily) let them know. The pattern may be more complex, so watch carefully for a few days. When you think you got it figured, remind them before the next one is "scheduled".

  15. #15
    There doesn't seem to be a pattern, and yes, they are aware of the situation. By unresponsive I mean that no website loads, power panel does load sometime, but very very hard, and only ping works fine.

    It seems to be working a bit better now (past 36 hours). One other thing. Is there any way I can see the location % of people's who are browsing the sites on the VPS
    . I know the logs for each domain, but that would be my last resort.

  16. #16
    Hi,

    This definitely seems like an overloaded hardware node.
    You provider should be able to locate and isolate the source of the resource hogging.

    Providers usually have graphs of resource usage of each VPS (At least with Xen this is possible)

    Regards,
    Jorge Luis

  17. #17
    I have just reached 24 hours since the VPS hasn't encountered any unresponsive times. Time will tell if the issue was solved or not.

    Any thoughts regarding VPS wide visitor's graph? I really don't want to go through all domains to make the % of visitors from X, Y or Z countries.

  18. #18
    Join Date
    Aug 2007
    Location
    India
    Posts
    117
    Run the following command to find the number of connections from an IP.

    netstat -plan|grep :80|awk {'print $5'}|cut -d: -f 1|sort|uniq -c|sort -nk 1
    Anoop
    █ Freelance Linux Admin
    Email: tracerouteme [at] gmail.com

  19. #19
    thanks tracerouteme, but that only gives me the IP of current visitors. I'd like to see all of them in a log, possibly using it later in webalizer or so.

  20. #20
    Hi,
    I'm not sure if you can do this in OpenVZ (I only use Xen) but you could try to create an iptables rule that logs all incoming connections to the port 80 of your server and then process the log latter.

    Something like this:
    iptables -I INPUT -p tcp --dport 80 --syn -j LOG

    The above rule will log all incoming conections to port 80 to your syslog

    Regards,
    Jorge

  21. #21
    Hello Jluis. thanks. I am on virtuozzo by the way.
    Yet again, after 36 hours going just fine, the VPS was unresponsive again for a few minutes, when load average was as high as 20, CPU usage 100% while 99% was in WA.
    Any thoughts?

  22. #22
    Join Date
    Nov 2006
    Posts
    50
    Open a ticket an explain the issue to them.
    About logging traffic with iptables it is possible like Jorge said to use a -j LOG although you will need some extra tool or script to read those entries from /var/log/messages, and you still need to enable klogd which is a subcomponent of syslogd to log kernel messages (-j LOG), I have been always seeing klogd disabled by default in VPS's, I am not sure why.

    Rui

  23. #23
    Join Date
    Nov 2006
    Posts
    50
    ...and maybe also it would be better to log just SYN packets like Jorge said.

  24. #24
    Join Date
    Nov 2006
    Posts
    50
    Quote Originally Posted by eu_host View Post
    Any thoughts regarding VPS wide visitor's graph? I really don't want to go through all domains to make the % of visitors from X, Y or Z countries.
    After thinking better, I believe it is possible in apache to configure a server wide log file that will log all traffic for all virtual servers (sites) together, then you can parse that log file with awstats which already has support to discriminate country of visitors with use of geoip.

    Rui

  25. #25
    Quote Originally Posted by eu_host View Post
    Hello Jluis. thanks. I am on virtuozzo by the way.
    Yet again, after 36 hours going just fine, the VPS was unresponsive again for a few minutes, when load average was as high as 20, CPU usage 100% while 99% was in WA.
    Any thoughts?
    I think the 99% in WA is the smoking gun you needed

    Basically it says that your processes are spending all of its cycles waiting for IO from the disk.

    I have never seen a server display 99% of WA time so this is definitely something serious with the hardware node or a VPS doing a lot of concurrent IO requests. Your provider needs to take imediate action on this issue since it is also degrading all other VPS on the node.

    Just to give you an example, the most heavilly loaded server in terms of IO that I manage sometimes peeks to 50~60% WA time in top and basically it gets unusable. The server in question hosts a graph processing tool that writes about 10000 files every 5 minutes (Polling cycle)

    Regards,
    Jorge

  26. #26
    Hey Jorge. Well, my provider told me indeed there were performance degradations on my node due to faulty iSCSI. I was told that all VPSs are switched to RAID 10 asap. However, I have experienced degradation since about 1 1/2 months ago and I am not sure if the issue is only this.

    I haven't seen the wa spike up since my last post, so I will have to wait and see if this was the sole problem.

    Anyway, thanks for kind support.

  27. #27
    Join Date
    Nov 2006
    Posts
    50
    Indeed I was afected by that issue also.
    Since then I have not noticed any more problems with performance.
    Perhaps the faulting disk was causing some decay in performance already before it finally failed completely, just speculating.
    If you keep having performance issues I can just advise you to report them again and explain as you have explained here.

    Rui

  28. #28
    Hi Rui. I am actually now thinking about moving to FH in UK datacenter. I have just finished some math and about 40-45% of traffic comes from EU countries.

    If one client of mine reports lag times, I will switch to UK (30 ms as opposed to 140 is quite a difference)

  29. #29
    Join Date
    Nov 2006
    Posts
    50
    Yes lag between U.S.A. and Europe can be around 100 ms or more.
    But I think that will not have big impact on regular web browsing speed.
    In fact I am from Europe and I have a web site in the U.S. and I cant notice an impact on site speed because of that.
    But of course going from 140 ms to 30 ms will make a difference.

    Rui

  30. #30
    I am in Europe too. All my sites are hosted in US. True, when VPS was in tip top shape I didn't see any differences.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •