Results 1 to 24 of 24
  1. #1
    Join Date
    Mar 2006
    Location
    Servers
    Posts
    1,588

    Virtuozzo is killing /usr/bin/mysqld_safe

    Hello,
    Virtuozzo 3.0 is killing VPS's /usr/bin/mysqld_safe process but leaving /usr/sbin/mysqld UP which is causing cPanel to be unable to automatically restart MySQL after that. Anybody have a clue how this problem can be solved ?



    Thanks,

  2. #2
    Join Date
    Apr 2005
    Location
    Cochin
    Posts
    2,446
    It cannot be VIrtuozzo, but the server administrator as a part of maintaining node stability. Make sure you are not overusing your resources
    Sam
    Supportlobby.com - Expertise in Windows/Linux server support, IaaS and PaaS Management
    Spiralbean.com - Custom Software Development
    Email: exec @ activelobby.net | Skype: unni_active

  3. #3
    Actually Virtuozzo does kill process when it runs out of memory. Thats the problem with Virtuozzo, you depend on the node total resources, and if its oversold then you dont get enough memory and process start to die.

    Try to get guaranteed minimum memory from your provider or see if your VM is not running out of RAM.

  4. #4
    Join Date
    Mar 2006
    Location
    Servers
    Posts
    1,588
    Hello,
    Yes the problem is version 3.0 of Virtuozzo is killing randomly sometimes. Never had such a problem with version 4.0.

  5. #5
    Join Date
    Sep 2008
    Location
    Dallas, TX
    Posts
    4,552
    Quote Originally Posted by WebHostDog View Post
    Hello,
    Yes the problem is version 3.0 of Virtuozzo is killing randomly sometimes. Never had such a problem with version 4.0.
    Why don't you ask why your provider hasn't upgraded to 4.0 yet? 3.0 is terribly old.
    Jacob Wall - GetCloak.com

  6. #6
    Join Date
    Mar 2006
    Location
    Servers
    Posts
    1,588
    I am the admin of this. Yes 3.0 is old yes but why to kill MySQL safe like that? This is causing MySQL is not restarted anymore.

  7. #7
    Join Date
    Sep 2008
    Location
    Dallas, TX
    Posts
    4,552
    Quote Originally Posted by WebHostDog View Post
    I am the admin of this. Yes 3.0 is old yes but why to kill MySQL safe like that? This is causing MySQL is not restarted anymore.
    Have you brought it up to Parallel's? I know you probably don't like their technical support, nor do many people that use Parallel's products but I think it will be beneficial in this kind of situation.
    Jacob Wall - GetCloak.com

  8. #8
    Join Date
    Mar 2006
    Location
    Servers
    Posts
    1,588
    They was not helpful at all. I was thinking about some safe list this VZ have but I am not pretty sure how this one this is structured.

  9. #9
    Quote Originally Posted by WebHostDog View Post
    I am the admin of this. Yes 3.0 is old yes but why to kill MySQL safe like that? This is causing MySQL is not restarted anymore.
    It's the OOM manager that may be killing the processes and not the actual Mysql service. Check the logs on the hardware node:

    dmesg | grep VEID

    Your VPS may need some more guaranteed
    | LinuxHostingSupport.net
    | Server Setup | Security | Optimization | Troubleshooting | Server Migration
    | Monthly and Task basis services.
    | MSN : madaboutlinux[at]hotmail.com | Skype : madaboutlinux

  10. #10
    Join Date
    Mar 2006
    Location
    Servers
    Posts
    1,588
    Yes it is the service killer but this VPS IDs are not abusing cause this is happening randomly even for guys which does not have any usage more than 150MB. I wonder if I can add this safe MySQL as trusted so never to be killed at least. If the actual MySQL process is killed it is fine cause will be automatically restarted.

  11. #11
    Quote Originally Posted by WebHostDog View Post
    Yes it is the service killer but this VPS IDs are not abusing cause this is happening randomly even for guys which does not have any usage more than 150MB. I wonder if I can add this safe MySQL as trusted so never to be killed at least. If the actual MySQL process is killed it is fine cause will be automatically restarted.
    Service killer will just kill the process that it considered "harmful", for example, the process that requested more memory in an oom (out of memory) condition.

    I haven't actually used virtuozzo, just openvz, but I think that, if you look at /proc/user_beamcounters for the VPS, you could be able to tell which parameter is the problematic.

    I hope this helps,

    Ildefonso Camargo

  12. #12
    Join Date
    Mar 2006
    Location
    Servers
    Posts
    1,588
    Yes I will check but it is pretty obvious this is happening totally random. And never happened with VZ 4. They are both running SLM model though.
    Last edited by WebHostDog; 11-02-2009 at 05:21 PM.

  13. #13
    Hey, if you're still having the problem, here's some info from the Parallels Tech Support team:

    Check how many times the kill signal was sent by OOM killer for this container with cat /proc/slm/groups/Container_ID/stats (kill_sent parameter).

    Also, there will be a reason_inst_cap, meaning that the memory size requested by processes inside CT has been limited by SLMMEMORYLIMIT. For SLM enabled containers the following formula could be used for overall [ideal] memory allocation: RAM + SWAP = 1 Gb + Sum (SLMMEMORYLIMIT values of all CTs).

    If there isn't enough memory, the average memory allocation for CTs is calculated in this way: Effective (SLMMEMORYLIMIT) = SLMMEMORYLIMIT * (RAM + SWAP - 1Gb) / Sum (SLMMEMORYLIMIT values of all CTs). The sum of all SLMMEMORYLIMIT values can be calculated with grep SLMMEMORYLIMIT /etc/sysconfig/vz-scripts/*.conf

  14. #14
    Join Date
    Mar 2006
    Location
    Servers
    Posts
    1,588
    Hello,
    It is not the memory cause I am running a script which is calculating every 5 min the free -m output of all VPSes. The max for one node having this problem is 7765M and the box have 16GB of ram. The swap is always zero as well. Even the container is hitting the limit how I may prevent MySQl to be killed but other processes to be killed like Apache ?



    Thanks,

  15. #15
    I believe its the OOM manager which is killing the process, Give the following command and change the ve number to yours , then see the output how many lines its counts.

    cat /var/log/messages | grep OOM | grep ve=123 | wc -l
    www.24x7servermanagement.com
    Server Management, Server Security, Server Monitoring.
    India's Leading Managed Service Provider !! Skype: techs24x7

  16. #16
    Quote Originally Posted by WebHostDog View Post
    Hello,
    It is not the memory cause I am running a script which is calculating every 5 min the free -m output of all VPSes. The max for one node having this problem is 7765M and the box have 16GB of ram. The swap is always zero as well. Even the container is hitting the limit how I may prevent MySQl to be killed but other processes to be killed like Apache ?



    Thanks,
    Hi!

    First: can you, please, post the output of: cat /proc/user_beamcounters for the node that is having the problem? then, after the issue happen again, post it again (unless it shows no changes at all, in which case, just say: happened again: no changes to UBC).

    This can take us a little off-topic, but: 32 or 64 bits OS? This is more related to MySQL optimization.

    I hope we can get the problem fixed,

    Ildefonso Camargo

  17. #17
    Quote Originally Posted by Jacob Wall View Post
    Have you brought it up to Parallel's? I know you probably don't like their technical support, nor do many people that use Parallel's products but I think it will be beneficial in this kind of situation.
    Its funny you say that because thats the impression most people have their support.

  18. #18
    Join Date
    Mar 2006
    Location
    Servers
    Posts
    1,588
    /proc/user_beancounters is for old RAM model. I am using the new SLM model. The idea is may be how to make VZ not killing MySQL but first Apache and other processes. They had such a file with priorities but I am not sure how this is working. Killing MySQL is not a problem if it was not killing the mysqld_safe process which is causing cPanel to be uneable to reboot the MySQL after that.
    So if I enter even only mysqld_safe in save list for VZ it will be fine I think.

  19. #19
    Hey WebHostDog,

    Have you solved the problem? Or do you need some more help?
    Last edited by Drew_Parallels; 11-08-2009 at 10:59 PM.
    Drew from Parallels
    Twitter: @ParallelsPanel

  20. #20
    Alright, this is what my tech contact told me. Sorry about the wall of text:

    It is really a memory related issue and it can be checked with grep exc /var/log/messages

    There should be something like: Sep 25 18:58:18 server1 kernel: kill_signal(123.0): selecting to kill, queued 0, seq 12, exc 4 0 goal 60 0... Sep 25 18:58:18 server1 kernel: kill_signal(123.0): selected 1, signalled 1, queued 1, seq 12, exc 60 0 red 32964 289 Sep 25 19:00:20 server1 kernel: kill_signal(123.0): selecting to kill, queued 0, seq 13, exc 40 0 goal 40 0... Sep 25 19:00:20 server1 kernel: kill_signal(123.0): selected 1, signalled 1, queued 1, seq 13, exc 40 0 red 32996 289 Sep 25 19:00:21 server1 kernel: kill_signal(123.0): selecting to kill, queued 0, seq 14, exc 19 0 goal 1429 0... The digits after 'exc' means how much total and low memory is not enough and in this case we see that total memory is the resource which server lacks of. Also there are should be records like: Sep 25 07:02:25 server1 kernel: OOM killed process httpd (pid=7051, ve=100) exited, free=450811 gen=190. Sep 25 11:51:18 server1 kernel: OOM killed process httpd (pid=4781, ve=100) exited, free=304447 gen=191. Sep 25 11:58:59 server1 kernel: OOM killed process httpd (pid=29207, ve=100) exited, free=296846 gen=192. And if it is really exists in /var/log/messages my previous recommendations will help.
    Drew from Parallels
    Twitter: @ParallelsPanel

  21. #21
    Join Date
    Mar 2006
    Location
    Servers
    Posts
    1,588
    Hello,
    Yes if really a memory issue I would like it not to kill the mysqld_safe process which is causing MySQL to be unable to start again before killall -9 mysqld is issued. This is happening non stop on VZ3 and very rarely on VZ4 version. How may I add mysqld_safe into the default template to be killed after Apache and email services ?

  22. #22
    Join Date
    Mar 2006
    Location
    Servers
    Posts
    1,588
    One VPS just got killed though this are the last 60 rows from dmesg:

    kill_signal(430.3): task ab0ef940, thg bf0e5100, sig 2
    kill_signal(430.3): task aa2a2ce0, thg bf0e5100, sig 2
    kill_signal(430.3): task c7e65310, thg bf0e5100, sig 2
    kill_signal(430.3): task c63c86f0, thg bf0e5100, sig 2
    kill_signal(430.3): task 56383900, thg bf0e5100, sig 2
    kill_signal(430.3): task 9f7cc0c0, thg bf0e5100, sig 2
    kill_signal(430.3): task cfa120c0, thg bf0e5100, sig 2
    kill_signal(430.3): task bd79b2d0, thg bf0e5100, sig 2
    kill_signal(430.3): task bb539980, thg bf0e5100, sig 2
    kill_signal(430.3): task 4b0f7980, thg bf0e5100, sig 2
    kill_signal(430.3): task b6fc8d20, thg bf0e5100, sig 2
    kill_signal(430.3): task ae88f2d0, thg bf0e5100, sig 2
    kill_signal(430.2): selecting to kill, queued 0, seq 631, exc 170 0 goal 170 0...
    kill_signal(430.2): task bcfd06f0, thg 34f9bc00, sig 1
    kill_signal(430.2): selected 1, signalled 1, queued 1, seq 631, exc 170 0 red 8532 778
    kill_signal(430.2): task bcfd06f0, thg 34f9bc00, sig 2
    nofork(430.2): delayed task c3d51940 for 592 msec
    nofork(430.2): delayed task 4d1e6080 for 1475 msec
    kill_signal(430.2): selecting to kill, queued 0, seq 632, exc 4055 0 goal 157589 0...
    kill_signal(430.2): selected 19, signalled 19, queued 19, seq 632, exc 157589 0 red 159559 14683
    OOM killed process php (pid=2848, ve=430) (mm=b03fec80) exited, free=19502 gen=14512.
    OOM killed process php (pid=2581, ve=430) (mm=b275c400) exited, free=21966 gen=14513.
    OOM killed process php (pid=2700, ve=430) (mm=c7f1d1c0) exited, free=21966 gen=14514.
    OOM killed process php (pid=2956, ve=430) (mm=aee30080) exited, free=23870 gen=14515.
    OOM killed process php (pid=2863, ve=430) (mm=9e74a100) exited, free=34110 gen=14516.
    OOM killed process php (pid=2691, ve=430) (mm=852f54c0) exited, free=35911 gen=14517.
    OOM killed process php (pid=2867, ve=430) (mm=cff46c80) exited, free=36174 gen=14518.
    OOM killed process php (pid=2828, ve=430) (mm=0de16f40) exited, free=38958 gen=14519.
    OOM killed process php (pid=2814, ve=430) (mm=67ca0f00) exited, free=41646 gen=14520.
    OOM killed process php (pid=2886, ve=430) (mm=cf818100) exited, free=45358 gen=14521.
    OOM killed process php (pid=2966, ve=430) (mm=bf7dcc00) exited, free=45550 gen=14522.
    OOM killed process php (pid=2847, ve=430) (mm=bf52af00) exited, free=47358 gen=14523.
    OOM killed process php (pid=2470, ve=430) (mm=b3952c00) exited, free=48686 gen=14524.
    OOM killed process php (pid=2811, ve=430) (mm=aee9c0c0) exited, free=49246 gen=14525.
    OOM killed process php (pid=2729, ve=430) (mm=ae95f700) exited, free=50350 gen=14526.
    OOM killed process php (pid=2871, ve=430) (mm=a3df2940) exited, free=53614 gen=14527.
    OOM killed process php (pid=2813, ve=430) (mm=49a6e9c0) exited, free=57470 gen=14528.
    OOM killed process php (pid=2960, ve=430) (mm=c26fcc80) exited, free=57662 gen=14529.
    OOM killed process php (pid=2524, ve=430) (mm=cfaa5a40) exited, free=57662 gen=14530.
    kill_signal(430.2): selecting to kill, queued 0, seq 633, exc 332 0 goal 8787 0...
    kill_signal(430.2): task ba830ce0, thg c26fcc80, sig 1
    kill_signal(430.2): task 7be55310, thg 0de16f40, sig 1
    kill_signal(430.2): selected 2, signalled 2, queued 2, seq 633, exc 8787 0 red 17190 1562
    kill_signal(430.2): selecting to kill, queued 0, seq 634, exc 1486 0 goal 121282 0...
    kill_signal(430.2): selected 15, signalled 15, queued 15, seq 634, exc 121282 0 red 127572 11661
    OOM killed process php (pid=4130, ve=430) (mm=c9274ec0) exited, free=21742 gen=14531.
    OOM killed process php (pid=3066, ve=430) (mm=cdaf96c0) exited, free=26638 gen=14532.
    OOM killed process php (pid=3062, ve=430) (mm=aee30080) exited, free=26638 gen=14533.
    OOM killed process php (pid=3065, ve=430) (mm=b775d980) exited, free=27657 gen=14534.
    OOM killed process php (pid=3064, ve=430) (mm=bf7dcc00) exited, free=28638 gen=14535.
    OOM killed process php (pid=3057, ve=430) (mm=cf818100) exited, free=33358 gen=14536.
    OOM killed process php (pid=3059, ve=430) (mm=aa30c180) exited, free=36686 gen=14537.
    OOM killed process php (pid=3053, ve=430) (mm=bdb629c0) exited, free=36782 gen=14538.
    OOM killed process php (pid=3039, ve=430) (mm=adbb3480) exited, free=37342 gen=14539.
    OOM killed process php (pid=4100, ve=430) (mm=9e74a100) exited, free=40062 gen=14540.
    OOM killed process php (pid=3063, ve=430) (mm=b275cc40) exited, free=40510 gen=14541.
    OOM killed process php (pid=2987, ve=430) (mm=33cf2980) exited, free=41086 gen=14542.
    OOM killed process php (pid=3056, ve=430) (mm=8d6791c0) exited, free=42766 gen=14543.
    OOM killed process php (pid=3061, ve=430) (mm=c49380c0) exited, free=43326 gen=14544.
    OOM killed process php (pid=4114, ve=430) (mm=b20de440) exited, free=45262 gen=14545.


    I do not even see mysql* in this kills ...

  23. #23
    If the solutions I posted earlier didn't solve your problem, then I'm not sure how to solve it. I know that Virtuozzo 4.0 and higher have much better memory management, so upgrading would be the most straightforward solution.

    Did you check your system against the formula for the sum of SLMMEMORYLIMIT which I posted earlier?
    Drew from Parallels
    Twitter: @ParallelsPanel

  24. #24
    Join Date
    Mar 2006
    Location
    Servers
    Posts
    1,588
    Hello Drew,
    There is no SWAP at all there is a lot of RAM. Seems we need to add mysqld_safe in this file somehow so it not get killed:


    /etc/vzslm.d/default.conf

    "init" 00000018 -1 9
    "httpd" 0000001c -1 2
    "httpsd" 0000001c -1 2
    "lighthttpd" 0000001c -1 2
    "mysqld" 0000001c -1 3
    "syslogd" 00000018 0 8
    "sshd" 00000018 0 8
    "inetd" 00000018 0 8
    "xinetd" 00000018 0 8
    "cron" 00000018 0 8
    "crond" 00000018 0 8
    "" 00000004 9 0
    "" 00000004 8 1
    "" 00000004 1 0

Similar Threads

  1. VPS RAM is Killing
    By Medhat in forum Hosting Security and Technology
    Replies: 3
    Last Post: 01-17-2008, 02:01 PM
  2. Do all virtuozzo VPS's come with virtuozzo power panel?
    By smsmasters in forum Web Hosting
    Replies: 1
    Last Post: 05-15-2007, 11:38 AM
  3. My car is killing me
    By Kamejoko in forum Web Hosting Lounge
    Replies: 18
    Last Post: 03-07-2004, 06:27 PM
  4. The Internet is killing me...
    By vito in forum Web Hosting Lounge
    Replies: 23
    Last Post: 07-12-2003, 11:21 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •