Results 1 to 17 of 17
  1. #1
    Join Date
    Aug 2002
    Posts
    165

    Softlayer: Intermittent "slow down" every few minutes

    Hi there,

    Firstly, I want to say that I signed up with Softlayer because I heard they had better support than EV1. If that's the case, EV1 must be really horrible.

    That being said, I've had an issue with my server every since I got it from them a few days ago. I moved a bunch of high traffic sites that use PHP and MySQL on a dual core dual processor server with 2 gigs of ram and a scsi 10k disk.

    It is running Plesk 8 (which I theorize 'could' be the problem).

    I am not stating it is a hardware problem or a software problem because I have no idea what the problem is. Here are the symptoms:

    - Every few minutes all my PHP and MySQL sites begin to run very slow. It takes about 10 seconds before a page is loaded.

    - During this time, sites that do not use MySQL and PHP seem to run fast, but perhaps a little slower than normal.

    - When i attempt to log into SSH during these moments, it pauses on the screen with the black cursor and waits and takes about 30 seconds to log in.

    - When I log in, or when I am already logged in and I view the load and memory usage during these times, there is no correlation between the cpu usage or memory usage and the slow downs

    - They occur all day and all night.

    - When these moments do not occur, the sites are all blazingly fast.

    - Load is rarely ever above 1

    - I have MySQL and PHP running on the same box.

    - I have disabled any possible Plesk cron jobs including drweb and the backup cron that runs every 15 mintues. However, I have manually run it and tested the server and it does not cause any slow down.

    - One cron job I have on one of the sites I host (http://www.perfectpeople.net) updates a new image every 25 minutes. This certainly does not cause the problem, and recently two images have shown up not found but the db is updated so you see that image not found error. This has never occurred before and I theorize this is the result of the same issue.

    ---------

    None of these issues occurred with any previous servers. Softlayer techs keep telling me everything is fine, repeating over and over that it loads fine on their side, being completely unhelpful.

    Some notes to avoid pointless responses:

    1. If other sites load fine, and ping is fine, it is not a bandwidth or traceroute issue.

    2. If CPU and memory usage is fine, it is not a load issue.

    ---------

    The only thing I can theorize based on my limited knowledge is that it could be a hard drive seek. I do not know if that will cause SSH connections to slow down and PHP and MySQL.

    Any help would be greatly appreciated.

  2. #2
    Join Date
    Aug 2002
    Posts
    165
    I logged in via SFTP and the pause is when it tries to validate the password. It seems a DB lookup takes a long time when this is happening. I think that relates to a hard drive seek but I can't be sure.

  3. #3
    Join Date
    Aug 2002
    Posts
    165
    PP Slowdown 04:45:49 27/10/06 - 7.0393290519714 secs 2:45 pm
    PP Slowdown 04:45:12 27/10/06 - 10.726112127304 secs 2:45 pm
    PP Slowdown 04:39:11 27/10/06 - 10.744872093201 secs 2:39 pm
    PP Slowdown 04:34:13 27/10/06 - 11.077215909958 secs 2:34 pm
    PP Slowdown 04:33:42 27/10/06 - 10.857461929321 secs 2:34 pm
    PP Slowdown 04:27:52 27/10/06 - 10.720928192139 secs 2:27 pm
    PP Slowdown 04:27:11 27/10/06 - 10.715999126434 secs 2:27 pm
    PP Slowdown 04:20:11 27/10/06 - 10.733088970184 secs 2:20 pm
    PP Slowdown 04:19:42 27/10/06 - 10.724981069565 secs 2:20 pm
    PP Slowdown 04:13:12 27/10/06 - 10.723094940186 secs 2:13 pm

    These are the results of a script I am running that downloads the root webpage from perfectpeople.net and emails me when the request takes longer than 6 seconds.

  4. #4
    Join Date
    Mar 2006
    Location
    New York USA
    Posts
    402
    I was on SL too, and i believe that their firewalls cause the issue, or atleast some sorts of throttling methods. I experinced almost the same issues you are facing, but usually i'd get banned by IP from my own sites, or e-mail would mess up, or lag would occur. At times subdomains would error out and say they didnt exsist when trying to view their respective pages. None-the-less i've moved off SL.

    Good Luck.

    -ShaunP

  5. #5
    Join Date
    Aug 2002
    Posts
    165
    Man... If this is an SL issue I'm going to kick myself. Everywhere I checked people seemed to love them!

  6. #6
    Join Date
    Aug 2002
    Posts
    165
    I've calculated that it's happening once every five minutes on average, and lasts about a minute. This means that for about 5 hours a day my server is slower than anything.

    I'm still awaiting their response. I can't seem to figure anything else out on my end.

  7. #7
    Join Date
    Sep 2003
    Location
    Montreal, Canada
    Posts
    373
    here's my best attempt to help:
    Did you perhaps install mrtg? with monitoring of every possible thing. cpu/ram/hdd/network/etc

    You *might* find something out of those graphs?

  8. #8
    Join Date
    Jul 2006
    Posts
    285
    I've noticed this happen since the update to RHEL 4.4. A kernel compiled from source does not fix it, so it's something else. I too suspected a glitchy HD but it clears up after a reboot for about a week, which pretty much rules out the network. SMART shows no problems. Pings are rock solid stable at the same time this happens and server load is very low. It only seems to happen on sites with php/mysql. Static sites run fine as far as I can tell. Try a reboot and see if it clears up for a few days. I'm curious about this myself.
    Last edited by Iwannasite; 10-28-2006 at 06:44 PM.

  9. #9
    Is there anything unique appearing in php/apache error logs?

  10. #10
    Join Date
    Aug 2002
    Posts
    165
    I have not checked the logs but will do that... although I'm not sure what to look for. I originally thought it might be Qmail or something because there was a "relaylock" error about ten seconds before every instance of the problem... but I disabled qmail entirely and it still continued.

    I have not installed MRTG and since I have plesk on and the SL people are apparently figuring out the issue I'd rather not try since I am not as familiar with installations on Linux as I[d like to be.

    I did a reboot about a day after the problem was ocurring and no change.

  11. #11
    Join Date
    Aug 2002
    Posts
    165
    I asked them to disable server monitoring and the issue went away.

    So it was certainly a problem on their end.

  12. #12
    Join Date
    Mar 2005
    Location
    Big Blue Marble
    Posts
    10
    Quote Originally Posted by Decius
    I asked them to disable server monitoring and the issue went away.

    So it was certainly a problem on their end.

    Hi,

    I am what appears to be what you had. I have been trying to figure it out for a couple of months now with no luck. All my sites were fine until I moved them to SL and the concern started right after the move.

    May I ask what server monitoring the disabled? How are you monitoring your server in case it goes off-line?

    Kind Regards,

    Les

  13. #13
    Join Date
    Jul 2006
    Posts
    285
    The solution that many people figured out including myself can be found in these threads.

    https://forums.softlayer.com/showthread.php?t=783
    https://forums.softlayer.com/showthread.php?t=632


    It is an issue with their internal resolvers. Set your machine to use it's own or use another dns service and the problem goes away. At least it did for several others. I don' t know how ping monitoring would have anything to do with it. Probably just a coincidence.

  14. #14
    Join Date
    Aug 2002
    Posts
    165
    Iwannasite is correct. The problem went away when the ping monitoring was disabled but it was the internal resolution that was the problem. They changed that at almost the exact same time as disabling the ping monitoring so I thought this was the issue but it wasn't.

  15. #15
    Join Date
    Mar 2005
    Location
    Big Blue Marble
    Posts
    10
    Quote Originally Posted by Iwannasite
    The solution that many people figured out including myself can be found in these threads.

    https://forums.softlayer.com/showthread.php?t=783
    https://forums.softlayer.com/showthread.php?t=632


    It is an issue with their internal resolvers. Set your machine to use it's own or use another dns service and the problem goes away. At least it did for several others. I don' t know how ping monitoring would have anything to do with it. Probably just a coincidence.
    Hi Iwannasite,

    I have my SL server via Server4Sale. I do have Limited SL portal access that includes the forum. I will go over there and take a read. This has been going on since I got the server and I was starting to get sad.

    Kind Regards,

    Les

  16. #16
    Join Date
    Feb 2006
    Location
    Swellyville
    Posts
    2,340
    Had some connectivity issues in the past with SoftLayer...nothing major though. In addition, when I did have issues, technical suppor was there to let me know what the problem was. However, I don't think it ever had anything to do with a firewalls, not sure.
    <<< Please see Forum Guidelines for signature setup. >>>

  17. #17
    Join Date
    May 2006
    Posts
    1,398
    yeah they are not perfect and most of the support doesnt have a clue. I recently had some ips that had routing problems and they were clueless. But the good thing is you dont need support much. I have noticed though that my ssh pings out all the time. It wont stay connected over 10 minutes while Im connected to my other servers for days.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •