Page 1 of 2 12 LastLast
Results 1 to 40 of 51
  1. #1
    Join Date
    Oct 2004
    Posts
    627

    wield I/O on HP server, SAS drives

    We have a brand new HP server, installed with a 2x SAS 15K drives on a RAID1 with HP 's embedded RID controller (256mb cache)

    We then did a test

    dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync
    16384+0 records in
    16384+0 records out
    1073741824 bytes (1.1 GB) copied, 28.094 seconds, 38.2 MB/s

    seems VERY slow considering server is zero load

    any possible places we should look at?

    hdparm looks better though

    ./hdparm -tT /dev/cciss/c0d0p1
    /dev/cciss/c0d0p1:
    Timing cached reads: 12744 MB in 2.00 seconds = 6386.43 MB/sec
    Timing buffered disk reads: 582 MB in 3.00 seconds = 193.75 MB/sec

  2. #2
    Join Date
    Jan 2011
    Location
    Varna, Bulgaria
    Posts
    1,270
    If it is brand new server, maybe the RAID is currently resyncing. Can you monitor the RAID status?

  3. #3
    Join Date
    Jan 2010
    Location
    so cal
    Posts
    232
    could be the drives, maybe they're not fully supported. What type/model are they? have you checked that you don't a dead hdd? could also be a faulty drive.

  4. #4
    Join Date
    Oct 2004
    Posts
    627
    how to test HDs please?

    We have linux CentOS+Cpanel installed for now

  5. #5
    Join Date
    Jan 2010
    Location
    so cal
    Posts
    232
    run this command:
    cat /etc/fstab

    Paste the results here. I mainly want to see what type of file system you're running.

    Also, paste the results from this command:
    df

  6. #6
    Join Date
    Oct 2004
    Posts
    627
    LABEL=/ / ext3 defaults 1 1
    LABEL=/tmp /tmp ext3 defaults 1 2
    tmpfs /dev/shm tmpfs defaults 0 0
    devpts /dev/pts devpts gid=5,mode=620 0 0
    sysfs /sys sysfs defaults 0 0
    proc /proc proc defaults 0 0
    LABEL=SW-cciss/c0d0p3 swap swap defaults 0 0
    /tmp /var/tmp ext3 defaults,bind,noauto 0 0


    Filesystem 1K-blocks Used Available Use% Mounted on
    /dev/cciss/c0d0p1 563665980 6403740 528167724 2% /
    /dev/cciss/c0d0p2 2030768 38756 1887188 3% /tmp
    tmpfs 2017532 0 2017532 0% /dev/shm

  7. #7
    Join Date
    Jan 2010
    Location
    so cal
    Posts
    232
    run this to fix and detect errors (may take a while):
    fsck -y -t ext3 /dev/cciss/c0d0p1

  8. #8
    Join Date
    Oct 2004
    Posts
    627
    /: recovering journal
    Clearing orphaned inode 142540801 (uid=0, gid=0, mode=0100644, size=0)
    /: clean, 158126/145489920 files, 6157024/145472583 blocks


    same slow

    16384+0 records in
    16384+0 records out-6.3.0-22.x86_64.rpm/pub/softlib2/software1/pubsw-linux/p15074
    1073741824 bytes (1.1 GB) copied, 38.5826 seconds, 27.8 MB/s


    em....

  9. #9
    Join Date
    Jan 2010
    Location
    so cal
    Posts
    232
    have you tried testing it with unixbench?

  10. #10
    Join Date
    Mar 2003
    Location
    California USA
    Posts
    13,290
    The DD test is a write test fwiw. I have not used that controller in some time, but I recall writes being really slow.
    Steven Ciaburri | Industry's Best Server Management - Rack911.com
    Software Auditing - 400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com
    Fully Managed Dedicated Servers (Las Vegas, New York City, & Amsterdam) (AS62710)
    FreeBSD & Linux Server Management, Security Auditing, Server Optimization, PCI Compliance

  11. #11
    Join Date
    Oct 2004
    Posts
    627
    compare to everyone here

    http://www.webhostingtalk.com/showth...1032917&page=4

    gosh, damn slow

    It is seagate SAS 450G 15K RPM brand new drives...

  12. #12
    Join Date
    Oct 2004
    Posts
    627
    Quote Originally Posted by rds100 View Post
    If it is brand new server, maybe the RAID is currently resyncing. Can you monitor the RAID status?
    How to check RAID status on HP server?

  13. #13
    Join Date
    Jan 2010
    Location
    so cal
    Posts
    232
    Come to think of it, those numbers are correct. You just have a slow raid controller

    Edit: unixbench is a more accurate test of your hdd.

  14. #14
    Join Date
    Oct 2004
    Posts
    627
    to enable write cache without BBU seems dangerous too...so we disable it

    em...some what I think HP 's RAID controller should not be like this...

  15. #15
    Join Date
    Oct 2004
    Posts
    627
    There are two kinds of "write ache"

    one is on raid card and one is on physical disk

    better to turn off both? or we could enable physical drive cache?

  16. #16
    Join Date
    Mar 2009
    Posts
    534
    Are you using authentic HP branded disks, or did you buy Seagates and install them into trays yourself?

    EDIT: Oh yeah, and please disclose the model of the server itself as well so I can check QuickSpecs for the onboard RAID controller model.

    --Chris
    Last edited by ObjectZone; 05-04-2011 at 09:43 AM.
    The Object Zone - Your Windows Server Specialists for more than twelve years - http://www.object-zone.net/
    Services: Contract Server Management, Desktop Support Services, IT/VoIP Consulting, Cloud Migration, and Custom ASP.net and Mobile Application Development

  17. #17
    Join Date
    Nov 2005
    Posts
    1,224
    Does the controller have a battery backed write cache? And if so, was the server recently powered up?

    Our HPs with BBWC disable all caching until the battery is charged, which usually takes a couple hours. Performance is degraded during this time, for obvious reasons.

  18. #18
    Join Date
    Jan 2010
    Location
    so cal
    Posts
    232
    The controller doesn't have a bbu at all. This is just a slow controller.

  19. #19
    Join Date
    Nov 2005
    Posts
    1,224
    Ouch. I learned that the hard way after deploying a handful of brand new ProLiants that came with the SC44Ge SAS controllers.

    Not only did the SC44Ge lack cache, but its driver also forcably DISABLED any caching done by the operating system itself.

    A pair of 7200 rpm SATA disks using nothing other than the "Enhanced performance" (maximum caching) option in Windows Server, worked significantly better than a pair of 15,000 rpm SAS disks connected to the "no cache available of any kind" SC44Ge setup.

    These brand new servers promptly fell flat on their faces under load, leaving us calling our vendor for an emergency shipment of P400/512(MB) BBWC controllers (which quite frankly are outstanding little units).

  20. #20
    Join Date
    Jan 2010
    Location
    so cal
    Posts
    232
    Is it possible to use an array on a new controller? or did you had to recreate the array?

  21. #21
    Join Date
    Nov 2005
    Posts
    1,224
    Quote Originally Posted by TinyVox View Post
    Is it possible to use an array on a new controller? or did you had to recreate the array?
    Even though the SC44Ge and P400 are HP RAID controllers, the array containers did not transfer. I had to start fresh. (per HP support)

    Fortunately the new servers were running a bunch of Hyper-V guests, so I just shifted them off to other servers while I rebuilt the containers and reinstalled the OSes. So it wasn't too bad.

  22. #22
    Join Date
    Nov 2009
    Location
    Cincinnati
    Posts
    1,583
    Run bonnie++..

    wget http://packages.sw.be/bonnie++/bonni....rf.x86_64.rpm
    rpm -ivh bonnie*
    mkdir scratch
    bonnie++ -d scratch -u root
    That will get you good seq read/write and random iop performance.

    I'd expect around 200/200 500sec/s.
    'Ripcord'ing is the only way!

  23. #23
    Join Date
    Mar 2009
    Posts
    534
    Quote Originally Posted by Sekweta View Post
    Ouch. I learned that the hard way after deploying a handful of brand new ProLiants that came with the SC44Ge SAS controllers.

    Not only did the SC44Ge lack cache, but its driver also forcably DISABLED any caching done by the operating system itself.
    Yes, this is indeed correct. A few select models of HP RAID controllers are pretty pathetic performance-wise and that was why I was asking OP for their model of system. I once unfortunately had to find out how bad the E200 card was. Even with a cache, it's performance was abysmal. Literally, it was only good enough for mirroring a set of boot drives that would ultimately run apps off of a SAN or some other extremely menial task. I replaced it with a P400/256 w/BBWC and it shot into full gear.

    These brand new servers promptly fell flat on their faces under load, leaving us calling our vendor for an emergency shipment of P400/512(MB) BBWC controllers (which quite frankly are outstanding little units).
    Haha. My solution as well. The P400 RAID card supports turning on the disk's internal caches in addition to the controller's cache. My server runs redundant power on A+B feeds, so I flip this on to gain the performance. If the RAID ever degrades or I initiate array expansion/extension, the cache will switch to read only and performance is degraded slightly until it's done.

    --Chris
    The Object Zone - Your Windows Server Specialists for more than twelve years - http://www.object-zone.net/
    Services: Contract Server Management, Desktop Support Services, IT/VoIP Consulting, Cloud Migration, and Custom ASP.net and Mobile Application Development

  24. #24
    Join Date
    Nov 2005
    Posts
    1,224
    Quote Originally Posted by ObjectZone View Post
    The P400 RAID card supports turning on the disk's internal caches in addition to the controller's cache. My server runs redundant power on A+B feeds, so I flip this on to gain the performance.
    You have more courage than I do, even with A/B power.

    For our needs, the P400/512 performs so well when mated with 15k Cheetah SAS disks, that we don't need to enable cache on the disk itself. And with BBWC, I don't lose sleep at night fearing a power failure will corrupt open files.

    Most of these servers are running enterprise-critical apps for our clients, so we have to err on the conservative side at the expense of bleeding-edge performance.

    Back in the 90s when we just did web hosting, it was fun building our own way-cool servers, overclocking CPUs and memory (within reason) to squeeze out faster cgi response, and other exciting tweaks that only a hardcore geek could love.

    Hosting enterprise applications means doing everything with factory assembled name-brand (HP) servers, everything run strictly to spec, tame, predictable, certified, redundant, all to get that uber-stable, uber-reliable environment that corporate users appreciate.

    It's so dull and boring, in its well heeled, well mannered, "suit and tie with wing-tipped shoes" kind of way. But it's also nice having gone more than 6 years now (knocks wood) without a single pager alert interrupting dinner or waking me up in the middle of the night.

    So in that regard, I literally do sleep better at night.

  25. #25
    Join Date
    Oct 2004
    Posts
    627
    onboard controller is smart array P212 with 256mb cache and no battery BBWC

    so, this is a dead end?!

  26. #26
    Join Date
    Mar 2009
    Posts
    534
    Quote Originally Posted by Sekweta View Post
    You have more courage than I do, even with A/B power.

    For our needs, the P400/512 performs so well when mated with 15k Cheetah SAS disks, that we don't need to enable cache on the disk itself. And with BBWC, I don't lose sleep at night fearing a power failure will corrupt open files.
    Actually, I got to thinking about this after I posted it - and think I mis-stated. When I saw your reply come to my inbox, I went to check the production server and confirmed that even now the production enterprise apps do not have internal disk caches enabled. The BBWC is the only place written data will hang out. My performance is top notch even in this configuration with 4x10k SAS drives. All HP branded/supported.

    I have a DL185 G5 which has the drive write caches enabled. It's only a development server with very little activity - and in order to save some scratch I built my own drives for this one. When HP controllers see unsupported drives, it seems to cut performance a lot. Enabling the drive write caches seems to resolve that issue, so this is where I actually tapped that risky feature.

    It's so dull and boring, in its well heeled, well mannered, "suit and tie with wing-tipped shoes" kind of way. But it's also nice having gone more than 6 years now (knocks wood) without a single pager alert interrupting dinner or waking me up in the middle of the night.

    So in that regard, I literally do sleep better at night.
    You and I are quite alike. Personally, I use HP because of my prior MANY years of experience going way back as far as Compaq Proliant 800's, 1600s, 1850R, you name it, I've probably used one. I find their management tools and iLO cards to be consistent and for the most part compatible across the entire lineup. The 1xx series seems to be the exception. I've noted a few items about them that makes me less enthusiastic about them.

    Anyway, getting off topic here.

    --Chris
    The Object Zone - Your Windows Server Specialists for more than twelve years - http://www.object-zone.net/
    Services: Contract Server Management, Desktop Support Services, IT/VoIP Consulting, Cloud Migration, and Custom ASP.net and Mobile Application Development

  27. #27
    Join Date
    Mar 2009
    Posts
    534
    Quote Originally Posted by PeterPP View Post
    onboard controller is smart array P212 with 256mb cache and no battery BBWC

    so, this is a dead end?!
    A "P" series card should be in the Performance series and should have great sequential read/write performance without any cache at all. When you start getting into a lot of random IOs you'll wish you had at least a little cache, but at least at this point it doesn't seem like this is your issue.

    Did you mention whether your hard drives were HP branded or assembled yourself/by the datacenter?

    --Chris
    The Object Zone - Your Windows Server Specialists for more than twelve years - http://www.object-zone.net/
    Services: Contract Server Management, Desktop Support Services, IT/VoIP Consulting, Cloud Migration, and Custom ASP.net and Mobile Application Development

  28. #28
    Join Date
    Feb 2006
    Location
    Boston, MA
    Posts
    58
    Quote Originally Posted by PeterPP View Post
    onboard controller is smart array P212 with 256mb cache and no battery BBWC

    so, this is a dead end?!
    No, it doesn't need to be a dead end. As many others have mentioned, if your controller and cache don't have a battery, the controller will disable the drive caches by default. This will ensure data integrity in the event of a power loss but at the expense of write performance.

    You can try buying a battery for the P212. Take a look at http://h18000.www1.hp.com/products/q.../13203_na.html. The part number seems to be 462969-B21.
    Akliz, Inc.
    www.akliz.net | 617-475-3266

  29. #29
    Join Date
    Oct 2004
    Posts
    627
    bonnie test, looks bad?

    bonnie++ -d scratch -u root
    Using uid:0, gid:0.
    Writing a byte at a time...done
    Writing intelligently...done
    Rewriting...done
    Reading a byte at a time...done
    Reading intelligently...done
    start 'em...done...done...done...done...done...
    Create files in sequential order...done.
    Stat files in sequential order...done.
    Delete files in sequential order...done.
    Create files in random order...done.
    Stat files in random order...done.
    Delete files in random order...done.
    Version 1.94 ------Sequential Output------ --Sequential Input- --Random-
    Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
    Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
    bear.potia.net 8G 754 97 36351 7 31613 6 2095 93 190503 14 575.7 9
    Latency 98074us 1887ms 1861ms 10231us 98490us 1087ms
    Version 1.94 ------Sequential Create------ --------Random Create--------
    bear.potia.net -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
    files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
    16 10432 12 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
    Latency 199us 1239us 586us 158us 8us 21us
    1.93c,1.94,bear.potia.net,1,1304577037,8G,,754,97,36351,7,31613,6,2095,93,190503,14,575.7,9,16,,,,,10432,12,+++++,+++,+++++,+++,+++++,+++,+++++,+++,++ +++,+++,98074us,1887ms,1861ms,10231us,98490us,1087ms,199us,1239us,586us,158us,8us,21us


    we purchased seagate drives from newegg and install it on HP server

  30. #30
    Join Date
    Oct 2004
    Posts
    627
    enable cache to see whether it is write cahe issue or not

    dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync
    16384+0 records in
    16384+0 records out
    1073741824 bytes (1.1 GB) copied, 7.29602 seconds, 147 MB/s

    compared to 3XMB/s with cache disabled, wow....

  31. #31
    Join Date
    Jan 2010
    Location
    so cal
    Posts
    232
    I wouldn't enable it though...

  32. #32
    Join Date
    Oct 2004
    Posts
    627
    ok, now

    Disk write cache=enabled
    RAID write cache=disable

    Write speed is still 147 MB/s on dd test

    is it MUCH better (compare to both disabled) to enable disk cache?

  33. #33
    Join Date
    Jan 2010
    Location
    so cal
    Posts
    232
    Yes, of course. Keep it like this

  34. #34
    Join Date
    Oct 2004
    Posts
    627
    ususally for majority?

    is "disk cache" always enabled on RAID without battery?

    (been confused about RAID card cache and disk cache)

  35. #35
    Join Date
    Jan 2010
    Location
    so cal
    Posts
    232
    Nope.......

  36. #36
    Join Date
    Oct 2004
    Posts
    627
    SO, HP 's raid card is .............

    for shared hosting, is 35mb/s write speed enough?

  37. #37
    Join Date
    Jan 2010
    Location
    so cal
    Posts
    232
    Well depends how much io operations will take place. This raid card is just horrible...my sata drives out perform your 15k sas drives. Your raid controller is really limiting your drives potential.

  38. #38
    Join Date
    Oct 2004
    Posts
    627
    yes, thinking to use just SATA on this poor machine

    miss adaptec 2405 on supermicro now ;p

  39. #39
    Join Date
    Nov 2005
    Posts
    1,224
    Quote Originally Posted by ObjectZone View Post
    When HP controllers see unsupported drives, it seems to cut performance a lot.
    Hmmm, I wonder if that only applies to certain drives. I buy all of our servers without disks, then order carriers and disks from another vendor separately. The pricing is substantially lower for the same disks (Seagate Cheetah 15k SAS, same thing HP ships, they just flash their own firmware) and the warranty is a couple years longer.

    We've had zero issues with compatibility (not even a "non HP disk" warning at POST time) and performance is superb. I'm confident we'd know if otherwise, because the enterprise app (remote desktop) servers are very busy all day and our clients would complain. And these servers get flat-out hammered on the first and last day of the month when everyone does their monthly reconciliations and reporting. Yet still no complaints from users, and my console sessions are quite responsive even under heavy load.
    Last edited by Sekweta; 05-05-2011 at 08:32 AM.

  40. #40
    Join Date
    Oct 2004
    Posts
    627
    Sekweta

    minds sharing some of your serve's IO test on those SAS drives??

    Do you always use raid with BBU though?

Page 1 of 2 12 LastLast

Similar Threads

  1. SAS Drives
    By Spunkyasp in forum WebHostingTalk Subscribers Club
    Replies: 8
    Last Post: 09-24-2010, 02:05 PM
  2. SATA drives with SAS drives?
    By bryonhost1 in forum Dedicated Server
    Replies: 4
    Last Post: 12-13-2009, 06:25 PM
  3. I need some SAS drives
    By Harzem in forum Other Offers & Requests
    Replies: 0
    Last Post: 10-06-2009, 12:19 PM
  4. Need low power server, 1U, 2 Hot Plug SAS drives
    By penguinx64 in forum Colocation and Data Centers
    Replies: 2
    Last Post: 05-27-2008, 12:36 AM
  5. SAS drives
    By DDT in forum Infrastructure & Hosting Company Discussions
    Replies: 11
    Last Post: 02-05-2008, 08:57 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •