Results 1 to 18 of 18

Thread: Host using SANs

  1. #1

    Host using SANs

    I am curious for those hosting companies that are out there using SANs what type of disks are you using SATA or SAS or FC drives. The next question is if you are using SATA/nerline SATA what raid levels are you using?

  2. #2
    Join Date
    Mar 2009
    Location
    /usr/bin/perl
    Posts
    971
    SAS in RAID-DP
    Ask me about CloudCentrum (coming soon) -- The complete, turn-key cloud software solution

  3. #3
    Join Date
    Jun 2002
    Location
    PA, USA
    Posts
    5,143
    We use several iSCSI SANs over the last 2.5 years.

    1. Dell/EMC Ax150i - 12 x SATA 7200 rpm drives. Used for web servers, RAID5 works perfectly fine.
    2. Dell PowerVault MD3000i - 15 drives, can mixed SATA and SAS drives. We used this for mail/database servers using SAS 15K drives in RAID10.
    3. Dell EqualLogic PS5500E - 48 x SATA 7200 rpm drives in 4U (GREAT!). With the high amount of spindles, these SATA drives actually gave the same IOPS as 16 x 15K SAS drives. So we used this for web, mail, database serveers. Currently RAID10 is setup on this array. Although due to the high number of spindles we don't actually see much performance difference between RAID10 and RAID50. We can change the RAID level from RAID10 to RAID50 down the road if we need the capacity.
    Fluid Hosting, LLC - Enterprise Cloud Infrastructure: Cloud Shared and Reseller, Cloud VPS, and Cloud Hybrid Server

  4. #4
    Quote Originally Posted by FHDave View Post
    2. Dell PowerVault MD3000i - 15 drives, can mixed SATA and SAS drives. We used this for mail/database servers using SAS 15K drives in RAID10.
    What kind of performance were you getting on the MD3000i? I tested a similar product, the Jestor 416is with 16 SATA drives, and it was simply worthless. Tried different switches, jumbo frames, and could not get decent performance. We ended up switching back to internal arrays. Have stayed away from iscsi products ever since.

  5. #5
    Join Date
    Aug 2009
    Location
    Zurich - Switzerland
    Posts
    28
    We also use both several FC and iSCSI SANs over the last 4 years.

    EMC Ax150 FC - 12 x SAS 10k rpm drives. (No more used in production)
    EMC AX4 FC - 12 x SAS 15k rpm drives. We used this for VMware ESX servers using drives in RAID5.
    EMC Clariion CX3-40. Can mix both SAS and SATA. This is the best in performance we have ever used. We're using this machine for our VMware ESX Hosting.
    HP DL380G5 and HP DL380G6 with LeftHand. The performance depends from what kind of server you build it from, but it gives you a lot of features like Mirroring/Clustering across different Storage and different Datacenters. We use this solution for Disaster Recovery solutions across different DC or for iSCSI storage for dedicated servers.

    At the and the best is the CX3-40 for performance, but the LeftHand SAN wins on EMC2 for its features.

  6. #6
    Join Date
    Aug 2009
    Location
    Zurich - Switzerland
    Posts
    28
    I forgot to say that on EMC CX3-40 we have 5000 IOPs and peak at 30000.
    On the LeftHand we are "only" getting 1500 IOPS and peaks at 8000.
    It's not too mutch but CX3 is FC and LeftHand is iSCSI.


    @forasse. Did you used TCP offload NICs ?

  7. #7
    We currently utilize several iSCSI SAN's within our environment, mostly the PS4000 and PS6000 with 15K SAS drives configured in a RAID10 array. The replication feature built in with the EqualLogic platform is simply amazing.
    Antony Mascarenhas How can I help? antony_m@zysek.com
    Zysek Technologies Pvt. Ltd. - Indian Datacenter ¦ Hyderabad & Mumbai
    Web Hosting · Virtual Servers · Dedicated Servers · Colocation · Managed Services

  8. #8
    Join Date
    Jun 2002
    Location
    PA, USA
    Posts
    5,143
    Quote Originally Posted by EngineNetworks View Post
    I forgot to say that on EMC CX3-40 we have 5000 IOPs and peak at 30000.
    I am always suspicious with quoted IOPS number, especially when cache is involved. I would propose using 100% random IO operation when quoting IOPS and compare one product over another.

    If my calculation is correct, typically a 15K rpm ~4.5 ms drives will give you ~250 IOPS assuming 100% random operation. Assume RAID0 (fastest IOPS on any raid level), then 16 drives (Typical) 15K rpm drives would yield a physical limit of 4000 IOPS. .

    So to get 30000 IOPS, one must ask:
    1. how many drives?
    2. what kind of drives?
    3. what RAID level?
    4. what's the assumption read/write ratio?
    5. what's the assumption % random IO operation?
    6. is your cache disabled? are you sure you are not benchmarking your cache?
    Fluid Hosting, LLC - Enterprise Cloud Infrastructure: Cloud Shared and Reseller, Cloud VPS, and Cloud Hybrid Server

  9. #9
    Join Date
    Jun 2002
    Location
    PA, USA
    Posts
    5,143
    Quote Originally Posted by antony_m View Post
    We currently utilize several iSCSI SAN's within our environment, mostly the PS4000 and PS6000 with 15K SAS drives configured in a RAID10 array. The replication feature built in with the EqualLogic platform is simply amazing.
    How do you replicate? Is that how you do your array backup?
    Fluid Hosting, LLC - Enterprise Cloud Infrastructure: Cloud Shared and Reseller, Cloud VPS, and Cloud Hybrid Server

  10. #10
    @FHDave
    We utilize the EqualLogic platform for snapshots and replication over WAN to our second location.
    Antony Mascarenhas How can I help? antony_m@zysek.com
    Zysek Technologies Pvt. Ltd. - Indian Datacenter ¦ Hyderabad & Mumbai
    Web Hosting · Virtual Servers · Dedicated Servers · Colocation · Managed Services

  11. #11
    Join Date
    Aug 2009
    Location
    Zurich - Switzerland
    Posts
    28
    Quote Originally Posted by FHDave View Post
    I am always suspicious with quoted IOPS number, especially when cache is involved. I would propose using 100% random IO operation when quoting IOPS and compare one product over another.

    If my calculation is correct, typically a 15K rpm ~4.5 ms drives will give you ~250 IOPS assuming 100% random operation. Assume RAID0 (fastest IOPS on any raid level), then 16 drives (Typical) 15K rpm drives would yield a physical limit of 4000 IOPS. .

    So to get 30000 IOPS, one must ask:
    1. how many drives?
    2. what kind of drives?
    3. what RAID level?
    4. what's the assumption read/write ratio?
    5. what's the assumption % random IO operation?
    6. is your cache disabled? are you sure you are not benchmarking your cache?
    This SAN is based on nearly 1 hundred of 15k rpm disks with FC Interface.
    The CX3 40 can support up to 240 Disks, but on this Storage you probabibly run out of Processor resources sooner the Disk speed bootleneck.

    I can suppose that all the IO is random becouse there's many esx servers connected with different VMs each.

  12. #12
    Join Date
    Jun 2002
    Location
    PA, USA
    Posts
    5,143
    Quote Originally Posted by antony_m View Post
    @FHDave
    We utilize the EqualLogic platform for snapshots and replication over WAN to our second location.
    What do you use for your snapshot respiratory?
    Fluid Hosting, LLC - Enterprise Cloud Infrastructure: Cloud Shared and Reseller, Cloud VPS, and Cloud Hybrid Server

  13. #13
    Join Date
    Jun 2002
    Location
    PA, USA
    Posts
    5,143
    Quote Originally Posted by EngineNetworks View Post
    This SAN is based on nearly 1 hundred of 15k rpm disks with FC Interface.
    So when you compare that to LeftHand, are you comparing the same number of spindles and drive types on LeftHand?
    Fluid Hosting, LLC - Enterprise Cloud Infrastructure: Cloud Shared and Reseller, Cloud VPS, and Cloud Hybrid Server

  14. #14
    Join Date
    Jun 2002
    Location
    PA, USA
    Posts
    5,143
    BTW, for those using FC, I wonder if you think it's worth it? For me the complexity is far more than desired. Besides, iSCSI SAN can now (or starting to) adopt the 10Gbps ports, more than double the 4Gbps FC ports. And the use of ethernet makes iSCSI much more expandable in the future than FC, IMHO.
    Fluid Hosting, LLC - Enterprise Cloud Infrastructure: Cloud Shared and Reseller, Cloud VPS, and Cloud Hybrid Server

  15. #15
    Join Date
    Aug 2009
    Location
    Zurich - Switzerland
    Posts
    28
    Quote Originally Posted by FHDave View Post
    So when you compare that to LeftHand, are you comparing the same number of spindles and drive types on LeftHand?
    No, we're not comparing with the same spindles.
    LeftHand SAN is running with 8 or 16 SAS Disks.

    As far as I see on those SAN, the interface speed like 4Gbps or 10Gbps is not so important like Controller processors.
    EMC2 CX3 is anything but cheap, but it gives you more troughput becouse it use multiple controllers.

    And yes it worth the price only if you host high quality services or very important application that need high availability and better performance.

  16. #16
    Join Date
    Nov 2007
    Location
    West Palm Beach, FL
    Posts
    275
    Quote Originally Posted by forasse View Post
    What kind of performance were you getting on the MD3000i? I tested a similar product, the Jestor 416is with 16 SATA drives, and it was simply worthless. Tried different switches, jumbo frames, and could not get decent performance. We ended up switching back to internal arrays. Have stayed away from iscsi products ever since.
    I can comment if you like. We also have an MD3000i -- all 15K 450GB SAS disks in RAID-6. We've filled out capacity on the unit and have close to 120 clients in production with good performance. What do I consider "good"? We directly compared our performance with those of our competitors who use a range of products/methods from RAID-10 on a local server to large Enterprise class SANs. Our performance is as good and in many cases better. Granted, workloads vary and so do how clients ultimately use the VMs. Nearly all of our machines are WEB workloads with some light SQL.

    The MD3000i was a fantastic entry into the SAN market. We are still very happy with the purchase, BUT.. it does have its pain points. Reporting (specifically performance) is nearly non-existent. Expanding LUNs is painful (operations can take 12-14 hours in our case). Management features can be lacking. No advanced features like replication or thin provisioning. Bang for the buck though, if you're just starting to get into SAN and/or iscsi it's a great value as long as you understand the limitations.

    The MD3000i did teach us though that our next SAN implementation didn't have to be SAS. We've purchased some Equallogic units that are all SATA. I think most people simply assume they have to have SAS when SATA is generally fine for their application. I really think this carries over from (it did in my case) feeling they'd never run SATA in a server, so why use it in a SAN. We were able to determine that our existing implementation would easily run on a PS6000e SATA box.

    For us, FC really wasn't an option. IMHO, unless you're running very heavy SQL loads (large dedicated clusters) it isn't generally needed in the "hosting" space. The cost and additional network infrastructure are prohibitive and the additional performance wasn't warranted. YMMV.

    We just received our first PS6000e I'm really looking forward to building out our next cluster. We anticipate having a couple PS6500's shortly as well. I think the Dell EQ boxes are really the way to go for iscsi.

    Good luck -
    Last edited by AI-Wayne; 11-19-2009 at 04:14 PM.
    Applied Innovations (www.appliedi.net)
    Microsoft Gold Certified Hosting Partner, ASP.net Featured Host.
    Specializing in Windows Hosting since 1999.

  17. #17
    Join Date
    Aug 2009
    Location
    Zurich - Switzerland
    Posts
    28
    Quote Originally Posted by AI-Wayne View Post
    For us, FC really wasn't an option. IMHO, unless you're running very heavy SQL loads (large dedicated clusters) it simply isn't needed in the "hosting" space. The cost and additional network infrastructure are prohibitive and the additional performance not needed.
    That's exactly what I mean.
    ENGINE NETWORKS - Blade Dedicated Server - VMware Cloud Hosting - Colocation
    Multiple datacenter, Geneva, Zurich, and Milan
    Since 2005 in the market - http://www.enginenetworks.net

  18. #18
    Join Date
    Jun 2002
    Location
    PA, USA
    Posts
    5,143
    Quote Originally Posted by EngineNetworks View Post
    No, we're not comparing with the same spindles.
    LeftHand SAN is running with 8 or 16 SAS Disks.
    In that case, it seems your LeftHand is performing bettern than the CX3-40. Even if you have 16 drives on your LeftHand array, that's about 6 times less than what you have on CX3-40, and only <4 times less in IOPS.

    As far as I see on those SAN, the interface speed like 4Gbps or 10Gbps is not so important like Controller processors.
    EMC2 CX3 is anything but cheap, but it gives you more troughput becouse it use multiple controllers.
    I learned it the other day, it does not matter what your controller can do. What matters is what throughput you can do per volume.

    And yes it worth the price only if you host high quality services or very important application that need high availability and better performance.
    All this I can do with my EqualLogic iSCSI SAN. And it seems your iSCSI LeftHand SAN beats the more expensive/complex setup on your CX3-40?
    Fluid Hosting, LLC - Enterprise Cloud Infrastructure: Cloud Shared and Reseller, Cloud VPS, and Cloud Hybrid Server

Similar Threads

  1. Winchester SANs still around?
    By Jeremy in forum Colocation, Data Centers, IP Space and Networks
    Replies: 2
    Last Post: 08-22-2009, 11:59 PM
  2. Rackspace sans bandwidth?
    By mripguru in forum Colocation, Data Centers, IP Space and Networks
    Replies: 2
    Last Post: 09-29-2005, 08:29 PM
  3. Sans Font
    By m2k961 in forum Web Hosting Lounge
    Replies: 2
    Last Post: 10-21-2003, 07:31 PM
  4. New Sans/FBI Top 20 list
    By Deahost in forum Web Hosting Lounge
    Replies: 3
    Last Post: 10-03-2002, 07:51 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •