Page 1 of 2 12 LastLast
Results 1 to 40 of 53
  1. #1
    Join Date
    Mar 2007
    Posts
    402

    Building a small data center

    I'm sure I got your attention with the title - I promise I won't ask if hosting out of my house is a good idea.

    As part of our new 12,000 sq. ft. offices, we're building out a small data center - about 1,000 square feet (up to 150-160 kW total). I have been blogging on some experiences along the way and would be happy to answer any questions.

    It's exciting and a huge headache at the same time. I hope I can provide some first-hand insights into the process. There is a lot involved that even I underestimated after having been in dozens of data centers over the years.

    You can track our progress on my blog at http://ctdoh.tumblr.com

    While not at all on the scale of many of the providers on here, I hope that some of you find it interesting nonetheless.
    iCall Carrier Services - Carrier-grade VoIP services from a licensed CLEC - http://carriers.icall.com
    Domestic termination and origination, toll-free origination, A-Z International termination, dedicated servers, and colocation in our wholly-owned datacenter
    Real-time ordering via our control panel or XML-based API with over 20,000 numbers in stock

  2. #2
    Join Date
    Apr 2009
    Posts
    1,143
    haha, you know.. for a second there i was thinking.. "Oh God...! not another one" - but its a very interesting blog to follow, i'll be checking it out in the future - Good luck with the project

  3. #3
    Join Date
    May 2006
    Location
    NJ, USA
    Posts
    6,456
    I just want my own cabinet full of my own servers. Just one.
    simplywww: directadmin and cpanel hosting that will rock your socks
    Need some work done in a datacenter in the NYC area? NYC Remote Hands can do it.

    Follow my "deals" Twitter for hardware specials.. @dougysdeals

  4. #4
    Join Date
    Jul 2007
    Location
    Ashburn, VA
    Posts
    1,314
    Quote Originally Posted by Dougy View Post
    I just want my own cabinet full of my own servers. Just one.
    Don't we all?
    --
    Interesting blog. I'll keep my eye on it.
    Preetam Jinka

    Isomerous - High performance web services for business and individuals.
    Bitcable Colocation, KVMs, cPanel hosting, Oracle expertise, and more.

  5. #5
    Join Date
    Nov 2007
    Location
    New Jersey, USA
    Posts
    4,680
    Now this is what I am talking about . Thank god it's not one of those other ones. I will keep my eye on the blog. Extremely interesting.

    - Daniel
    Managed & Unmanaged SSD VPS's by tmzVPS.com | FAST PURE SSD & HDD Plans!
    █ Locations in USA & United Kingdom - Legal Adult Content Allowed! 24/7 Certified Support.
    cPanel/WHM Included | Premium Softaculous | Data Migration Included with Managed Support.
    █ INSTANT SETUP | Paypal, Skrill, 2checkout, BitCoin & all Major Credit Cards Accepted! Skype: tmzvps.chat

  6. #6
    Join Date
    Jun 2001
    Location
    Denver, CO
    Posts
    3,301
    That's a pretty good critical load in a small space. Back of the napkin calculation yields about 45 tons of cooling needed for 150kW. Looks like you only have one or two crac units in your drawings, though?
    Jay Sudowski // Handy Networks LLC // Co-Founder & CTO
    AS30475 - Level(3), HE, Telia, XO and Cogent. Noction optimized network.
    Offering Dedicated Server and Colocation Hosting from our SSAE 16 SOC 2, Type 2 Certified Data Center.
    Current specials here. Check them out.

  7. #7
    thanks for posting. very interesting stuff.
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  8. #8
    Join Date
    Feb 2011
    Posts
    669
    Very interesting.

    We have a similar small data center and your blog would have been very useful 10 years ago (we made a few mistakes)

    Currently we have no back up power but fortunately we are in Santa Clara and have only had one power outage in 10 years I have been here and that lasted less than the UPS time! (Santa Clara has its own utility not subject to black outs etc and rated about 5 9's on its own).

    I was thinking of adding a Natural Gas Generator, (to eliminate fuel storage) and wonder what others think?

    Dave.

  9. #9
    Join Date
    Jan 2006
    Location
    Alabama
    Posts
    1,276
    Thanks for that good read!
    KnownHost Managed VPS Specialists
    Fully Managed VPS and Dedicated Servers

    KnownHost is hiring!
    Click here for more information!

  10. #10
    Quote Originally Posted by david_halliday View Post
    Very interesting.

    We have a similar small data center and your blog would have been very useful 10 years ago (we made a few mistakes)

    Currently we have no back up power but fortunately we are in Santa Clara and have only had one power outage in 10 years I have been here and that lasted less than the UPS time! (Santa Clara has its own utility not subject to black outs etc and rated about 5 9's on its own).

    I was thinking of adding a Natural Gas Generator, (to eliminate fuel storage) and wonder what others think?

    Dave.
    My understanding is that nat gas is good in terms of cost, both for the generator and the fuel, but I think diesel is more often used for datacenter backups because you can get a tank full of diesel, but with nat gas you have to rely on the utility lines continuing to supply gas. As power failures can happen for a wide variety of reasons, many of those reasons could also cause the supply of nat gas to fail as well. Just something to keep in mind.
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  11. #11
    Join Date
    Feb 2011
    Posts
    669
    I get the point about the Natural Gas supply being an issue but as our cooling system relies on it in any case it actually will not detract from overall reliability.

  12. #12
    Quote Originally Posted by david_halliday View Post
    I get the point about the Natural Gas supply being an issue but as our cooling system relies on it in any case it actually will not detract from overall reliability.
    I'm curious how a cooling solution relies on natural gas?
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  13. #13
    Join Date
    Apr 2009
    Location
    Toronto
    Posts
    251
    Quote Originally Posted by david_halliday View Post
    Very interesting.

    We have a similar small data center and your blog would have been very useful 10 years ago (we made a few mistakes)

    Currently we have no back up power but fortunately we are in Santa Clara and have only had one power outage in 10 years I have been here and that lasted less than the UPS time! (Santa Clara has its own utility not subject to black outs etc and rated about 5 9's on its own).

    I was thinking of adding a Natural Gas Generator, (to eliminate fuel storage) and wonder what others think?

    Dave.
    In the event of earth quakes natural gas can often be shut off to prevent leaks/explosions etc. If your in a low risk zone though, they are great!

  14. #14
    Join Date
    Jan 2006
    Location
    Alabama
    Posts
    1,276
    Quote Originally Posted by funkywizard View Post
    I'm curious how a cooling solution relies on natural gas?
    heating? .
    KnownHost Managed VPS Specialists
    Fully Managed VPS and Dedicated Servers

    KnownHost is hiring!
    Click here for more information!

  15. #15
    Quote Originally Posted by EHJonathan View Post
    heating? .
    the person posting said they relied on natural gas for cooling. I didn't understand that statement. Don't see any datacenters needing any additional heating beyond that produced by the servers.
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  16. #16
    Join Date
    Jan 2008
    Location
    Chicago, IL
    Posts
    337
    For a sec I thought "Oh god, didn't this come up last week", but this looks pretty pro. Good job, and good luck.

  17. #17
    Join Date
    Feb 2011
    Posts
    669
    Works a little like this..

    http://www.gasairconditioning.org/ro...w_it_works.htm

    We also have a more normal electrically driven system.

    Dave.

  18. #18
    Join Date
    Jan 2003
    Location
    Chicago, IL
    Posts
    6,889
    Quote Originally Posted by Jay Suds View Post
    That's a pretty good critical load in a small space. Back of the napkin calculation yields about 45 tons of cooling needed for 150kW. Looks like you only have one or two crac units in your drawings, though?
    In our new builds we've been using 70 ton Stulz units. Larger units often yield higher efficiency. Two of those would easily get you N+1 redundancy.
    Karl Zimmerman - Steadfast: Managed Dedicated Servers and Premium Colocation
    karl @ steadfast.net - Sales/Support: 312-602-2689
    Cloud Hosting, Managed Dedicated Servers, Chicago Colocation, and New Jersey Colocation
    Now Open in New Jersey! - Contact us for New Jersey colocation or dedicated servers

  19. #19
    Quote Originally Posted by david_halliday View Post
    Works a little like this..

    http://www.gasairconditioning.org/ro...w_it_works.htm

    We also have a more normal electrically driven system.

    Dave.
    How does the energy cost for that kind of thing compare to a strictly electrical system? I assume it still needs a fair amount of electricity... or no?
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  20. #20
    Join Date
    Feb 2011
    Posts
    669
    We dont know yet, we were thinking of going Natural Gas (both cooling and backup power) but I think MooreAdmin comment is spot on, for us this would be totally stupid as we are in earthquake land! Back to the traditional I guess.

  21. #21
    Join Date
    Mar 2007
    Posts
    402
    Quote Originally Posted by Jay Suds View Post
    That's a pretty good critical load in a small space. Back of the napkin calculation yields about 45 tons of cooling needed for 150kW. Looks like you only have one or two crac units in your drawings, though?
    You're correct. 150 kW is the max I would feel comfortable with in the space. In actuality, it will be closer to 100 kW in execution. We have initial planning for two ~85kW UPS systems and two 20-ton CRACs. The walls on three sides are easily moved/removed. On one corner in particular, we have allowed space for an easy expansion for a third 20-ton unit allowing for approximately N+1 at full capacity of the room. We have another 4,000 square feet of expansion space available if we need it.
    iCall Carrier Services - Carrier-grade VoIP services from a licensed CLEC - http://carriers.icall.com
    Domestic termination and origination, toll-free origination, A-Z International termination, dedicated servers, and colocation in our wholly-owned datacenter
    Real-time ordering via our control panel or XML-based API with over 20,000 numbers in stock

  22. #22
    Join Date
    Mar 2007
    Posts
    402
    Quote Originally Posted by MooreAdmin View Post
    In the event of earth quakes natural gas can often be shut off to prevent leaks/explosions etc. If your in a low risk zone though, they are great!
    From several consultants/vendors I've spoken with, a dual-plant system is becoming somewhat commonplace. Diesel is the initial fuel source, then switching to natural gas for a longer term (past 24-48 hours) solution.

    Hopefully for most installs this is a purely theoretical situation that never has to be dealt with. Unfortunately, you do have to spend the money up front.
    iCall Carrier Services - Carrier-grade VoIP services from a licensed CLEC - http://carriers.icall.com
    Domestic termination and origination, toll-free origination, A-Z International termination, dedicated servers, and colocation in our wholly-owned datacenter
    Real-time ordering via our control panel or XML-based API with over 20,000 numbers in stock

  23. #23
    loving the videos I'm seeing regarding the starline busway power system. I can definitely see some advantages there not having to deal with an electrician to build out additional power circuits.
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  24. #24
    Join Date
    Jan 2003
    Location
    Chicago, IL
    Posts
    6,889
    Quote Originally Posted by funkywizard View Post
    loving the videos I'm seeing regarding the starline busway power system. I can definitely see some advantages there not having to deal with an electrician to build out additional power circuits.
    In a colocation environment, like we have, the Starline is AMAZING, and I highly recommend it. Now, union labor here in Chicago is also expensive, so that plays into it, but being able to have the power flexibility for customers is nearly priceless.
    Karl Zimmerman - Steadfast: Managed Dedicated Servers and Premium Colocation
    karl @ steadfast.net - Sales/Support: 312-602-2689
    Cloud Hosting, Managed Dedicated Servers, Chicago Colocation, and New Jersey Colocation
    Now Open in New Jersey! - Contact us for New Jersey colocation or dedicated servers

  25. #25
    Join Date
    Dec 2007
    Location
    Indiana, USA
    Posts
    16,084
    Very nice blog - I'd like to do something like this in the future so by all means don't hesitate to share all of the intimate details We do need to get several more cabinets before I can really even think about starting something like this (even if very small) so it'll be a while
    Michael Denney - MDDHosting LLC
    New shared plans for 2016! Check them out!
    Highly Available Shared, Premium, Reseller, and VPS
    http://www.mddhosting.com/

  26. #26
    Join Date
    Feb 2005
    Location
    Toronto, Canada
    Posts
    5,158
    Very interesting blog

  27. #27
    Join Date
    Mar 2004
    Location
    Seattle, WA
    Posts
    2,561
    Interesting...

    What software did you use to create the 3D layout?
    ColoInSeattle - From 1U to cage space colocation in Seattle
    ServerStadium - Affordable Dedicated Servers
    Come visit our 18k sq ft. facility in Seattle!
    Managed Private Cloud | Colocation | Disaster Recovery | Dedicated Servers

  28. #28
    Join Date
    Mar 2007
    Posts
    402
    Quote Originally Posted by VN-Ken View Post
    Interesting...

    What software did you use to create the 3D layout?
    It's all done in the free version of SketchUp. I've made a lot of models for cabinets, racks, ladder rack, and even some network items like Cisco 6509s and Juniper routers. I'll probably upload the models to the blog for free use sometime in the near future.

    SketchUp is fantastic once you figure out the idiosyncrasies.
    iCall Carrier Services - Carrier-grade VoIP services from a licensed CLEC - http://carriers.icall.com
    Domestic termination and origination, toll-free origination, A-Z International termination, dedicated servers, and colocation in our wholly-owned datacenter
    Real-time ordering via our control panel or XML-based API with over 20,000 numbers in stock

  29. #29
    Join Date
    Oct 2005
    Location
    Tucson AZ
    Posts
    367
    Quote Originally Posted by KarlZimmer View Post
    In a colocation environment, like we have, the Starline is AMAZING, and I highly recommend it. Now, union labor here in Chicago is also expensive, so that plays into it, but being able to have the power flexibility for customers is nearly priceless.
    We initially had starline and their initial product was... suffice to say.. crap (talking 10 years ago). They've taken a page out of Siemens book now and learned how to properly make a busway.

    We're using a Siemens busway system and I wouldn't trade it for anything, the ability to add outlets of any kind anywhere in a snap is priceless. So, I'm with Karl... if you can, get a bus system.

  30. #30
    Join Date
    Mar 2007
    Posts
    402
    Quote Originally Posted by LoginTech View Post
    We initially had starline and their initial product was... suffice to say.. crap (talking 10 years ago). They've taken a page out of Siemens book now and learned how to properly make a busway.

    We're using a Siemens busway system and I wouldn't trade it for anything, the ability to add outlets of any kind anywhere in a snap is priceless. So, I'm with Karl... if you can, get a bus system.
    We were dead-set on the Starline after we viewed a demo of it. It's great. However, half the room is already going to be filled with our existing equipment that we know exact power draws and requirements on. Our landlord offered to do the electric drops under the existing permit and basically free to us, so it was hard to say no. In the end, I'm glad we didn't go with it due to space issues. We probably would've had to mount the busway rotated 90 degrees.
    iCall Carrier Services - Carrier-grade VoIP services from a licensed CLEC - http://carriers.icall.com
    Domestic termination and origination, toll-free origination, A-Z International termination, dedicated servers, and colocation in our wholly-owned datacenter
    Real-time ordering via our control panel or XML-based API with over 20,000 numbers in stock

  31. #31
    Join Date
    Oct 2005
    Location
    Tucson AZ
    Posts
    367
    Quote Originally Posted by voipcarrier View Post
    We were dead-set on the Starline after we viewed a demo of it. It's great. However, half the room is already going to be filled with our existing equipment that we know exact power draws and requirements on. Our landlord offered to do the electric drops under the existing permit and basically free to us, so it was hard to say no. In the end, I'm glad we didn't go with it due to space issues. We probably would've had to mount the busway rotated 90 degrees.
    Hey, if you have a known load and spec'd deployment plan.... and it's free... you'd be a fool to not have done that.

  32. #32
    Nice blog! Definitely will be following this to see how it comes together. I'm a ways off from endeavoring anything on this scale, but it's still really interesting nevertheless.

  33. #33
    Quote Originally Posted by KarlZimmer View Post
    In a colocation environment, like we have, the Starline is AMAZING, and I highly recommend it. Now, union labor here in Chicago is also expensive, so that plays into it, but being able to have the power flexibility for customers is nearly priceless.
    How much does a busway like that cost anyway?
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  34. #34
    Join Date
    Oct 2005
    Location
    Tucson AZ
    Posts
    367

  35. #35
    Join Date
    Jan 2003
    Location
    Chicago, IL
    Posts
    6,889
    Quote Originally Posted by funkywizard View Post
    How much does a busway like that cost anyway?
    Hard to give a blanket answer, depends on the actual build, labor costs, etc. For us, it is about 20% more than pre-wiring the entire facility at once (bulk/volume discount basically), but then we're dealing with a set configuration and needing to bring in electricians to make changes as needed, etc. If we're doing circuits a couple at a time, the busway is a good bit cheaper and we can install a circuit in 2 minutes, not 2 days.
    Karl Zimmerman - Steadfast: Managed Dedicated Servers and Premium Colocation
    karl @ steadfast.net - Sales/Support: 312-602-2689
    Cloud Hosting, Managed Dedicated Servers, Chicago Colocation, and New Jersey Colocation
    Now Open in New Jersey! - Contact us for New Jersey colocation or dedicated servers

  36. #36
    Quote Originally Posted by KarlZimmer View Post
    Hard to give a blanket answer, depends on the actual build, labor costs, etc. For us, it is about 20% more than pre-wiring the entire facility at once (bulk/volume discount basically), but then we're dealing with a set configuration and needing to bring in electricians to make changes as needed, etc. If we're doing circuits a couple at a time, the busway is a good bit cheaper and we can install a circuit in 2 minutes, not 2 days.
    Certainly sounds reasonably priced to me then.
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  37. #37
    Join Date
    Apr 2006
    Location
    Phoenix
    Posts
    808
    From the last time I looked at it, the way to make it cost effective is to use 1 box connected to the bus-line for every 2-3 cabinets, which was not an option for us, that is why we did not go that direction.

    I know everyone does the math in their head to what a datacenter costs and says, wow I could really save some money here but what they are not taking into account is this.

    The cost difference between having space in a world class facility, with 40-50 staff members, full time facility, security etc. Fully certified (SAS/PCI/etc) and a smaller facility, staffed with 3-10 people, using used gear, that doesn't have the certifications/security etc and does not show as well is what - 300$/rack?

    Lets say a hosting company gets huge and needs to take down 50 racks. We are talking about $15k a month savings to be a lower quality facility, and take a huge capital risk. 15k/month may sound like a lot, but it is really not. It is one decent level employee, to risk your entire company...

    The numbers just do not work out.
    Jordan Jacobs | VP, Products|SingleHop| JJ @SingleHop.com
    Managed Dedicated Servers | Bare-Metal Servers | Cloud Services

  38. #38
    Join Date
    Mar 2007
    Posts
    402
    Quote Originally Posted by JordanJ View Post
    From the last time I looked at it, the way to make it cost effective is to use 1 box connected to the bus-line for every 2-3 cabinets, which was not an option for us, that is why we did not go that direction.
    They make dual-breaker boxes. Although if you generally run two 20 or 30A drops per cab, then you're back to a single box per cabinet.

    Quote Originally Posted by JordanJ View Post
    I know everyone does the math in their head to what a datacenter costs and says, wow I could really save some money here but what they are not taking into account is this.

    The cost difference between having space in a world class facility, with 40-50 staff members, full time facility, security etc. Fully certified (SAS/PCI/etc) and a smaller facility, staffed with 3-10 people, using used gear, that doesn't have the certifications/security etc and does not show as well is what - 300$/rack?

    Lets say a hosting company gets huge and needs to take down 50 racks. We are talking about $15k a month savings to be a lower quality facility, and take a huge capital risk. 15k/month may sound like a lot, but it is really not. It is one decent level employee, to risk your entire company...

    The numbers just do not work out.
    I'm not sure if this was directed to me, but I'll answer it anyways. In our case, we most definitely did not simply do the math in our head. This was a drawn out and thoroughly investigated decision.

    We're not a hosting company - the success of our build out does not rely on 100 customers taking space with us. We have a quantifiable cost savings based on our own space, power, and bandwidth requirements. Additionally, since we're getting a CLEC license in Texas we can do all sorts of neat stuff with right of way and conduit access if we ever decide to expand and take that route.

    In some cases it may be a financial risk, but in our business model it made sense for us to do. Part of why I put this blog up is for people to understand what's involved. It is by no means cheap and I'm not saying every customer with a few colo racks should go out and do this.
    iCall Carrier Services - Carrier-grade VoIP services from a licensed CLEC - http://carriers.icall.com
    Domestic termination and origination, toll-free origination, A-Z International termination, dedicated servers, and colocation in our wholly-owned datacenter
    Real-time ordering via our control panel or XML-based API with over 20,000 numbers in stock

  39. #39
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    681
    Quote Originally Posted by voipcarrier View Post
    I'm sure I got your attention with the title - I promise I won't ask if hosting out of my house is a good idea.

    As part of our new 12,000 sq. ft. offices, we're building out a small data center - about 1,000 square feet (up to 150-160 kW total). I have been blogging on some experiences along the way and would be happy to answer any questions.

    It's exciting and a huge headache at the same time. I hope I can provide some first-hand insights into the process. There is a lot involved that even I underestimated after having been in dozens of data centers over the years.

    You can track our progress on my blog at http://ctdoh.tumblr.com

    While not at all on the scale of many of the providers on here, I hope that some of you find it interesting nonetheless.

    For years now I'm wondering if it's worth to start a collaborative wiki with things to check for datacenter moves and datacenter planning. What do you think?

    ...the oldest book I have (actually, inherited) on the topic is from the 70's, and I'm yet to see any new ones.
    Every time a new datacenter is built, many new fun stories emerge from the things we overlooked.


    Good luck, I have seen how many things can go wrong in this. May you solve them all without too much sweat.

  40. #40
    Quote Originally Posted by voipcarrier View Post
    I'm not sure if this was directed to me, but I'll answer it anyways.
    I'm guessing it was directed at me since I had a similar thread regarding this
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

Page 1 of 2 12 LastLast

Similar Threads

  1. Question about building a data center
    By Arber in forum Colocation and Data Centers
    Replies: 37
    Last Post: 08-27-2010, 07:16 PM
  2. Building A Data Center
    By ultimategp5 in forum Colocation and Data Centers
    Replies: 20
    Last Post: 11-01-2009, 07:57 PM
  3. Building a data center
    By the_pm in forum Dedicated Server
    Replies: 6
    Last Post: 03-15-2007, 08:24 AM
  4. Building A Data Center
    By Ian Gordon in forum Colocation and Data Centers
    Replies: 18
    Last Post: 01-11-2007, 08:26 PM
  5. Building a data center
    By deeboy in forum Running a Web Hosting Business
    Replies: 5
    Last Post: 09-04-2001, 12:06 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •