Page 1 of 4 1234 LastLast
Results 1 to 25 of 87
  1. #1
    Join Date
    Jun 2009
    Location
    Washington
    Posts
    374

    Cool Building, running & managing your own Server Farm

    Building, running and managing your own Server Farm.

    This thread is dedicated to those who find themselves on or considering this path.

    Here, I am not hosting any ‘foreign business interests’ or associating with any other ‘hostile networks’. I am needing to create many sub-nets, with 3 to 5 WAN’s inbound and outbound. I have a ‘research sub-net’ needing special considerations and will link associates/contractors by VPN. The product-line will also require Radius Server(s). Plus more details and requirements that make the whole project most complicated.


    (Thanks to ‘leeware’ for this beginning theme)

    1.0 Objective: Building a 30-50 Server facility:

    I checked my records and made this decision 2 - ˝ months ago. Since then I have acquired 28 servers, an internet security device, two 3com’s, two 4-port kvm’s, a few small Cisco switches and a host of power related stuff.

    2.0 Justification for Building vs leasing:

    Control. Costs. Open options. Unique demands more easily appreciated by myself as I am the one possessing the vision. Choosing who I work with as I do the hiring/contracting or initiate associations.

    We are generally molded by personal experiences and these set-up our prejudices. I am certainly no exception. My experiences with hosts generated an excellent introduction to the wide world of websites/control panels etc and some products that support it. But the too often cryptic communications, some surprise costs due to my ignorance and that vulnerable ‘boxed in’ feeling made me wonder…

    2.1 What is your rational for building vs leasing?

    One of my major ‘cross the line’ rational came when I discovered I could buy a used server each month for a similar cost of leasing one. Next, were all my own confounded ‘special circumstances’ that kept cropping up and of course these were not the fault of any hosting company. Lastly technology came to my street in the form of good multiple pipes and my zoning is acceptable because I am off a major highway.

    3.0 Business Objectives:

    My business objectives center around a very long-run project that I have already long ago completed and keep updated to viability. And I would rather keep that subject to another venue.

    4.0 Budgetary Information:

    I have enough ‘resources’ to pull it off.

    4.2 What is your operating budget (How much money do you plan to spend on an ongoing basis to keep the place running for 12-24 months?)

    Same as previous and I will note that ‘time and attention’ is keenly important.

    4.3 More importantly, where is this money going to come from? If from investors they generally require a sound business plan.

    I require a sound business plan and have it.

    5.0 Return on Investment:

    Accomplish goals on and on…

    5.1 After you account for the initial capital expense to get everything up and running + your on going operating costs, how long will it take you to make back your initial investment and then reach profitability? (12/24/36 months for longer?)

    3-6 months.

    Special note: I always pay attention to fundamentals. Question my assumptions and verify facts with research and testing. I ask lots of questions and get professional feedback. Hence why I respect being a member of this forum.

    Sincerely,

    Mavus
    ABSF
    Arrogant Bastard Server Farm

    And as the last sales person told me…

    “I wanted to get back to you with the best information I could gather, especially with a business name like that!!”


    Next posts:

    Details of what 2 & ˝ months of gathering resources with research is like.
    ABSF
    Arrogant Bastard Server Farm
    Built from scratch Data Center serving
    100 year-old Metaphysical Library

  2. #2
    Join Date
    Jun 2009
    Location
    Washington
    Posts
    374

    Coming up to speed via some 'information' and some organization

    (Left off in previous post a few arrays...)

    Continuing on...


    List of books acquired, read and now being referenced or studied where appropriate last two months or so.

    TCP/IP guide by Charles M. Kazerok
    Upgrading and Repairing Servers by Scott Mueller etc.
    Linux Administrator Street Smarts by Roderick W. Smith
    Deluxe Edition of CompTIA Security+ by Emmett Dulaney
    Counter Hack Reloaded by Ed Skoudis with Tom Liston
    Cisco PIX Firewalls by Richard A. Deal
    CCNA Cisco Certified Network Associate 5th Edition by Todd Lammie
    Linksys Networks, 3rd edition by Walter Glenn
    Run your own Web Server using Linux & Apache, by Stuart Langridge etc.
    Ubuntu Linux Toolbox by Christopher Negus & Francios Caen
    Also purchased and read two books on electrical house wiring and one on house remodeling with emphasis on environmental control.

    And a dozen or so user manuals of various devices and servers.

    =========================

    What I quickly noticed is for forward planning one need start with 'security', stay with it and end with it.

    I read all these books and guides entirely and like the 'Upgrading and Repairing Servers' the best.

    Admittedly to bring what quickly came to be a very complicated quick education needed some organizing in my head.

    Also note, this does not 'include' the list of a dozen or so 'special use or special purpose' software applications that I came in contact with during this period of time or the time on business related websites.

    So I organized in, or after this manner:

    1) Full Scope of Business Plan with emphasis on 'requirements'

    2) Environment; specifically temperature, space, and power.

    3) Network; as security, organization and access

    4) Redundancy; by way of storage, back-up and maintenance

    5) OS/Platform; LAMPV

    6) CMS or Applications; MOODLE and various CMS

    8) Proprietary Software; SNE, The Library (I wrote both)

    9) Maintenance/Management Tools; various

    10) Identification of the CRITICAL PATH

    -=====================================

    A few thoughts and observation:

    This journey is not for the faint of heart or a 'slow study'.

    Next posts will cover the outline above is some manner...

    Mavus,
    Last edited by Mavus; 07-25-2009 at 02:22 AM.
    ABSF
    Arrogant Bastard Server Farm
    Built from scratch Data Center serving
    100 year-old Metaphysical Library

  3. #3
    Join Date
    Jun 2009
    Location
    Washington
    Posts
    374

    hot musings...

    Here is a poem for ya...

    When its too hot you think of the wimpy air conditioner
    And dream of the 'big one' you'll put in better next

    When you think of all your bloody requirements
    You pinch yourself to challenge that crazy feel'in

    You remember the 30' ceiling fan still boxed in the garage
    And new electrical wiring waiting for cool air in the attic

    Then you think; 'what truly is better use of my powers'?
    A rain dance or conjuring Fall weather before its time?

    Still plenty to work on...

    ...maybe just configure a server!
    ABSF
    Arrogant Bastard Server Farm
    Built from scratch Data Center serving
    100 year-old Metaphysical Library

  4. #4
    Join Date
    Nov 2007
    Location
    Northallerton
    Posts
    249
    Shame it does not rhyme

  5. #5
    Join Date
    Jun 2009
    Location
    Washington
    Posts
    374
    Quote Originally Posted by coeplicltd View Post
    Shame it does not rhyme
    Ya, probably too hot to rhyme today.

    It will be a 'cat nap' and back to work.
    ABSF
    Arrogant Bastard Server Farm
    Built from scratch Data Center serving
    100 year-old Metaphysical Library

  6. #6
    Join Date
    Jun 2009
    Location
    Washington
    Posts
    374

    * heat waves...

    2) Environment; specifically temperature, space, and power.

    Good thing this is planned out over the span of another year, as right now we are in a heat-wave and we cannot get to work till it cools way down...

    Note to Self: Always pay close attention to environmental planning and resources.

    I cannot muscle in a perfect environment, so air flow management is first, some muscle later.

    I planted trees to shade this side of the house a few years ago, however they seem to grow at their own pace?!

    For those who intend to do this, be prepared to change hats more often and more sharply than for certainly any other entrepreneur undertaking I have ever been party to or know about.
    ABSF
    Arrogant Bastard Server Farm
    Built from scratch Data Center serving
    100 year-old Metaphysical Library

  7. #7
    Join Date
    Mar 2006
    Location
    Reston, VA
    Posts
    3,131
    Quote Originally Posted by Mavus View Post

    I planted trees to shade this side of the house a few years ago, however they seem to grow at their own pace?!
    you really didn't just say you planted tree's to help cool your house that you are running all of this out of?
    Yellow Fiber Networks
    http://www.yellowfiber.net : Managed Solutions - Colocation - Network Services IPv4/IPv6
    Ashburn/Denver/NYC/Dallas/Chicago Markets Served zak@yellowfiber.net

  8. #8
    Join Date
    Jun 2001
    Location
    Denver, CO
    Posts
    3,302
    Quote Originally Posted by Spudstr View Post
    you really didn't just say you planted tree's to help cool your house that you are running all of this out of?
    No kidding. For 50 server environment, depending on specs, you will need approximately 2-3 tons of cooling. Shade trees will not exactly help ... I guess you could get about 4 12000 BTU (1 ton) window units, assuming you got that many windows, and then you would technically have N+1 cooling ...
    Jay Sudowski // Handy Networks LLC // Co-Founder & CTO
    AS30475 - Level(3), HE, Telia, XO and Cogent. Noction optimized network.
    Offering Dedicated Server and Colocation Hosting from our SSAE 16 SOC 2, Type 2 Certified Data Center.
    Current specials here. Check them out.

  9. #9
    Join Date
    Jun 2009
    Location
    Washington
    Posts
    374

    * ...still hot, well the hottest day yet in our recorded history here, HOT

    Quote Originally Posted by Jay Suds View Post
    No kidding. For 50 server environment, depending on specs, you will need approximately 2-3 tons of cooling. Shade trees will not exactly help ... I guess you could get about 4 12000 BTU (1 ton) window units, assuming you got that many windows, and then you would technically have N+1 cooling ...
    Well I actually have the 'homemade' server racks spread in two different buildings separated from each other by 100 feet. One sports a high ceiling and is well ventilated; a garage/shop.

    So no, I will not have 50 machines in one space, more like 30 over two. But admittedly even 15 is yet to be proven out as manageable. However, I have options besides the 'trees' <smiles>

    One, I can wait till fall to turn most of them on; two I can host out to a company down in Seattle for the hottest summer months next year. Three, I am adding a near professional ventilation scheme for both buildings throughout remainder of this year and next.

    The question is effectiveness and timing. And I hope to get by with two comparable units such as you pointed out; one for each building/server room. The house one will be able to re-purpose existing unused, 'under the house duct-work' and focus on this server/library room entirely. It is now one larger room where I took out the common adjoining closets in each and made one long room.

    But then, 90% of what I am doing here is something I have never done before, as I am actually a writer/researcher and poet.

    But hey... I am coming up to speed in a fury...
    ABSF
    Arrogant Bastard Server Farm
    Built from scratch Data Center serving
    100 year-old Metaphysical Library

  10. #10
    Join Date
    Dec 2006
    Location
    Cardiff, Wales
    Posts
    803
    Well. Where to begin.

    If i give you a rough of idea where we are standing.

    We are currently in the middle of negotiating the 110,000sq ft land owned by our local council to build our own 80,000sq ft teir 3+/4 N+1 Redundant DC/exchange. Not only it will be one of the greenest, but it will have to start with a 200Gbit Connectivity quickly growing to nearly 600gbit. This might sound a hell of alot of connectivity, but if you knew who our clients were, then you'd obviously say ah. Were currently looking at costs of 500mil GBP at the moment, but obviously your no where near as big as that.

    Feel free to PM if you need any help.

    It really is a Jungle out there.

    Best of luck!

    Nathaniel
    SIP Trunking and VoIP Lines, Numbering (DIDs and DDIs), Low Cost Minute add-ons, Secure SIP and VoIP. Business and Enterprise Grade Quality, Low Cost and Highly Competitive. Available at: https://www.voipyonder.com/

  11. #11
    Join Date
    Jun 2009
    Location
    Washington
    Posts
    374

    Smile thanks!!

    Quote Originally Posted by CymraegWalesHosting View Post

    Feel free to PM if you need any help.

    It really is a Jungle out there.

    Best of luck!

    Nathaniel
    Gosh, actually thanks a lot and I will! And good fortune to your project.

    I am starting with three pipes totaling 21.5 up and more than that down but that is not so important. I might double the up in a year or so with a good take off.

    And I just learned about IDS these last few days. <smiles>
    Before I had security in a bunch of pieces; hardware, software and disconnected ideas. However the Intrusion Detection System as a 'whole idea' really pulled it together for me in my mind.

    Kind Regards and thanks again!
    ABSF
    Arrogant Bastard Server Farm
    Built from scratch Data Center serving
    100 year-old Metaphysical Library

  12. #12
    Join Date
    Mar 2006
    Location
    Reston, VA
    Posts
    3,131
    Quote Originally Posted by Jay Suds View Post
    No kidding. For 50 server environment, depending on specs, you will need approximately 2-3 tons of cooling. Shade trees will not exactly help ... I guess you could get about 4 12000 BTU (1 ton) window units, assuming you got that many windows, and then you would technically have N+1 cooling ...

    well maybe if he lived in iceland where it never gets above 65 that would do the trick too... just need a crap ton of forced air from the outside
    Yellow Fiber Networks
    http://www.yellowfiber.net : Managed Solutions - Colocation - Network Services IPv4/IPv6
    Ashburn/Denver/NYC/Dallas/Chicago Markets Served zak@yellowfiber.net

  13. #13
    Join Date
    Jun 2009
    Location
    Washington
    Posts
    374

    Talking Not Iceland but still north and on the water

    Quote Originally Posted by Spudstr View Post
    well maybe if he lived in iceland where it never gets above 65 that would do the trick too... just need a crap ton of forced air from the outside
    <smile>

    Actually, the average temperature here from October 1st to March 31 is 43 degrees night and day.

    The average temperature here from April 1 to September 30 is 56 degrees night and day.

    So any environmental system need move in outside air to cool most of the time just fine. I will be installing a large 30 inch ceiling fan and an interior wall fan when I want to circulate some hot air in the winter elsewhere in the house and medium-sized fans on the two nearby windows to bring in cool air. The floor sports to vents to feed 30 to 1 by area overkill cooled air in the few weeks it is hot/warm day/night each summer.

    I expect to run 10 to 20 units in this environment just fine. We'll see of course, but I know this weather here very well. It is running these servers that is new/unknown.

    But then that is one of the reasons I am posting this thread here...
    <big grin>

    Kind Regards,
    ABSF
    Arrogant Bastard Server Farm
    Built from scratch Data Center serving
    100 year-old Metaphysical Library

  14. #14
    Join Date
    Jun 2009
    Location
    Washington
    Posts
    374
    Hi all!

    And especially all courageous souls that are keen on doing it ALL!!

    Some days better than others so seeming especially since now the weather is back to 'normal'; cloudy and cool.

    I have engineered (ok nice word), mapped out my environmental strategy, space use and server access. As well have identified five major milestones running 15-18 months. Three months fudge room, I hope I do not have to use it all.

    After I complete some more and have actual running tests I will hopefully be able to pontificate on wondrous success!! I have an air flow scheme to build in next few days and do have a little help.

    Whew, this is not easy.

    The server farm operation and business for which it is created takes up some 450 (flat) square feet mostly split in half betwixt two buildings 100 feet apart; but every (cubic) square foot is largely spoken for.

    later...
    ABSF
    Arrogant Bastard Server Farm
    Built from scratch Data Center serving
    100 year-old Metaphysical Library

  15. #15
    Join Date
    Oct 2009
    Location
    ZombieLand
    Posts
    205
    Quote Originally Posted by Jay Suds View Post
    No kidding. For 50 server environment, depending on specs, you will need approximately 2-3 tons of cooling. Shade trees will not exactly help ... I guess you could get about 4 12000 BTU (1 ton) window units, assuming you got that many windows, and then you would technically have N+1 cooling ...
    all server farms should move to the snow belt like me and open some windows

    no cooling costs for 6 months every year. hehe

  16. #16
    Join Date
    Jun 2009
    Location
    Washington
    Posts
    374
    Quote Originally Posted by iloveunicorns View Post
    all server farms should move to the snow belt like me and open some windows

    no cooling costs for 6 months every year. hehe
    Hi, thanks for enlivening my thread!

    Where north are you?

    I found that just opening windows that most generally start 4 and 1/2 up from the floor, even when it is cold outside does not particularly work. If however you post them in a living room where you can open the door, then you got something!

    I use a bedroom and unattached garage shop. But mostly the challenge is found in the house where it is heavily insulated, not in a single sheet unfinished garage.

    My solution:

    Place air vents up the server racks (these server racks are handmade of wood) 2 inch and 4 inch, ten feet long PVC pipes. My experiment included hundreds of drilled holes in the PVC from the ceiling down to 3 feet off the floor. This PVC extends up through the attic past the roof line another 4 feet with a weather top like any pipe for same used for hot water, house piping etc that also extend out the roof.

    And then I cut a 30 inch by 30 inch hole in one end of the room from the floor up to a un-used 34 inch cubed space that once housed the first half-sized water heater long abandoned. After removing said old water-heater (this was not easy!) I placed a 30 inch ceiling fan at 45 degrees where it draws air in through outside wall which opening I enlargened and placed locked grates and air filters. Now it drives cool or cold air to the largest server rack only two feet away from the floor level. Said server rack extends into the room 5 feet long, 2 feet wide and 6 1/2 feet high and sports 54U of servers and a few routers, KVM's, a monitor, key board. Amongst the 54U of servers is 5 arrays which is counted as part of the 54U.

    When I flip the fan on it opens the grated vents and blows the outside cold air straight in through and around the servers where at the other end of the room the PVC pipes collect this air and by convection draws it up and out since it is still 'warmer' and positively ionized in comparison to the outside heavily negatively ionized cool air up some 14 feet from the floor to up over the house.

    More servers, equipment, monitors etc., are running, the more it works.

    Now come next summer, I will place outside air conditioner that will vent under the house through a now abandoned floor vent which is also but two feet from the main server rack.

    I wired in 6 20amp lines all direct. I will put in two more lines for the more fans and air conditioner. (I have them half in but am in no hurry now) And another 30 amps in the room inherited from earlier efforts when it the room was just the computer room.

    So just because it is cold outside will not of itself necessarily solve it unless you post your server rack just inside the front or back door. <smiles>

    A tad bit 'inconvenient' I recon.

    I also 'sealed' the house server room somewhat from the rest of the house, etc, etc.

    This venture is not for the faint of heart.
    Last edited by Mavus; 11-03-2009 at 03:53 AM.
    ABSF
    Arrogant Bastard Server Farm
    Built from scratch Data Center serving
    100 year-old Metaphysical Library

  17. #17
    Join Date
    Oct 2009
    Location
    ZombieLand
    Posts
    205
    i am at erie pa. 6 months of snow. Great cooling.

  18. #18
    Join Date
    Oct 2007
    Location
    Reading, UK
    Posts
    602
    Nice thread, I've nothing to say for once except that it was really interesting. Oh, and how did you come to the name of 'Arrogant Bastard Server Farm'?
    Toby
    Xeo Solutions - Discovering Innovation
    Managed Service Provider & Cloud Hosting | Launching Late August 2019

  19. #19
    Join Date
    Oct 2009
    Location
    ZombieLand
    Posts
    205
    Quick question what kind of internet do you get at your home.

  20. #20
    Join Date
    Jun 2009
    Location
    Washington
    Posts
    374
    Quote Originally Posted by Toby H View Post
    Nice thread, I've nothing to say for once except that it was really interesting. Oh, and how did you come to the name of 'Arrogant Bastard Server Farm'?
    It is my favorite beer. 'Arrogant Bastard Ale' brewed in San Diego. I drink it often and collect the caps... <smiles>

    It captures the spirit of doing something difficult anyway.

    However, the name is relative to me, as it is just a container for my on-line school and digital library which will have each their own M.O. No outside customers are planned at this time. Only hosting concerns I have an interest in, figuratively or literally.
    ABSF
    Arrogant Bastard Server Farm
    Built from scratch Data Center serving
    100 year-old Metaphysical Library

  21. #21
    Join Date
    Jun 2009
    Location
    Washington
    Posts
    374
    Quote Originally Posted by iloveunicorns View Post
    Quick question what kind of internet do you get at your home.
    Comcast Business and Verizon FIOS at about 8 dollars a gig up. You can have mulitiple lines, as this is business zoned and unincorporated. But I do not have immediate plans for having more than 25 up in the first leg of this operation.
    ABSF
    Arrogant Bastard Server Farm
    Built from scratch Data Center serving
    100 year-old Metaphysical Library

  22. #22
    Join Date
    Jun 2009
    Location
    Washington
    Posts
    374
    Quote Originally Posted by iloveunicorns View Post
    i am at erie pa. 6 months of snow. Great cooling.
    Yep, you could pile the snow up just outside the window and post a smaller fan in the sill just fine!!
    ABSF
    Arrogant Bastard Server Farm
    Built from scratch Data Center serving
    100 year-old Metaphysical Library

  23. #23
    Join Date
    Mar 2006
    Location
    Reston, VA
    Posts
    3,131
    If you want cheap power and cooling, go deploy a datacenter in iceland. I know someone who is.
    Yellow Fiber Networks
    http://www.yellowfiber.net : Managed Solutions - Colocation - Network Services IPv4/IPv6
    Ashburn/Denver/NYC/Dallas/Chicago Markets Served zak@yellowfiber.net

  24. #24
    Join Date
    Oct 2009
    Location
    ZombieLand
    Posts
    205
    Quote Originally Posted by Spudstr View Post
    If you want cheap power and cooling, go deploy a datacenter in iceland. I know someone who is.
    But wouldn't the internet and electricity be expensive there.

  25. #25
    Join Date
    Jun 2009
    Location
    Washington
    Posts
    374
    Quote Originally Posted by Spudstr View Post
    If you want cheap power and cooling, go deploy a datacenter in iceland. I know someone who is.
    Here:

    Great cooling
    Cheap Electricity, natural gas as power back-up.
    Great connectivity at resonable pricing.

    ICELAND great cooling for fish? I guess banking flew south just lastly.

    I am here north of Seattle. Electricity is cheap, Boeing and Microsoft are my neighbors and it is constantly cool for 9-10 months through the year. Not often hot or cold for any real length of time.
    ABSF
    Arrogant Bastard Server Farm
    Built from scratch Data Center serving
    100 year-old Metaphysical Library

Page 1 of 4 1234 LastLast

Similar Threads

  1. Need a server building? 1U UK Server Building, Core2/Celeron/Atom and more!!
    By keywolf in forum Other Web Hosting Related Offers
    Replies: 4
    Last Post: 06-19-2009, 04:09 PM
  2. Down on the server farm
    By alteree in forum Web Hosting Lounge
    Replies: 1
    Last Post: 05-27-2008, 01:04 PM
  3. Server Farm
    By reller81 in forum Hosting Security and Technology
    Replies: 0
    Last Post: 11-08-2006, 03:59 PM
  4. Replies: 9
    Last Post: 06-19-2006, 02:16 PM
  5. What Exactly Is A Server Farm?
    By ClusterMania in forum Dedicated Server
    Replies: 6
    Last Post: 11-04-2001, 12:19 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •