Results 1 to 20 of 20

Hybrid View

  1. #1

    for those looking for good infrastructure solution - PLS READ!!!

    Folks - I have some bad situation and some good.... so, please read. I hope this helps some of you investigating hosting/infrastructure options.

    I want to share with the forum readers a situation I had run into. Having learned so much from this forum, I would like to contribute my experience.

    I am part of a new startup with several internet services under development. We have investigated various options of infrastructure setup in preparation for our new services. Like many, we have the obvious requirements in mind, such as scalability, fault-tolerance, etc... We considered various setups, including dedicated servers with load balancers, SSI such as OpenSSI and Open Mosix, Amazon EC2/S3, and we finally decided to try 3Tera's AppLogic grid as soon as we found out about its availability.

    We had prior experience in large infrastructures that handled financial institution transactions, so we understood some of the pitfalls/complexities involved in ongoing operations. So, a main concern we had was, we wanted something that can help us use our time and resources efficiently, so that we don't spend a lot of time in maintenance and operations.

    We wanted ways that can keep us focused on working on our business, not battling with IT operations. When we took a look at 3Tera's product, we thought that it provided us with the phenomenal capability to easily scale, and allow us to have an infrastructure set up with high level of hardware utilization with minimal waste.

    So, very quickly, we committed and signed up with a service provider that supported 3Tera's AppLogic. We were happy with what we saw, and basically considered the infrastructure decision as a closed issue.

    Well..... not too long after we committed, we got a huge surprise! We received a notice from the service provider that they can no longer support the grid service, and we were given several days to migrate whatever we had off of the grid onto somewhere else. Luckily, we were not yet in production mode, but.... we had to first get our test environment off and figure out what infrastructure to go with and where/how to move. And we had LESS THAN 4 days to do all that!!!!

    Our grid app was made to have NAS, ruby on rails with lighttpd, load balancers, firewall, mail gateway, log servers and mysql databases. Even though we also have our own rack servers on hand ready to ship and deploy to whatever datacenter, the thought of trying to migrate all those app functions to dedicated servers or private racks with our own servers....., we didn't think that we could do it in 2-3 days.

    I quickly contacted 3Tera, and explained the situation we were facing. Although I didn't have a direct relationship with 3Tera, the 3Tera team immediately assured me that they will do whatever they can to help. At that point, I was considering looking at alternative solutions again. But.... to us, the 3Tera AppLogic is truly an elegant solution, and the level of attention from 3Tera convinced me to give it another shot and see what happens.

    Quickly, 3Tera team was able to put me in contact with Layered Technologies, and explained to me how Layered Tech has been committed in ramping up their learning and support of 3Tera's AppLogic grid product.

    By the time the first few conversations were over with 3Tera and Layered Tech, Layered Tech built up a new private grid for us to migrate to. Both 3Tera and Layered Tech teams offered to migrate for us, but I wanted to see if 3Tera's product is really as good as they claim....so, I tried to migrate our grid app myself.

    The migration turned out to be an extremely simple process, that basically involved issuing a migrate command on the old grid, instructing it to migrate the app to the new grid. The command took some time to finish executing, as it had to move quite a bit of data. But once completed, all we had to do was reconfigure the app on the new grid with new ip addresses for the app, the gateway, netmask and dns ip address, which only took a few minutes. Amazingly, we were then able to fire up our grid app on the new grid! No hardware issues to deal with what-so-ever!

    The whole situation all happened within a bit over 2 days elapsed time. I'm talking about everything including :
    - start of contact with 3Tera,
    - thru Layered Tech building up new grid,
    - to complete migration of grid app from old grid to new grid that included loadbalancers, NAS, app software, firewall, mail gateways, mysql, etc....

    Everything migrated from 1 set of hardware in one data center to a newly built grid hardware in another data center all around 2 days elapsed time! Total time I spent on dealing with this whole situation was only a few hours!

    Throughout that time of migration, both 3Tera and Layered Tech took the initiative to follow closely and provide answers to questions. They treated me as if I had been a customer for them for years! It's like the attention and services I get at my Mercedes dealership!

    I'm absolutely amazed and really appreciate the level of support that both 3Tera and Layered Technologies provided me. They did not have to be involved to help me, but they chose to and were able to beautifully resolve my crisis. Not only they were able to come to my rescue, but they were able to turn a bad situation around for me in what I believe within an impossible timeframe. To me, the 3Tera product is phenomenal, and the people behind both 3Tera and Layered Technologies showed me how great these two companies are to their customers.


    For selfish reason, I wish these two companies their long term success, because I'm staying with them, and investing our future in them. I for one have no desire to battle hardware and operational issues. I want my team to focus on developing our business, and I believe these two companies, 3Tera and Layered Technologies are key components to our solution to help us maintain our focus. For those of you that are looking for a solution to your web business - I hope this helps you in saving you some time and energy, and allows you to focus on what's most important - developing and operating your business.

    I'm open to PM. Thanks for reading, and good luck to you!


  2. #2
    Join Date
    Feb 2007
    Posts
    70
    I know this is going to sound veeeryy noob.. but what is 3Tera?

    nice story.. happy ending... glad to know that Layered Tech is willing to help when necessary as I am planning on getting a server from them..

    c'ya

  3. #3
    Hi Malware - please check out 3tera.com - they make the grid software that powers thegridlayer.com , and thegridlayer.com is part of Layered Tech.

    Companies are nothing but it's people...and that's why I wanted to share my good findings with others like you.

    Best to you!

  4. #4
    Join Date
    Feb 2007
    Posts
    70
    I did check 3Tera.com and I got out of there even more confused.. =/
    From what I think I understood, is it a cluster manager?
    Thanks and sorry about going off your topic.. =/

  5. #5
    Join Date
    Apr 2002
    Location
    Seattle, WA
    Posts
    955
    Interesting for a new person to on their first post worship LT.
    I <3 Linux Clusters

  6. #6
    Hi x86brandon - sorry if it came across as worship, that's not my intent. This is my first time posting into the forum.... after years of absorbing info, I just hoped to contribute back with what I thought was a remarkable experience. Those 2 companies saved me a LOT of headaches and time. The result of the experience was so positive, I got the urge to write in the middle of the night and share!


    Hi Malware,

    I hope I can help to clarify. Glad to be of assistance if possible.

    The best way of explaining it is, I think, describing it from a usage perspective. The system allows a system user to define/create the logical representation of the application environments that is needed, and removes the physical complexities that would typically be needed for that application. It also makes it possible to have high usage density of hardware by virtualization.

    So, let's say that I have one or multiple applications that I need to put on servers. One application may just be a simple web server serving static contents, and another application may be a bit more complex where I have a load balancer fronting 8 web servers with a dedicated db server. If I deploy that on physical boxes, I can have possibly 11 physical boxes, or I can deploy onto something like vps, and get 11 slices. The drawback with each can include cost and difficulty in scalability.

    Let's say that I now want a testing environment to be similar to what I will have in production. Now, one way to do that is to duplicate whatever I have planned for in production. So, either get another 11 physical boxes or get another 11 vps.

    With this 3Tera AppLogic/GridLayer, they provide me with an alternate solution. I can start with x number of physical servers that forms this grid. And using a web browser accessing an interface that's similar to using Visio, I can define all my application needs (the production environment planned, which is the 11 logical boxes), and the grid software will create those logical boxes forming my application. I can then ssh into each one of those logical boxes with full root access, as if they were completely separated.

    Now, I can take this first app, and duplicate it, and have the duplicate become my test app. Then, in total, I now have 22 separate virtual servers that I can ssh into.

    If I have more capacity left over on the grid, I can put on even more apps. If I need to add more capacity (cpu, memory, storage), that's possible too (and possible to do that without having to touch any of my codes).

    The reason why I would want to do this is :

    1 - easy scalability

    2 - tremendous flexibility in usage

    3 - easy manageability

    4 - high density of resource usage. if I deploy many racked servers or lease dedicated servers, it's most likely that many of the physical boxes will be under utilized at times, and difficult to maximize the use of all my hardwares. This grid allows very high density to be possible without sacrificing my apps.

    5 - I don't want to have to worry about hardware. In my case, I could have had major headaches moving from one data center to another. I was able to move my apps with a total of few hours of my time (and actually a few minutes of work). I don't know of any other solutions that gives me that kind of capability. I can envision that when it comes time to hardware upgrades, I'll be able to get my apps over to new platforms without major hassle.

    Anything that don't give me headaches and saves me time and money gets my vote! Hope this helps. Let me know if you need more clarifications.

  7. #7
    Join Date
    Apr 2002
    Location
    Seattle, WA
    Posts
    955
    One thing you will notice is there is TON's of overhead with 3tera.

    I did a small deployment for one of my clusters and was not happy with performance. I moved everything onto physical servers except for a couple database servers which stay on the grid, because MySQL clusters are a pain.

    What it amounted to is 5 servers choked doing what 3 do quite well.
    I <3 Linux Clusters

  8. #8
    Join Date
    Nov 2006
    Location
    Southern California
    Posts
    173
    Brandon, how come you left your database servers on the grid if there was TON of overhead? The database tends to be the most sensitive part of the application... What kind of app were you running?

  9. #9
    Brendan - sounds like you were able to find benefit with the grid too.

    Regarding performance - we mainly focus on app tuning from user perspective. We focus on making functions/interfaces as easy and quick to use as we can from user view. Our entire app is dynamic, but we carefully analyzed and use various techniques such as fragment caching on server side to help with performance improvements, and have been able in some cases to gain a 10 fold improvement, cutting some transactions down to a few milliseconds.

    We've done tests isolating variables, so that it would narrow down to strictly the app, and I would say that most often, it's the way the app is written that should be checked out. My dev team writes codes specifically looking for bottlenecks in the app.

    DB performance has not been a problem for us during our tests. We also use ip data, which as you know have millions of records of data to be looked up, but that has been handled with ease in our tests. We also do image scan on pixel by pixel basis in real time, but we developed using lower-level language to handle that. At the end, it's all about balancing needs.

    Given that access to web apps contains many variables that are out of our control (user may be on slow links as an example), we figured that instead of focusing on optimal hardware configs on our side, we would focus on the user experience. Since hardwares constantly improves and gets speedier, I want to make sure that we can quickly move and take advantage of them when the time comes. Apps and hardwares in general have very short life cycles, so we look for ways to quickly adapt over time with minimal hassles.

  10. #10
    Join Date
    Apr 2002
    Location
    Seattle, WA
    Posts
    955
    MySQL is difficult to get on a cluster, so even with the overhead, it still works better than a master/slave setup.

    I have a number of adult sites, adult ad farms, and a bunch of forums I host for people.. Pretty intense database stuff.

    The ad software does quite a bit of read/writes to the database, since there is some geo ip stuff going on there too. So the typical answer of 1 server to write to replicating to X slaves that are load balanced reads... doesn't work. I use that for dynamic image serving and content serving though.
    I <3 Linux Clusters

  11. #11
    Join Date
    Apr 2002
    Location
    Seattle, WA
    Posts
    955
    Well, say what you wish, but it doesn't cut the mustard with fairly high traffic adult services.

    4 physically different servers handled way more than 4 virtual servers on the grid.
    I <3 Linux Clusters

  12. #12
    Join Date
    Nov 2006
    Location
    Southern California
    Posts
    173
    Guys, you are obviously in different positions here. Perfmon has control over the code and the architecture of the application. In this sense, my experience is very similar to his - most bottlenecks are in the app. I would even go further and say that most of the time it is the structural problems (e.g. things waiting on other things, or one process starving an unrelated process for memory, etc.), rather than the code itself, that cause significant performance issues.

    Perfmon, you should recognize that unlike you, Brandon has very little control over both the code and the structure of what he is running - most of the time, it is probably other people's sites anyway. So it is likely that his hands are pretty tied.

    Brandon, I am very curious about the bottlenecks you have encountered. I have no doubt that they are real, and would like to hear more about them so that we can tune AppLogic better. Would you consider spending some time on PM or email/phone with me on this next week?

  13. #13
    Join Date
    Apr 2002
    Location
    Seattle, WA
    Posts
    955
    I am able to speak at any time, that is fine.

    I do have a certain amount of control over the app on the web side, and most of it has to do with threading. I have tried several things like running one Apache instance on one environment on the grid, as well as several. It just seems like there are issues with threading.

    What I am doing slowly, is converting from the 4 servers as DB grid that are high end machines, to smaller machines, without raid, and using a SAN.
    I <3 Linux Clusters

  14. #14
    Join Date
    Mar 2004
    Location
    Singapore
    Posts
    6,990
    Thanks for sharing with us your experience. Do post here often with the updates you have on the infrastructure. This would be a great learning experience for us and many others on the forum.

    Quote Originally Posted by perfmon View Post
    Folks - I have some bad situation and some good.... so, please read. I hope this helps some of you investigating hosting/infrastructure options.

    I want to share with the forum readers a situation I had run into. Having learned so much from this forum, I would like to contribute my experience.

    I am part of a new startup with several internet services under development. We have investigated various options of infrastructure setup in preparation for our new services. Like many, we have the obvious requirements in mind, such as scalability, fault-tolerance, etc... We considered various setups, including dedicated servers with load balancers, SSI such as OpenSSI and Open Mosix, Amazon EC2/S3, and we finally decided to try 3Tera's AppLogic grid as soon as we found out about its availability.

    We had prior experience in large infrastructures that handled financial institution transactions, so we understood some of the pitfalls/complexities involved in ongoing operations. So, a main concern we had was, we wanted something that can help us use our time and resources efficiently, so that we don't spend a lot of time in maintenance and operations.

    We wanted ways that can keep us focused on working on our business, not battling with IT operations. When we took a look at 3Tera's product, we thought that it provided us with the phenomenal capability to easily scale, and allow us to have an infrastructure set up with high level of hardware utilization with minimal waste.

    So, very quickly, we committed and signed up with a service provider that supported 3Tera's AppLogic. We were happy with what we saw, and basically considered the infrastructure decision as a closed issue.

    Well..... not too long after we committed, we got a huge surprise! We received a notice from the service provider that they can no longer support the grid service, and we were given several days to migrate whatever we had off of the grid onto somewhere else. Luckily, we were not yet in production mode, but.... we had to first get our test environment off and figure out what infrastructure to go with and where/how to move. And we had LESS THAN 4 days to do all that!!!!

    Our grid app was made to have NAS, ruby on rails with lighttpd, load balancers, firewall, mail gateway, log servers and mysql databases. Even though we also have our own rack servers on hand ready to ship and deploy to whatever datacenter, the thought of trying to migrate all those app functions to dedicated servers or private racks with our own servers....., we didn't think that we could do it in 2-3 days.

    I quickly contacted 3Tera, and explained the situation we were facing. Although I didn't have a direct relationship with 3Tera, the 3Tera team immediately assured me that they will do whatever they can to help. At that point, I was considering looking at alternative solutions again. But.... to us, the 3Tera AppLogic is truly an elegant solution, and the level of attention from 3Tera convinced me to give it another shot and see what happens.

    Quickly, 3Tera team was able to put me in contact with Layered Technologies, and explained to me how Layered Tech has been committed in ramping up their learning and support of 3Tera's AppLogic grid product.

    By the time the first few conversations were over with 3Tera and Layered Tech, Layered Tech built up a new private grid for us to migrate to. Both 3Tera and Layered Tech teams offered to migrate for us, but I wanted to see if 3Tera's product is really as good as they claim....so, I tried to migrate our grid app myself.

    The migration turned out to be an extremely simple process, that basically involved issuing a migrate command on the old grid, instructing it to migrate the app to the new grid. The command took some time to finish executing, as it had to move quite a bit of data. But once completed, all we had to do was reconfigure the app on the new grid with new ip addresses for the app, the gateway, netmask and dns ip address, which only took a few minutes. Amazingly, we were then able to fire up our grid app on the new grid! No hardware issues to deal with what-so-ever!

    The whole situation all happened within a bit over 2 days elapsed time. I'm talking about everything including :
    - start of contact with 3Tera,
    - thru Layered Tech building up new grid,
    - to complete migration of grid app from old grid to new grid that included loadbalancers, NAS, app software, firewall, mail gateways, mysql, etc....

    Everything migrated from 1 set of hardware in one data center to a newly built grid hardware in another data center all around 2 days elapsed time! Total time I spent on dealing with this whole situation was only a few hours!

    Throughout that time of migration, both 3Tera and Layered Tech took the initiative to follow closely and provide answers to questions. They treated me as if I had been a customer for them for years! It's like the attention and services I get at my Mercedes dealership!

    I'm absolutely amazed and really appreciate the level of support that both 3Tera and Layered Technologies provided me. They did not have to be involved to help me, but they chose to and were able to beautifully resolve my crisis. Not only they were able to come to my rescue, but they were able to turn a bad situation around for me in what I believe within an impossible timeframe. To me, the 3Tera product is phenomenal, and the people behind both 3Tera and Layered Technologies showed me how great these two companies are to their customers.


    For selfish reason, I wish these two companies their long term success, because I'm staying with them, and investing our future in them. I for one have no desire to battle hardware and operational issues. I want my team to focus on developing our business, and I believe these two companies, 3Tera and Layered Technologies are key components to our solution to help us maintain our focus. For those of you that are looking for a solution to your web business - I hope this helps you in saving you some time and energy, and allows you to focus on what's most important - developing and operating your business.

    I'm open to PM. Thanks for reading, and good luck to you!


  15. #15
    Join Date
    Jun 2002
    Location
    PA, USA
    Posts
    5,143
    We are looking for similar solution, so thank you for the post.
    Fluid Hosting, LLC - Enterprise Cloud Infrastructure: Cloud Shared and Reseller, Cloud VPS, and Cloud Hybrid Server

  16. #16
    Join Date
    Aug 2001
    Location
    San Francisco, CA
    Posts
    193
    Unless I see someone using 3tera for their web2.0 startup, i mean a real vc funded start-up, if you know what i'm talking about, then i might just try it. that means it's been proven and tested to handle millions of page loads a day. in the meantime, entrepreneurs looking for a scalable infrastructure can check out Amazon s3 and Ec2.

  17. #17
    Join Date
    Nov 2006
    Location
    Southern California
    Posts
    173
    Quote Originally Posted by berlin View Post
    Unless I see someone using 3tera for their web2.0 startup, i mean a real vc funded start-up, if you know what i'm talking about, then i might just try it. that means it's been proven and tested to handle millions of page loads a day. in the meantime, entrepreneurs looking for a scalable infrastructure can check out Amazon s3 and Ec2.
    Berlin,

    How about a $30B a year enteprise deploying AppLogic? Would that satisfy your exacting criteria? Stay tuned...

    In the meantime, you are offering to people to check out something that is still in Beta and compare with an award-winning product.

  18. #18
    Join Date
    Feb 2004
    Posts
    600
    Quote Originally Posted by VladMiloushev View Post
    Berlin,

    How about a $30B a year enteprise deploying AppLogic? Would that satisfy your exacting criteria? Stay tuned...

    In the meantime, you are offering to people to check out something that is still in Beta and compare with an award-winning product.
    It would depend on what it's being used for and why.
    I'm pretty sure Microsoft has a Celeron 800mhz processor somewhere... that doesn't mean Microsoft is suggesting everyone switch to Celeron 800mhz processors
    Dating Revolution Method - Book on how to meet and attract women

  19. #19
    Join Date
    Nov 2006
    Location
    Southern California
    Posts
    173
    Ewhost, thanks for the complement. I would much rather sell Toyotas by the shipload to Hertz than a few Ferraris to rich kids. Do the math

  20. #20
    Join Date
    Nov 2006
    Location
    Southern California
    Posts
    173

    BT taps 3Tera for Utility Computing

    Quote Originally Posted by ewhost View Post
    It would depend on what it's being used for and why.
    I'm pretty sure Microsoft has a Celeron 800mhz processor somewhere... that doesn't mean Microsoft is suggesting everyone switch to Celeron 800mhz processors
    Here is the deal I was mentioning - British Telecom is building a utility computing service using AppLogic:

    http://www.gridcomputingplanet.com/article.php/3684196

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •