Results 1 to 20 of 20
  1. #1
    Join Date
    Feb 2011
    Location
    Denver, CO
    Posts
    124

    OnApp Network Isolation

    Has anyone configured network isolation within OnApp? We've been playing around with it in testing and found there is no layer 2 isolation. Meaning one customer can attack another customer's VM. ARP poisoning, even sniffing traffic. When I asked OnApp about this they kicked back and said to use vlans, but switches are basically limited to 1k vlans, maybe 4k depending on how we set it up. So vlans per customer isn't a solution. I believe KVM and Xen both support network isolation.
    Mike Kauspedas - GearHost
    PaaS Cloud for .NET and PHP Developers
    Publish using Git, Visual Studio 2013 or FTP. Signup - No credit card required
    Personal blog www.mikesaysmeh.com

  2. #2
    Join Date
    Aug 2011
    Location
    Dub,Lon,Dal,Chi,NY,LA
    Posts
    1,838
    Why would 1024 Vlans be a limitation? You won't really get close to 1,000 VMs on a single OnApp controller without some effort anyway...
    dediserve www.dediserve.com
    Leading provider of enterprise SSD cloud platforms with 15 clouds in 3 regions
    Dublin, London, Amsterdam, Vienna, Frankfurt, New York, Chicago, Dallas, Los Angeles, Toronto, Singapore, Jakarta, Hong Kong, Sydney

  3. #3
    Join Date
    Feb 2011
    Location
    Denver, CO
    Posts
    124
    That's a limit on the switch and I don't think may hosting companies want to buy a new 10g switch for each CP.

    But back to the crux of the issue, no network isolation in a Cloud?
    Mike Kauspedas - GearHost
    PaaS Cloud for .NET and PHP Developers
    Publish using Git, Visual Studio 2013 or FTP. Signup - No credit card required
    Personal blog www.mikesaysmeh.com

  4. #4
    Join Date
    Aug 2011
    Location
    Dub,Lon,Dal,Chi,NY,LA
    Posts
    1,838
    No network isolation in a rack of dedicated servers? Or a shared colo rack?

    I'm not sure I understand your concern?
    dediserve www.dediserve.com
    Leading provider of enterprise SSD cloud platforms with 15 clouds in 3 regions
    Dublin, London, Amsterdam, Vienna, Frankfurt, New York, Chicago, Dallas, Los Angeles, Toronto, Singapore, Jakarta, Hong Kong, Sydney

  5. #5
    Join Date
    Feb 2011
    Location
    Denver, CO
    Posts
    124
    FYI, I asked OnApp about the VM limit per CP and they said there is no limit beyond the physical hardware required to run the mysql db. they claim to have a customer with 1300 VMs.

    Obviously in a dedicated environment you would use private vlans, and the amount of customers is a LOT smaller.

    I don't understand the response to simply dismiss the issue. Why wouldn't you want to isolate your tenants? If not for security, at least for privacy. Part of the appeal of the cloud is building an environment with heavy automation and lower costs. Think about it, malicious customer signs up in the middle of the night, passes fraud checks, then proceeds to use their server to attack your network. Hundreds of customers are down for an hour or more before you realize what is happening. In our testing we have reproduced this type of attack.
    Mike Kauspedas - GearHost
    PaaS Cloud for .NET and PHP Developers
    Publish using Git, Visual Studio 2013 or FTP. Signup - No credit card required
    Personal blog www.mikesaysmeh.com

  6. #6
    Join Date
    Dec 2000
    Location
    Scotland
    Posts
    134
    Speaking from our experience, we see service providers who are absolutely committed and focused on isolating customers through VLANs, and other service providers who hate VLANs with a passion for various reasons around switch limitations, IP address wastage etc, it does depend greatly on their typical customer base.

    From our point of view we believe customers should be able to use whichever model they are most comfortable with, so alongside our existing public & private VLAN models have built a brand new networking model (using existing, mature & stable routing technologies) we call Public Virtual IP, which enables individual IPs to be allocated to individual VMs without the typical problems of a flat network such as spoofing or sniffing issues, we achieve this through integrating the routing mesh directly into the individual compute nodes to tightly control it.

    Don't want to bore everyone with the technical details of how it works, but is very popular with customers who are not so keen on VLANs and can share more details if needed.

    With the advent of software defined networking (excluding the hype) in general, a variety of companies are looking at these various problems to find new models that don't have the drawbacks of more traditional technologies.


    Regards,

    Tony
    Founder & SVP Product
    Flexiant Ltd
    Simplifying the Cloud - Designed for Service Providers
    http://www.flexiant.com

  7. #7
    Join Date
    Feb 2011
    Location
    Denver, CO
    Posts
    124
    You hit the nail on the head Tony, we don't want to use vlans for those reasons but we need to isolate our tenants. You're claiming your software provides both?
    Mike Kauspedas - GearHost
    PaaS Cloud for .NET and PHP Developers
    Publish using Git, Visual Studio 2013 or FTP. Signup - No credit card required
    Personal blog www.mikesaysmeh.com

  8. #8
    Actually they are limiting the source and destination IPs to the inside bridge interfaces. So a successful ip takeover shall not work - at least for layer3. Also sniffing is limited to layer2 regular broadcast.

    Maybe i will just write an API call that is creating an ACL for the switch or the hypervisor-nodes (mac:ip) and the issue is gone.
    Last edited by ipclear; 01-19-2014 at 05:49 AM.

  9. #9
    In OnApp the anti IP spoofing/theft option can be enabled under Settings > Configuration > Defaults for KVM Hypervisors, and is automatically in place for Xen Hypervisors.

    With this enabled most hosts tend to be happy to then use a shared VLAN for multiple clients VMs, then just add additional private VLANs for those clients who have an actual requirement for this.

  10. #10
    Join Date
    Dec 2000
    Location
    Scotland
    Posts
    134
    Quote Originally Posted by ghMike View Post
    You hit the nail on the head Tony, we don't want to use vlans for those reasons but we need to isolate our tenants. You're claiming your software provides both?
    It does. <<snipped>>

    Regards,

    Tony
    Last edited by Postbox; 01-20-2014 at 07:46 PM.
    Founder & SVP Product
    Flexiant Ltd
    Simplifying the Cloud - Designed for Service Providers
    http://www.flexiant.com

  11. #11
    Join Date
    Feb 2011
    Location
    Denver, CO
    Posts
    124
    <<snipped>>
    Anti-spoofing did resolve the issue with layer2 isolation for us.
    But I really want to learn how to enable this CIM within OnApp. I can follow instructions if there is a doc/KB.
    Last edited by Postbox; 01-20-2014 at 07:49 PM.
    Mike Kauspedas - GearHost
    PaaS Cloud for .NET and PHP Developers
    Publish using Git, Visual Studio 2013 or FTP. Signup - No credit card required
    Personal blog www.mikesaysmeh.com

  12. #12
    Join Date
    Aug 2011
    Location
    Dub,Lon,Dal,Chi,NY,LA
    Posts
    1,838
    CIM is 'default' behaviour in OnApp, you don't need to do anything to make it work, if anything you have to work around it to get certain network behaviors to work (like custom gateways for VPN's, etc)
    dediserve www.dediserve.com
    Leading provider of enterprise SSD cloud platforms with 15 clouds in 3 regions
    Dublin, London, Amsterdam, Vienna, Frankfurt, New York, Chicago, Dallas, Los Angeles, Toronto, Singapore, Jakarta, Hong Kong, Sydney

  13. #13
    Join Date
    Feb 2011
    Location
    Denver, CO
    Posts
    124
    <<snipped>>

    If that were true then there would be layer2 isolation which there was not until we enabled the anti-spoofing beta for KVM.
    Last edited by Postbox; 01-20-2014 at 07:50 PM.
    Mike Kauspedas - GearHost
    PaaS Cloud for .NET and PHP Developers
    Publish using Git, Visual Studio 2013 or FTP. Signup - No credit card required
    Personal blog www.mikesaysmeh.com

  14. #14
    Join Date
    Aug 2011
    Location
    Dub,Lon,Dal,Chi,NY,LA
    Posts
    1,838
    We only use Xen, and OnApp has worked that way out of the box since day 1
    dediserve www.dediserve.com
    Leading provider of enterprise SSD cloud platforms with 15 clouds in 3 regions
    Dublin, London, Amsterdam, Vienna, Frankfurt, New York, Chicago, Dallas, Los Angeles, Toronto, Singapore, Jakarta, Hong Kong, Sydney

  15. #15
    Quote Originally Posted by dediserve View Post
    We only use Xen, and OnApp has worked that way out of the box since day 1
    Yes, its been that way since day one with Xen, but was added as an option for KVM after the initial integration which currently is not enabled by default - but we are currently re-evaluating that.

  16. #16
    Join Date
    Feb 2011
    Location
    Denver, CO
    Posts
    124
    FYI guys, we did the same testing with another provider who uses Xen and were able to reproduce the issue. Spoofing with layer 2 ... spoofing, I think there is some confusion regarding sniffing vs spoofing. I'm guessing they may be on an older version that doesn't support isolation with Xen, or they have it configured in a way to allow it, or Xen does not protect from spoofing.

    To get more technical here is what we did to test:
    - We have one vlan that all virtual machines share. Which is ok according to OnApp's site.
    http://onapp.com/cloud/features/security/
    - Added two new users.
    - Created two new separate networks and separate network zones. (I was first told by OnApp support that separate networks were required to provide isolation.)
    - Create two new virtual machines and assigned to each respective network.
    - Test using tcpdump and ettercap with a linux VM. Send out spoofed ARP packets as MITM with IP forwarding enabled.
    - Target VM in OnApp (the other VM we created) respond to attack.

    Through this test we did determine that there is layer 3 isolation. IE we cannot sniff traffic from one virtual machine to another. But sniffing and spoofing are two separate vulnerabilities.

    I also asked several times regarding the differences between Xen and KVM and why someone would choose one over the other and there was never any mention regarding Xen providing isolation out of the box and KVM not. (It took three different support people responding before someone suggested the anti-spoofing option.)

    Our end goal is simply to provide isolation to virtual servers.
    Mike Kauspedas - GearHost
    PaaS Cloud for .NET and PHP Developers
    Publish using Git, Visual Studio 2013 or FTP. Signup - No credit card required
    Personal blog www.mikesaysmeh.com

  17. #17
    Join Date
    Jul 2010
    Location
    The Netherlands
    Posts
    36
    ghMike your hitting the nail on the right spot. Your saying Xen is porviding it out of the box, where can I find more details about this?

    Thanks
    Serveo | reliable cloud hosting

    Locations NL- Interxion AMS-5, PL- PLIX
    Shared hosting | Reseller hosting | Cloud server | Dedicated server | Rackspace

  18. #18
    Hi Guys

    Without going into deep details, CIM is out isolation layer, because OnApp knows what VM's a user owns and what IP's it has beed assigned we (in the case of Xen for example) set the internal firewalls at the HV level to not allow IP broadcasting that are not assigned to the residing VM and don't allow sniffing of traffic between the VM's

    An example would be that anybody who tried to start an IP that OnApp did not assigned it to them, would be of course able to add the IP. but the HV FW would not allow it to be broadcast.

    Mike, did I saw it correctly on one of our internal tickets this is now working correct for you ? if not feel free to ping me at [email protected]
    Carlos Rego
    OnApp CVO

    The Cloud Engine

  19. #19
    Join Date
    Feb 2011
    Location
    Denver, CO
    Posts
    124
    What we found was an issue with spoofing which was resolved by enabling the anti-spoofing option. OnApp did prevent sniffing. We still believe that Xen allows spoofing and suggest OnApp investigate. You can use the instructions I left to reproduce the test.
    Mike Kauspedas - GearHost
    PaaS Cloud for .NET and PHP Developers
    Publish using Git, Visual Studio 2013 or FTP. Signup - No credit card required
    Personal blog www.mikesaysmeh.com

  20. #20
    Join Date
    Oct 2013
    Posts
    42
    Any more info on this? We're trying to make the decision right now for a new cloud of KVM vs Xen.

Similar Threads

  1. DDOS attack isolation
    By B-H - Bob in forum Employment / Job Offers
    Replies: 1
    Last Post: 09-15-2013, 03:08 PM
  2. [FEATURED] OnApp Cloud: Hardware SAN v OnApp Storage (SANity)?
    By [email protected] in forum Colocation and Data Centers
    Replies: 212
    Last Post: 04-24-2013, 05:59 PM
  3. OnApp Cloud Storage on Gbit network
    By HHKNet in forum Cloud Hosting
    Replies: 67
    Last Post: 02-27-2013, 03:36 AM
  4. Network isolation feature
    By thewebhostingdir in forum VPS Hosting
    Replies: 2
    Last Post: 09-24-2011, 03:34 PM
  5. VPS resource isolation
    By SysAdminMan in forum VPS Hosting
    Replies: 6
    Last Post: 02-19-2008, 12:27 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •