Results 1 to 25 of 25
  1. #1

    Advise needed for VPN servers for 10,000 users

    Dear friends,

    I have a need to setup VPN servers that can handle up to 10,000 symultaneous users. Assuming 1Mbps for each that is up to 10Gbps bandwidth!

    Is there anyone out there have experience with such a heavy load of requirement? I really really appreciate some advise :-)

    In my mind I am thinking to setup 10 servers, each can handle up to 1000 users. I assume I need to look for a colocation service that provides 1Gbps unmetered service. Is this realistic?

    Also if you know any good (fast) colocation service in Northern Virgina please let me know.

    Thank you very much!
    James

  2. #2
    Do you already have the software and servers picked out? I really can't see why you wouldn't go with Juniper for this, unless cost is a deciding factor. I'd say it would be good to start testing this out if you are going to use open source software to do this because you may need to tweak various parameters, and document the management of everything.

    I don't understand how this could be a remote access VPN, is this some type of proxy setup? As for the bandwidth that's an awful lot, it's going to be expensive.
    ActiveHost Corporation - Hyper-V, New York Co-location, VPS, Dedicated & Shared Hosting
    Fully Supporting: Windows 2008, ASP.NET 3.5, SQL 2008, Silverlight 3
    14 Years in Business with our own multi-million dollar data center
    www.activehost.comsales@activehost.com
    1-888-500-6799

  3. #3
    Thanks ah-quinn. The server runs proprietary software with proprietary technology. It is not a traditional VPN. I understand the bandwidth requirement is huge, that is why any advise are welcome :-)

    I doubt 1 server is not able to handle 1000 connections (remember traffics are both up and down) even if I have a 1Gbps connection.

  4. #4
    Join Date
    Aug 2003
    Location
    /dev/null
    Posts
    2,132
    You can have multiple 1Gbps connections on a same server also. The problem is how much processing will these thousand users utilize. Will they need processing power local in the server (like remote desktop) or its just proxying?

  5. #5
    Join Date
    Mar 2010
    Location
    Minneapolis
    Posts
    205
    Quote Originally Posted by iptelligent View Post
    You can have multiple 1Gbps connections on a same server also.
    Reference; http://en.wikipedia.org/wiki/Link_aggregation

  6. #6
    Interesting, is it same as having multiple NICs on the computer? I kind of like the idea of multiple NIC (I assume I can modify our program to bind to different NICs for load balancing). It will also give me multiple IP address hopefully.

    On the other hand I always think that bandwidth costs more than machines in the long run, so it may not be of that much value to combine servers into one since I need to pay for bandwidths on each NIC anyway.

  7. #7
    You can load-balance IPSEC based VPNs, and you could route to different IPs on each interface. the bonding is a way of making two switchport work as one at layer2 (ethernet), where traffic to one IP could end up being an ethernet frame sent to NIC1,NIC2, NIC3 etc,.

    Optimizing these data transfers over the network has been done before, companies that make linux based routers, etc. You may want nice NIC cards. Here is an article that has many of the settings if default isn't fast enough:
    "Enabling high speed data transfers from Pittsburgh Supercomputing Center"

    http://www.psc.edu/networking/projects/tcptune/
    ActiveHost Corporation - Hyper-V, New York Co-location, VPS, Dedicated & Shared Hosting
    Fully Supporting: Windows 2008, ASP.NET 3.5, SQL 2008, Silverlight 3
    14 Years in Business with our own multi-million dollar data center
    www.activehost.comsales@activehost.com
    1-888-500-6799

  8. #8
    Join Date
    Aug 2003
    Location
    /dev/null
    Posts
    2,132
    NoviceJ: how are you going to justify the 10k IP addresses? It's a theoretical/rethorical question, as proxies or VPNs aren't "acceptable" justification in the eyes of many RIRs...

  9. #9
    Join Date
    Feb 2010
    Location
    Atlanta, GA
    Posts
    173
    Your biggest obstacle will be getting 10,000 IP numbers. Depending on what these VPN’s will be used for you could assign the users private IP numbers and have them share a gateway IP to the Internet however this will create porting problems.

    Realistically the VPN companies I’ve worked with will only put a max of 500 users per server since your crypto will max out at 500 users… however there is a way to accelerate the AES crypto to 2000 users per server… but that’s a closely guarded secret of the VPN companies I can't talk about here.

  10. #10
    Join Date
    Feb 2010
    Location
    Atlanta, GA
    Posts
    173
    Quote Originally Posted by novicej View Post
    Dear friends,

    Assuming 1Mbps for each that is up to 10Gbps bandwidth!
    With VPN servers you have a connection coming in and one going out per user. If you give each user 1 Mbit they will be making a 1 Mbit connection in, and a 1Mbit connection out... so they will be using twice the bandwidth you assign them... keep this in mind. So now you need 20 Gbit of bandwidth

  11. #11
    Join Date
    Apr 2009
    Location
    Sofia, Bulgaria
    Posts
    200
    OVH have such high BW offers, but I believe it will be better to use multiple servers on multiple pipes.
    Oh, and 1Mbps? Who will use such services. Are you sure there is 10k people on the planet which are happy with such speed?

  12. #12
    Join Date
    Feb 2010
    Location
    Atlanta, GA
    Posts
    173
    Quote Originally Posted by NikolayM View Post
    Are you sure there is 10k people on the planet which are happy with such speed?
    Maybe China ?

  13. #13
    Join Date
    Aug 2003
    Location
    /dev/null
    Posts
    2,132
    Quote Originally Posted by Atlas Global View Post
    With VPN servers you have a connection coming in and one going out per user. If you give each user 1 Mbit they will be making a 1 Mbit connection in, and a 1Mbit connection out... so they will be using twice the bandwidth you assign them... keep this in mind. So now you need 20 Gbit of bandwidth
    No, he needs 10Gbps. Buying per megabit, it's symmetrical. The only method where you are charged IN+OUT is by data transfer (x GB per month).

  14. #14
    Join Date
    Mar 2006
    Location
    Reston, VA
    Posts
    3,131
    Why do you need 1 IP per person? You should be able to NAT whatever users you have per machine to 1 ip. Why be so wasteful? Then again depends on the technology you are using.
    Yellow Fiber Networks
    http://www.yellowfiber.net : Managed Solutions - Colocation - Network Services IPv4/IPv6
    Ashburn/Denver/NYC/Dallas/Chicago Markets Served zak@yellowfiber.net

  15. #15
    Join Date
    Aug 2009
    Location
    Orlando, FL
    Posts
    1,063
    Quote Originally Posted by ah-quinn View Post
    Do you already have the software and servers picked out? I really can't see why you wouldn't go with Juniper for this, unless cost is a deciding factor. I'd say it would be good to start testing this out if you are going to use open source software to do this because you may need to tweak various parameters, and document the management of everything.

    I don't understand how this could be a remote access VPN, is this some type of proxy setup? As for the bandwidth that's an awful lot, it's going to be expensive.
    +1. The SA series is hands down the best VPN device. Like ah-quinn said it is VERY expensive. I have several of them in production and they are rock solid. I was contracting at Disney for a while and they had a cluster of them doing way more than 10,000 users.

  16. #16
    To answer some of the questions: 10,000 users only use private IPs. Also I am hoping 1Mbps is on average under the condition when the server is busy, if there are less user then we allow more bandwidth for each.

    I do have one question though, does a server allow you to have one NIC but with multiple real IP addresses? Like a subnet range of IP addresses.

  17. #17
    Join Date
    Mar 2006
    Location
    Reston, VA
    Posts
    3,131
    Quote Originally Posted by novicej;
    I do have one question though, does a server allow you to have one NIC but with multiple real IP addresses? Like a subnet range of IP addresses.

    Most modern operating systems allow you to alias ips to interfaces, utilize 802.1q vlan tagging, 802.1ad(bonding) and even to source based routing out various network cards.

    So yes. you can basically do whatever you would like
    Yellow Fiber Networks
    http://www.yellowfiber.net : Managed Solutions - Colocation - Network Services IPv4/IPv6
    Ashburn/Denver/NYC/Dallas/Chicago Markets Served zak@yellowfiber.net

  18. #18
    Join Date
    Aug 2006
    Location
    Ashburn VA, San Diego CA
    Posts
    4,615
    Quote Originally Posted by Spudstr View Post
    Why do you need 1 IP per person? You should be able to NAT whatever users you have per machine to 1 ip. Why be so wasteful? Then again depends on the technology you are using.
    Mainly because of torrenting.
    Fast Serv Networks, LLC | AS29889 | DDOS Protected | Managed Cloud, Streaming, Dedicated Servers, Colo by-the-U
    Since 2003 - Ashburn VA + San Diego CA Datacenters

  19. #19
    Join Date
    Mar 2006
    Location
    Reston, VA
    Posts
    3,131
    Quote Originally Posted by FastServ View Post
    Mainly because of torrenting.
    Well. We all know the feelings on that and lets not dig into that subject.
    Yellow Fiber Networks
    http://www.yellowfiber.net : Managed Solutions - Colocation - Network Services IPv4/IPv6
    Ashburn/Denver/NYC/Dallas/Chicago Markets Served zak@yellowfiber.net

  20. #20
    Join Date
    Feb 2010
    Location
    Mumbai
    Posts
    680
    I guess, ask WorldVPN.net admin about this
    Just contact them, I hope they can provide you the suitable details about VPNs
    ¦¦█¦¦ Artnet Datacenter - Poland (Gdańsk & Gdynia) based instant setup express dedicated servers, GPU based servers, colocation & storage VPS

    » E✘OTICVM - Listing exotic VPS locations around the world!

  21. #21
    Join Date
    Sep 2004
    Posts
    1,007
    My ISP, which primarily sells DSL (5Mbps down, 800Kbps up) puts (if memory serves) about 5000-6000 users per GigE (at which point an acceptable amount of leeway for bursts is maintained). Last I heard the users were doing about 0.1Mbps on average, so the average usage would likely be 500-600Mbps, with the rest reserved for bursting. So I wouldn't necessarily try to put 10k users onto a single GigE.

    Do you really need to support 10k users right off the bat? That sounds very expensive. It might be smarter to start small with a scalable solution; low-end dedicated servers or high-end VPS would allow you to scale from one single server on up as required. without blowing huge amounts of money on expensive hardware or 10 gig commits that you may never use.

    Of course, bandwidth tends to be a lot more expensive this way... A truely scalable solution would be something cloud-based (such as Linode), although you're looking at about $64 per symmetrical megabit for Linode due to them being a higher-end provider (but if you're buying lots of bandwidth they'd undoubtedly cut a deal). You could also start with a box from 100TB ($1.25 per symetrical megabit) and scale up by adding additional servers as required.

    I also suspect that it's easier to justify a large number of IPs if they're spread around lots of hardware...

    So, I'll just make sure that it's clear that I'm not an expert in any of this, but rather than take a big risk, if I had to do it, I'd start with a small but scalable platform to avoid huge upfront costs.

  22. #22
    Join Date
    Aug 2003
    Location
    /dev/null
    Posts
    2,132
    Guspaz, take into account also the costs of dedicated IPs on this model of one-address-per-client. There will be a limit where 100TB won't allow him to outgrow a certain number of addresses per server (256) or even in total; nor would announce their own AS and IP blocks (that could not be easily justifiable to ARIN, which would let the annual costs for the IPs drop substantially). 10k addresses require a lot of justification and some paperwork, and in the case of 100TB/SL network, around $10k in monthly fees just for the addresses.
    Last edited by cresci; 04-28-2010 at 12:45 PM. Reason: Pressed Save at the wrong time (touchpad whacky)

  23. #23
    Join Date
    Mar 2010
    Location
    Minneapolis
    Posts
    205
    Quote Originally Posted by iptelligent View Post
    Guspaz, take into account also the costs of dedicated IPs on this model of one-address-per-client. There will be a limit where 100TB won't allow him to outgrow a certain number of addresses per server (256) or even in total; nor would announce their own AS and IP blocks (that could not be easily justifiable to ARIN, which would let the annual costs for the IPs drop substantially). 10k addresses require a lot of justification and some paperwork, and in the case of 100TB/SL network, around $10k in monthly fees just for the addresses.
    This is the important post here.

    Do not go into this thinking the one IP per customer is viable without serious financial and legal backing. With IPv4 depletion the odds of you getting a /18 with a flimsy premise is very low.

  24. #24
    Join Date
    Sep 2004
    Posts
    1,007
    I'm somewhat ignoring the IP issue; the OP may not need dedicated IPs as much as he needs. It's not entirely clear that the OP does need dedicated IPs (he's indicated he wants it, but does he need it?)

    The BitTorrent example was given as to why you can't have 10k users NAT'd to a single IP. Let's not get silly about the uses of BitTorrent and just discuss it for what it is, a single application that makes many outgoing connections, using up a lot of outbound ports on the NAT'd IP.

    The solution seems obvious; use a relatively small pool of IPs for NAT rather than just a single one.

    As for the limit of 256 IPs per server at 100TB (or other providers, I doubt Linode is going to let you put even that many IPs on a single node), it's not entirely clear that this is a problem. The OP wanted to reserve 1Mbps per user, and 256 IPs on a server means he can get (let's say) 255 users on a single box, leading to a cost of $1.78 per user, with 1.25Mbps per user average.

    That sounds pretty decent to me. He can start with one box, scale that up to 255 users, and when he needs more, purchase a second box. Scaling to 10k boxes might get expensive, but he could always transition from dedicated server hosting to colocating some Juniper hardware at some point, and still get the advantages of building his customer base with a low startup cost.

  25. #25
    Quote Originally Posted by Atlas Global View Post
    however there is a way to accelerate the AES crypto to 2000 users per server… but that’s a closely guarded secret of the VPN companies I can't talk about here.
    nics with onboard hardware crypto engines

    sometimes available used on ebay

    also available as crypto blade switches for blade chassis


    /shrug
    edgedirector.com
    managed dns global failover and load balance (gslb)
    exactstate.com
    uptime report for webhostingtalk.com

Similar Threads

  1. For Sale: Large forum : +300,000 posts +1,000 users
    By Klian in forum Other Offers & Requests
    Replies: 14
    Last Post: 08-14-2006, 10:14 AM
  2. Replies: 9
    Last Post: 07-07-2006, 12:41 PM
  3. Replies: 5
    Last Post: 03-24-2006, 12:17 AM
  4. Replies: 7
    Last Post: 03-31-2004, 03:33 PM
  5. Replies: 8
    Last Post: 01-07-2003, 01:55 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •