Web Hosting Talk : Web Hosting Main Forums : Colocation and Data Centers : Advice needed - Switch configuration

[pentiumone133]

Hi again friends. The WHT community has been extremely helpful so far in regards to several of my prior threads, and I hope to pick your brains again for what will likely not be the last time.

Firstly, I will reference my previous thread here and again commend the community for making a great recommendation for me as to what network hardware will fulfill my needs for a small colocation project. I chose a used cisco 3550 from ebay. For the sake of anyone reading this without first reading my past thread, I will summarize what I am doing.

Some developer friends and I who all currently have our own dedicated servers or colocated servers mostly at other facilities realized that we could save some money if we pooled our resources and got a large amount of space somewhere. We are all fairly seasoned linux admin and what not but the part that continues to plague us is the networking aspect of how we will have this all setup. Basically what we want is to have a setup similar to how it would be at a commercial colocation or dedicated server facility. I can google my way through how to perform the actual implementation, however I wanted to verify that what I'm doing is correct. This is in a nutshell what I think I have to do to the switch to achieve the desired results.

Each server has 2 network interfaces, as well as 1 out of band management card providing IPKVM, etc. 1 interface will be used for the public facing internet, with several IP addresses assigned to the interface within the OS. These public connections need to be vlanned on the switch, into seprate vlans in order to segregate the machines from eachother. Each of these ports will be rate limited to a portion of the bandwidth provided to us by the provider. The second interface on each system will go un-used.

The management cards will all be vlanned together, along with some method of VPN-ing into this VLAN. We will likely dedicate an older p3 system for specifically this purpose.

The switch I have has 2 upstream ports for GBIC devices. I have one of these ports occupied by a copper rj45 gbic.

I need some clarification on the following points:

* Is it best to connect the ethernet drop from my provider to the copper gbic on the switch? This makes sense to me, but I'm trying to understand the significance of these 2 ports when compared to the rest of the interfaces on the switch.

* If I vlan each machine onto its own vlan and thus it's own port, wont it only be able to communicate with itself? How do I make the port which my providers ethernet drop is connected to a member of multiple vlans? Is this best practice? What is the proper way of ensuring that each of these machines can all talk to the internet while remaining isolated from each other?

* What will I need to ask my provider to do on their end to accommodate such a setup? I've read about vlan trunking a little bit. Is this what I'm looking for or am I barking up the wrong tree

* How do I restrict what IP addresses specific ports can assign to the machines attached to them. For example how can I make sure, if we all have 5 public IP addresses per system, that my friend cannot accidentally assign one of my IP addresses to his machine. Is this even possible? I have read that it is possible and one one of the main reasons I sprung for the 3550 over other models.

Is there anything I am missing or overlooking with what how I am envisioning this to be setup? Like I said, our goal is to mimic the setup of a (good) dedicated server or colo provider.

Like I said, I'm not looking for specific command by command instructions on how to set this up. I'm perfectly capable of googling around to find my answers, I just need to know what to learn how to do and to verify that I'm looking for the right things.

Any input on any aspect of what I've asked so far is welcome and encouraged. Thankyou for reading my essay of a post and I look forward to the discussion to follow.

Will

[leeware]

(1) Connecting to the provider: It really doesn't matter but you could and this is what I would do because it would at a minimum establish that the port is used for a special connection i.e. uplink to the outside world among other things.

(2) VLANs are to switches are what VPSes is to a hardware server. It basically lets you take a physical switch an create smaller switches inside. Personally, I don't think is is necessary to create such a complicated configuration for your particular use case. However, if you wish to create a configuration that makes each server think that it is the only one connected to the switch + the uplink your proposed configuration would work. i.e. make the uplink part of each of the VLANs. However, this configuration to me seems rather pointless as a switch is not a HUB.


(3) Since all of the VLANs are inside of your switch you shouldn't need to do anything with your provider to get this to work. The VLANs are only relevant for routing frames on your switch and not on the rest of the network. (see my comments about the configuration being pointless.) Typically, you want to create VLANs to separate IP space onto their own Layer2 network. This is an alternative to doing overlays where you have one physical/logical layer2 setup and a bunch of layer3 sub-nets riding on top of that infrastructure. This can be problematic in non-secure environments which is something that you have probably heard about.

(4) Considering that your IP addresses are likely all going to be from the same block e.g. 192.168.1.100, 192.168.1.101, 192.168.1.103 etc. Your configuration is going to be unnecessarily complicated for reasons that don't have clear benefits but it is certainly possible. Furthermore, for this reason if one of your handful of users assigns a duplicate IP address from the allocation out of the block something is simply not going to work. There is no fool proof way of avoiding this.

(5)(good) is subjective and relative. For example, I run a variety of configurations but those configurations are based on specific requirements such as: (a) customers might have systems scattered across 8 sites but want all servers to appear on a private layer2 network (802.1q VLAN)

(b) customers have different IP networks with routers and want the layer2 setup to force a particular logical structure at layer2 (port-based-vlans)


(c) customers have no specific requirements i.e. mainly stand-alone-machines where the traffic is basically server-to-internet (overlay)


Hope this helps.

[BradQ]

Originally Posted by pentiumone133

* Is it best to connect the ethernet drop from my provider to the copper gbic on the switch? This makes sense to me, but I'm trying to understand the significance of these 2 ports when compared to the rest of the interfaces on the switch.

Your GBIC is, as the name would suggest, gigabit. The specialized port is connected to more switching fabric, as well as allowing other interfaces like fiber. There's no reason to use it to "differentiate" itself and I'd advise not using it for its own sake if the speed difference weren't there.

Originally Posted by pentiumone133

* If I vlan each machine onto its own vlan and thus it's own port, wont it only be able to communicate with itself? How do I make the port which my providers ethernet drop is connected to a member of multiple vlans? Is this best practice? What is the proper way of ensuring that each of these machines can all talk to the internet while remaining isolated from each other?

Your device is a layer 3 device to do IP-level routing. VLANs can communicate just as your regular server can communicate with the Internet, through its gateway (in this case, likely your switch)

Originally Posted by pentiumone133

* What will I need to ask my provider to do on their end to accommodate such a setup? I've read about vlan trunking a little bit. Is this what I'm looking for or am I barking up the wrong tree

No, unless you're using it as a strictly layer 2 device. This is the whole point of having a router.

Originally Posted by pentiumone133

* How do I restrict what IP addresses specific ports can assign to the machines attached to them. For example how can I make sure, if we all have 5 public IP addresses per system, that my friend cannot accidentally assign one of my IP addresses to his machine. Is this even possible? I have read that it is possible and one one of the main reasons I sprung for the 3550 over other models.

This is the whole point of a virtual LAN, and you'll be using your router to assign IP addresses exactly as such.

Originally Posted by pentiumone133

Is there anything I am missing or overlooking with what how I am envisioning this to be setup? Like I said, our goal is to mimic the setup of a (good) dedicated server or colo provider.

Yes. That you definitely need someone at least somewhat experienced with layer 2/layer 3 issues. It won't come at great cost, but you need someone to talk about if you're planning on doing your own routing apart from the WHT community. Get a CCNA guide (the material is not hard) or hire someone; you'll need it.

[BradQ]

Originally Posted by leeware

(4) Considering that your IP addresses are likely all going to be from the same block e.g. 192.168.1.100, 192.168.1.101, 192.168.1.103 etc. Your configuration is going to be unnecessarily complicated for reasons that don't have clear benefits but it is certainly possible

I'm not sure what this has to do with OP's question, and it seems to contradict the instructions you gave presuming he's doing layer 3 routing. The answer to the unasked question you answered is that he'll need a /30 at a minimum, with 3 IPs dedicated to routing overhead for any VLAN. This certainly isn't "unnecessarily complicated" if a user only has a single IP but would prefer not to be ARPed over by the adjacent customer who thinks he can get additional IPs by just binding more in his range.

There absolutely is a "foolproof way of avoiding this", it's creating virtual LANs in which others' IPs aren't routable to you. It's also exactly why he was advised to buy this switch.

[leeware]

@BradQ

A lot of what will happen in terms of his routing options / choices will have to do with his actual IP allocation. Maybe they will give him a /30 or they will give him a larger block/mask and gateway for which he would need to setup the IP routing to address the concerns he has but this is currently an unknown at least to me. As for the concept of inter VLAN routing, Perhaps I misread his question or concerns because I was speaking more to a situation where someone within his group, within his properly routed sub-nets would attempt to assign IP addresses that were already in use.

[BradQ]

Originally Posted by leeware

@BradQ

A lot of what will happen in terms of his routing options / choices will have to do with his actual IP allocation. Maybe they will give him a /30 or they will give him a larger block/mask and gateway for which he would need to setup the IP routing to address the concerns he has but this is currently an unknown at least to me. As for the concept of inter VLAN routing, Perhaps I misread his question or concerns because I was speaking more to a situation where someone within his group, within his properly routed sub-nets would attempt to assign IP addresses that were already in use.

Perhaps it's my fault, but this makes no sense to me. No one with a 3550 in-rack will receive a /30 as their sole allocation (unless you want to have a single routed IP; use it carefully!). I don't understand anything you're saying about IP continuity; you're far more likely to receive a contiguous netblock, sure. That doesn't really affect a bit how he VLANs nor does it make wishing to create and modify them on the fly "unnecessarily complicated".

And the situation you pose is simple. If servers on the same VLAN are ARPing over one another, they should probably be segregated into different VLANs...