Redundant Networking Shenanigans

[ProfHawking]

I need a little advice from any network wizards please!!
I'm setting up some new servers and hoping to make things more resilient than the current set up, by using two switches instead of one.
I need a crash course in redundant networks, or at least some pointers in the right direction. I've been unable to find anything via google that's helped me so far.

The plan I have in my head is to set up something like this:



The router runs MicroTek RouterOS, and has plenty of network interfaces. (Its highly configurable, but a bit complicated)
The switches we have to use are two Netgear GS116E 's
The new server (and 4 others we hope to set up the same way) all have at least 2 network interfaces.

We have from the datacenter a single 1000mb link, so i know this is never going to be truly redundant, but its a start.
They supply us with a /29 subnet for network communication, and a /28 subnet of public IPs.

At present, our servers each have a single NIC connected to a switch, and a single IP assigned (one of the public IPs).
Each single public IP in use has to remain pointed at server is it currently assigned to.


So, how can I give each server two IPs (not public) and have the public IP it was using route to the server down either connection (whichever is connected ok).

Any help would be massively appreciated

[buzzmong]

I think this is a question for Wiggy and the other CISCO quallied chaps isn't it?

I'd also say you need to link the two switches directly as well. Also see: Fully Connected Mesh topology.

[fabyak]

Multihoming is your friend. What are your servers running?

[ProfHawking]

I have been researching Spanning Tree protocol, i will add mesh & multihoming to my list!
All of the servers are running Windows Server 2008.

The thing i'm worried about with STP is the switches, they say "Loop detection" but I'm as of yet dunno if that is true STP or not...

[ProfHawking]

Ok, netgear say that it although the switches do Loop bocking and broadcast storm prevention, its not "true" STP, and therefore may not quite be what we are after.
So its either splash cash on posh switches, or simplify it somehow

[FatherJack]

I'm not sure the configuration you've drawn there gives you any extra resiliency since both switches go back to the same router, infact it's probably less resilient than two cables going direct to the router, as you have two extra switches that can fail.

It depends what you are guarding against. If at least one switch has to be there because that's how your network is wired, then this configuration will guard against that switch, the NICs or the cables failing. However, unless those switches are independently-powered, it won't guard against power failure at the switches location. Do your switches have a history of failing? Do people sometimes pull the wires out?

Also, in order to use two NICs with one IP, you need to do network adapter teaming on the server. This has a bit of a history of being unreliable, sometimes getting stuck in a loop, where both NICs constantly fail over to the other one, so again can actually decrease your reliability. One way of avoiding this is to connect up and configure both NICs separately, then if the primary one fails, you just manually reconfigure the secondary one remotely and without having to open the server.

The trouble with the resiliency route is that you can go insane and spend fortunes thinking of every little thing that can go wrong - have clustered servers connected to distributed SANs with their configuration backed up over various sites. Then some fool makes a change that's replicated across the whole system and it all goes wrong and you have to justify why your 'resilient' system couldn't cope.

Also spanning tree is used to find fast routes through complicated networks, which isn't what you've drawn here. Each route is explicit and should probably be hard-coded the same way it is hard-wired - if you connected the two switches and told spanning tree to "work it out" - that just adds another layer of complexity, and possible failure.

[ProfHawking]

That is a simplified diagram of what we have, but on the whole, we're now going to go down the "keep it simple stupid" route.
I've asked this question to a few people, and you know what, most come back with something similar to what you've said FJ.

We're going to put two switches in the rack, same config etc. Then if one fails, then we get on site tech's to plug everything into the backup switch. Not exactly a smooth transition, but quick ish, and non of the extra hassle of doing a proper resilient network. (When there isn't the budget to do it properly!)

Thats the basic problem. If reliability was everything, then there would be diverse power feeds, two independent network uplinks, two routers, clustered servers, fancy networking using BGP or something.. I dunno. But anyway, its all overkill and there is no budget for any of that with this project. Its not worth complicating it for the tiny weeny chance that a switch fails.

[buzzmong]

We're going to put two switches in the rack, same config etc. Then if one fails, then we get on site tech's to plug everything into the backup switch. Not exactly a smooth transition, but quick ish, and non of the extra hassle of doing a proper resilient network. (When there isn't the budget to do it properly!)

To make it simpler and require no plugging or unplugging of cables and being able to be done at the flick of a switch, you should look into an AB switch, something like this one (not vouching for it, found via a quick and dirty Google).

Plug your server into the back (input), then the network switches into the A or B on the front. Flick of a button to decide which switch is recieving the input from the back*.

They seem to be called Protection Switches, might be worth looking at if you suspect it's not going to be a blue moon event.


*Technically that seems to work the other way around, with front panel being inputs and back being out. Shouldn't mean it can't work opposite to what was intended, after all, data flows both ways under normal operation.

[FatherJack]

We're going to put two switches in the rack, same config etc. Then if one fails, then we get on site tech's to plug everything into the backup switch. Not exactly a smooth transition, but quick ish, and non of the extra hassle of doing a proper resilient network. (When there isn't the budget to do it properly!)

Sometimes simple is best, when we installed the email system at the university, we had all the bells and whistles - SANs replicated across campus and a server farm of 30-odd servers just for one Exchange installation, everything RAID, with even duplicate trays of RAIDs in the SAN in case of tray failure, maximised spindles for performance and disk-to-disk backup, off-site mirroring and tape-streaming - oh and a backup datacentre/SAN on call just in case.
But, for the heartbeat monitors for the cluster (extra NICs connected to each other with local, off-the shelf switches - hardly any traffic, no need to spend big) we just bought twice the number of switches and put a spare one next to each of them.

That switch Buzz posted might guard against half-network failure, but is itself a single point of failure.

One thing I would recommend though, if your servers support them, is to network up the ILO ports (that's HP/Compaq, but other manufacturers probably have an equivalent LOM) on a private VLAN. That way you can always get onto them in an emergency - even when they're turned off. It actually works out cheaper than network-enabled KVM or power-cycling devices, which in my experience are a dog's dinner.

[Dog Pants]

network up the ILO ports (that's HP/Compaq, but other manufacturers probably have an equivalent LOM) on a private VLAN.

Dell use DRAC, which is for all intents and purposes identical but with a more Transylvanian name. We use them as engineering wires exactly as FJ describes and it works a treat (as far as I know, the server team have never complained about it, unlike pretty much everything else).

[ProfHawking]

yeah we have one hp, the rest are dell. I'll get round to that at some point.
For now, KISS, and meanwhile i hope to learn more about the intracities of routeros (its like cisco, but cheaper) so that when we do complex, its more understandable.