Configuring 4 Cisco 2960 Switches together with a pair of Force10 S55 Core Switches

The cheapest way to create redundancy is to do the following:

Switch A -> Core A
|
|
Switch B -> Core B

Spanning tree will bring down the link between Switches A and B unless one of the Core switches dies or the link goes down.

For performance reasons, split the primary NIC's on your server between Switches A and B equally.

Also regarding teaming the NICs, they will share a virtual mac so they need to both be in the same switch (or in a stack which is effectively the same switch).

To add to my reasoning, to be redundant without a cross-connect between the access-layer switches, you would need to connect each 2960s to BOTH S55's. I imagine this involves fiber and is therefore expensive. You will end up putting 2 SFP's in each 2960 and 2 in each S55 resulting in a total of 8 SFP's, patch cords and fiber strands in use. The only advantage of this design is that you can bundle the links together since the S55s are stacked as Art mentioned to increase your uplink to 2Gb. But if this is not necessary, it is an expensive way to do it when you can accomplish redundancy by using a CAT6 patch cable between the switches and connecting them in the manner I suggested. Either way - good luck!

If I understand your topology correctly, there is no need to connect your 2960-S switches together at the access layer. While it can vary based on unique requirements, it is not a Cisco or Dell best practice to cross-connect access-layer switches.

And as long as you dont cross connect at the access layer, spanning tree protocol will not constrain any of the links in your topology.

While it would be a preferred design to connect each 2960 to each network core, this is not necessary. Because your servers are dual-homed to redundant 2960's, if one of your 2960 or S55 switches were to fail, the backup link should take over subject to the timers your fail-over method uses, so if this is acceptable, you are all set. If you did choose to add a redundant connection so that each 2960 has a connection to each S55, that could be configured as a link aggregation group since the S55's are stacked.

You are correct that you cant use standard lacp to create active/active links from your server unless you stack your 2960's; however many server nic's and hypervisor solutions now have nic load balancing options that are completely switch agnostic. If you are using server virtualization, you can most likely turn this feature on. If not, check with your nic vendor or in your advanced nic settings to see if this feature is available on your platform.

You need to define how long you can be down before making any choice at all.

A stacked core provides good redundancy (especially when it comes to power failure, if those Force10 switches support power stacking). Running two links from each edge switch to the core removes the potential downtime if a core member dies, as it will use either STP or LACP (more likely) to keep the link up.

If uptime were highly critical at the edge layer you'd need stacked switches out on edge aswell etc.. redundancy is a never ending game :)

I am not a cisco guy so someone will probably be able to give better details but here goes:

1&2: If you enable STP (spanning tree protocol) you can have both an individual uplink from each 2960 to each s55 and a link between the 2960's. STP will take handle the potential loops and will only utilise for instance the link between the 2960's if one looses it's uplink.

If you use a protocol such as LAG or LACP, then you can have two links from a server with the same IP connected to the same switch for redundancy and increased throughput.

Finally I've noted in q1 you say the switches don't stack but you seem to suggest they can in q3.