How to keep your physical server environment manageable

Hmmm, that's disgusting - but thanks for posting a picture worth-a-thousand-words. 8)

• Set a standard and stick to it, for example...
  • In every rack, the bottom X RUs are reserved for switches
  • Public interfaces will be green
  • Private interfaces will be red
• Use racks that have facilities for cable management on the doors.
• Use the same model of rack globally
• Label all cables (at both ends) with a barcode (or similar) writer
• Label the Racks as per the grid that they are if you look at them plane-view
• Inventory everything
• Don't let staff walk in and walk out of the server room
  • Only operations team should be allowed to touch the servers
  • Only the networks team should be allowed to touch the cables

Label both ends of every cable.

The cable should clearly show where each end is going, and you should be as specific as possible.

For example, a label in our data center looks like this:

<-- ATL-SW-1 PORT 23
SERVER-01 NIC 2 -->

The same label is put on both ends of the cable so no matter which end you are looking at you know exactly where the end point is.

With this simple process even a mess of spaghetti cables can easily be handled. The biggest problem with spaghetti cables is when you are forced to hand trace a cable to its end point.

However I do recommend cable management to avoid the physical mess :-)

Use velcro tie wraps when running horizontal cable.

• Set a simple coloring standard for cables, and stick by it, for example:
  • Blue for intranet
  • Green for public/DMZ
  • Red for VOIP telephony (if applicable)
  • Gray for bridging and servers
• Label both ends of cables with a good description of where they're going (server/socket).
• Use a label printer, and use quality nylon-based stickers, plain paper stickers wear out and fall off really fast.
• Don't use longer-than-required cables. If it's a short reach, use a short cable. Saves gobs of space and mess.
• Try not to have cables hanging in front of the panels/switches. Route them from the panel, through the back of the rack, and back in front to the switch/server.
• Immediately replace cables with broken lock-pins on the end. (this is more of a general advice)
• Install rack cable management accessories like hair-comb-like attachments to the back of the network jacks panel and door/sidewall mounted clips and straps.
• When having numerous cables going to/from the same location (like multiple blade server cables going to a switch) - enclose them in a cable organizer. This can drastically reduce the amount of cabling you have to untangle when looking/moving something.
• Same goes for cabling that goes outside the server room. Whether it goes above or below the (acoustic) ceiling - try to have a snake-tray or similar cable grouping system, don't just let the cables lie "naked".

It's an obvious one, but use rackmount servers and KVM switches.

Set things up so that your cable runs can be as short as possible. If you can get a space of 1U left between each patch panel in the rack, that's a great place to put switches and it means that you can use quarter meter lengths with no trailing whatsoever, and no need for cable management either (you don't need to manage what isn't there).

Separate things out into different racks if possible. If not, at least arrange them so that, for example, all switches for the first floor are physically located together.

FULLY DECOMMISSION OLD GEAR! If a switch dies, don't leave it in the rack and put a new one in wherever you have space - haul it out of there.

Label the front AND the back of your machines

1. Reduce the number of physical servers by using virtualization.
2. Use meaningful server names that would allow you to easily locate the server. e.g., elements in the periodic table (that's an idea from Stackoverflow)
3. Use a cable tie to group similar cables.

Blades help.

Nearly everything you can do has been mentioned already. It all comes down to being methodical and strict. Having to hand trace a cable at 2 am, armed only with a tile lifter and a torch is no fun at all.

One addition is to make sure that every cabinet door can be closed, i.e. no bodge jobs connecting device together that bypass the structured cabling. If one is allowed, then others will follow and ultimately you end up with a mess. Before any changes are made, make sure that you can achieve it with structured cabling. If not, you need to re-plan it properly, free up dead ports etc. If not you'll end up with a 20m cable chucked over the top of a row of cabs that is doing something important.

I primarily work in a startup environment. My current organization is a startup, and a previous business model was consulting startups. The biggest key to organization was outsourcing data center operations to dedicated hosting companies like Rackspace and Servepath, and also using cloud and virtualization technologies such as Amazon EC2 and KVM.

I haven't had to trace a cable in almost a year, and finding systems is a matter of looking at the master server that tracks all the nodes that check in for their configuration.

It's much nicer than a previous position where our servers were at a hosting center, but we had to physically manage everything (which resulting in cable messes, despite best efforts and proper labelling).