Server room survival kit?

Things that I always carry on my person, so would be present:

• cell phone
• iPod
• pen/notepad
• thumb drive
• multitool

Things that I keep in my laptop bag so I don't have to think about it:

• "carb bars" (I don't know what these are, but they last forever. My wife made me start carrying them after I had to sleep in a data center during a blizard.)
• quarters for snack/pop machines
• a baggy of splenda (nothing worse than being stuck with people who only drink their coffee black)
• notepad
• Post-it notes
• recovery disks (live CDs)
• USB/serial/RS-232 cables and adapters (the 5-in-1 cable kit specifically, though I;ve tweaked it to have things like T1 loopbacks)
• penlight
• electrical tape
• CO scissors (the kind that central office guys always carry around that you can cut and strip wires with)
• screwdriver (the kind with 6 ends)
• a small hand mirror for looking behind/around things

Things I keep in my toolbag - not guaranteed to have with me always, but I usually know if I'll need it:

• a second 5-in-1 kit and some more cables
• crimpers with RJ45 and RJ11 ends
• labels (like, mailing labels - very sticky and handy for rapid labeling until a professional job can be done with a label maker
• screwdrivers/plyers/end cutters/small socket set - basic tools
• a huge screwdriver that can either be used as a crow bar or to reach the mounting screws on devices that stick out of the rack (like mid-mounting a 40-inch server in a 2-post telco rack)
• velcro wraps and wax string (never used wax string? Try it, it's awesome)
• a collection of writing utensils including sharpies and wax pencils to write on racks
• a collection of screw driver heads - flat, phillips, hex, torx(sp?), and some other specialty ones
• spare heavy-duty power extension cords and a three-plug expander
• a decent digital mutilmeter
• duct tape

I think I have more, but that's the basics. Everything on that list addresses a specific need I've had in my career. The laptop bag is heavy but well worth the bulk in saved trouble. The tool bag I'm rather proud of, it's not big (it's one of those "big mouth" bags that opens like a doctor's black bag), maybe 18 inches long and 12 wide. I spent a great deal of time customizing the contents to maximize the value for the volume. For instance, I threw away the bulky plastic container the socket kit came in; I built a much smaller organizer for it. Same with the screw heads - I built a cloth with elastic on it that the heads slide into. It's also modular - all the screw drivers are in a large pencil case, so I can find them easily and, if I know I will only need them, I can just grab them out of my car and carry them into the DC instead of the whole tool bag.

• A workspace with enough room to work comfortably on a broken 19" server, with screen, keyboard, mouse. Separate from the racks.
• An old PC. Optimally with controllers and slots to fit every piece of hardware you may have to analyze. Mine speaks SCSI wide & narrow, IDE, SATA, PCI, USB, Firewire 400. Keep a small stash of old computers, if you can. They will come and ask if you can rescue the data from this 5.25" disk one day.
• A notebook on the side. The rescue PC has no internet connection, to make sure it cant be infected.
• A big enough disk to put data on that you may have to rescue.
• Room for spare parts and cables. Room for a museum of old stuff you might need for old systems.
• Cart. In case you have heavy servers, a lift of some sort.
• A selection of tools you know you will need. You can keep this small, if you have a complete set elsewhere.
• Telephone with outside access, if your mobile doesnt allow that.
• Pen and Paper.
• Spare parts for your most important servers. If you have several identical machines, keep one spare. It is the organ donor. It may be used for testing new setups, but be prepared to rip it apart.
• A few switches, network converters, cables of all kinds.

Generally make the server room your fortress of solitude, where you can retreat when the brown stuff hits the rotating thing. Nothing like coming out smiling after one hour of hacking, and the broken server is back up, with all data.

I'd add

A magnifying glass** so you can read the ridiculously teeny-tiny writing you get on some equipment, and a mini-maglite so you can use it.

** yes, I am seriously old

Aside from tools I would highly recomend a small first aid kit, and some nonmessy snack foods that keep in storage well. Being able to put a bandage on a paper cut or other small nick on the spot is nice insted on having to hunt down someone from security just for a small bandage. The snacks are good for when it has been two hours too long and you are still more or less stuck in the computer room.

First, let's assume you have the essentials, including a cell phone and something that tells time (the cell phone will do).

Let's remember, if you have to go to the server room, it means something doesn't work. Otherwise, you could fix it from your desk. So, you should at least have:

• A USB stick with your favorite utilities
• A blank USB stick to move files on
• A flashlight
• A dustmask (if you have allergies)
• A screwdriver set
• A tray to put screws in after you remove them
• Pen and paper to take notes with
• An OS image CD, if it doesn't fit on the stick.

After that, it depends on how thick you want your bag to be and your budget. You could do well with:

• A roll-up mini USB-keyboard, if the server room doesn't have one
• A laptop to test things "from the outside" and access the internet if your phone doesn't do that already
• A few extra cables, just in case

Lost of good suggestions so far.

How about ear plugs/ear protection. With all the fans running in your typical server room, the noise level is probably above what is considered "safe levels" by your typical ETLA government agency (hence the ear protection).

I bet you could even make a case for eye protection as well.

A second person/working alone policy. Lots of stuff can happen when you least expect it. You get pinned by a piece of equipment. You get electrocuted. Heck, something as simple as you get locked in. If you have trusted backup who knows when you are starting/stopping work and when you are supposed to check in, they can come to the rescue. Otherwise, you could be waiting a long time, or dead.

Telephones as mentioned above, but with a long enough cord to take the handset to any cabinet. And yes, corded phones -- there's likely to be enough signal on whatever frequency you choose for there to be a problem with cordless.

While I'm on the subject, even if the site uses VoIP phones, you need a non-VoIP, non-PBX, direct line to handle the instances where the VoIP or PBX equipment is down.

Other stuff: Printed reference material - phone numbers, networks, remote host dependencies, etc. Stuff that you might need to bring up the server where the online copies reside.

Larger, easier to handle screwdrivers, torx wrenches, wire cutters.

The small packs are nice in an emergency, and they go with you, but trying to use those little guys for hours on end can begin to hurt your hands.

A big, big roll of sheet plastic and duct tape.

For when the ceiling leaks (water), or someone decides they have to drill holes in the walls (dust tent), or when you have to rig up some emergency cooling.

• A sweatshirt/hoodie. It's cold in there! It's hard to have a steady hand if you're shivering. If you're not cold, take the hoodie off.
• A laptop, of course. Make sure you have all the utilities and documentation you will need pre-installed.
• USB and PS/2 keyboard and mouse if you don't keep spares in your server room. We keep spares there, and all of our machines take USB peripherals, so that makes things easier.
• A monitor (LCD is much easier to carry) if you don't keep a spare with the servers. Again, better to have a spare already in the room.
• lc mentioned multiple USB sticks, and I second that. You might consider an external USB hard drive if you have to move a lot of data.
• Everyone mentioned a cell phone already and I have to agree. You never know when you have to call for help or have someone on the outside run a test.
• Anti-static wriststrap if you're going to open up a server. Electro-static discharge (ESD) can destroy components even if you didn't see/feel/hear the spark. It can also break things in subtle ways that are extremely hard to diagnose later.
• Camera (cell phone cam usually good enough). Recording labels, cable routings, server contents, etc is much easier if you can take a picture. Also good for recording water damage, etc.
• Pen and paper are obviously useful.
• Masking tape. It peels off more cleanly than duct tape, it's cheaper, and can be used both for labels and for binding errant cables. It's also easier to tear. You can use it to post warnings and reminders as well. You can even use it to keep little screws attached to the case they belong to.
• Screw driver set (lc and others mentioned this). You will likely need the full range of Phillips (+) and slotted (-) screwdriver sizes. Always use the largest that fits comfortably in the screw, so you don't strip the head. Socket drivers are also handy, and less likely to strip the screw.
• Penlight (small flashlight/"torch") to see clearly in those tight spaces. Works well with the camera.
• Small magnet on a stick/tweezers to grab dropped screws from tight spaces. BE CAREFUL WITH THESE! Most magnets are too weak to affect a sealed hard drive, but don't wave one willy-nilly inside a running server. Tweezers are usually uninsulated metal, and pointy metal objects should not be inserted into runinng servers either. But it's much nicer than having to hold an open server upside-down and shake it to get little screws out.
• Needle-nose pliers. Comes on most multitools. You never know when you need them.

EDIT: - Canned air! Servers collect dust quickly, and it can clog heatsinks and fans. It's also really gross. If you have to go into a server, blow the dust out of there.