Gary L Kelley - Journal

I have been teased my entire career about my nearly obsessive behavior around keeping data center rooms neat and tidy. While I’d love to blame my Mother for my neatness, the truth is keeping a data center clean is about one word: discipline.

Having a neat and tidy data center environment sends a reinforcing message to everyone entering about the gravity of the work performed by the systems in the area. This is important for staff, vendors, and clients.

The observational characteristics I look at when I walk in a data center are:

Life Safety – are the aisles generally clear? Are there Emergency Power Off switches and fire extinguishers by the main doors, is there a fire suppression system in place, is the lighting all working….

Cleanliness – Forty years ago data centers were kept spotless to prevent disk failures. A speck of dust might make a hard drive disk head fail. These days, disks are generally sealed, and can operate in pretty rough environments (consider the abuse of a laptop disk drive.)
While disk drives are generally sealed, why should data centers be dirty? Look for dust, dirty floors, and filthy areas under raised flooring. One data center I went in had pallets of equipment stored in the space…was the data center for computing or warehousing?

Underfloor areas – are the underfloor areas, assuming use as an HVAC plenum, generally unobstructed? More than one data center I’ve been in had so much cable (much abandoned in place) under the floor the floor tiles wouldn’t lay flat. This impacts airflow, and makes maintenance a challenge.
I also like to see if the floor tiles are all in place, and if some mechanism is used to prevent cold air escaping through any penetrations. 30% of the cost of running a data center is in the cooling, and making sure the cooling is getting where it needs to be is key. (While at the opposite end of the space, I like to see all ceiling tiles in place. Why cool the area above the ceiling?)

HVAC – are the HVAC units working properly? Go in enough data centers, and you’ll learn how to hear if a bearing is failing, or observe if the HVAC filters are not in place. As you walk the room, you can simply feel whether there are hot spots or cold spots. Many units have on board temperature and humidity gauges – are the units running in an acceptable range?

Power Distribution Units – are the PDUs filled to the brim, or is available space available? Are blanks inserted into removed breaker positions, or are their “open holes” to the power. When on-board metering is available, are the different phases running within a small tolerance of each other? If not, outages can occur when hot legs trip.

Hot Aisle/Cold Aisle – Years ago all equipment in data centers was lined up like soldiers. This led to all equipment in the front of the room being cool, and all the heat cascading to the rear of the room. Most servers today will operate as high as 90 degrees before they shut themselves down or fry. By having a hot aisle/cold aisle orientation, including blanks in empty shelves on servers, cooling is most effectively in place. Some organizations have moved to cooling being in the racks as a designed alternative.

Cable plant – the power and communications cable plants are always an interesting tell tale sign of data center disciplines. Cables should always be run with 90 degree turns (no transcontinental cable runs, no need for “cable stretching”). Different layers of cables under a raised floor are common (power near the floor, followed by copper communications then fiber). (A pet peeve of mine in looking at the cable plant is how much of the data center space is occupied with cables. Cables need to get to the equipment, but the cable plant can be outside the cooled footprint of the data centers. Taking up valuable data center space for patch panels seems wasteful. One data center devoted 25% of the raised floor space for cable patch panels. All this could have been in not conditioned space.)

Error lights – As you walk around the data center, look to see what error lights are illuminated. Servers are often monitored electronically, and error lights utility is lessened is a argument. That said, error lights on servers, disk units, communications units, HVAC, Power Distribution units and the like are just that: errors. The root cause of the error should be eliminated.

Leave Behinds – what’s left in the data center is often an interesting archeological study. While most documentation is available on line, manuals from systems long since retired are often found in the high priced air and humidity controlled data center environment. Tools from completed projects laying around are a sign thoughtfulness isn’t in place for technicians (I’ll bet their own tools are where they belong).

Security – data centers should be locked, and the doors should be kept closed. Access should be severely limited to individuals with Change or Incident tickets. This helps eliminate the honest mistakes.

While far from an inclusive list, this article is to help silence my lifelong critics about my data center obsessions. These are simple things anyone can do to form a point of view on data center disciplines. Obviously follow ons with reporting, staff discussions, etc. is appropriate.