When to Deliver Bad News

Timing around delivering “bad news” can be tricky.  Deliver too quickly, and you’ll be accused of panicking.  Wait too long and it will be suggested you hid a problem.

It’s our experience giving people bad news sooner rather than later is the key.  If you are working on a major program, you might list “bad news” on a Risk/Issues register…where a risk is something that might happen, and an issue is something that has happened.

We always try to work an issue before raising a red flag.  After all, it’s our job to work items and make them non-issues.  Sometimes there just is no way out.

When you identify an issue, always try to suggest mitigations…so the issue just doesn’t lay on the table without hope.

A few other factors always come in to play:

• Nobody likes surprises – the last thing an executive wants is to be surprised by an issue.  Most executives are accomplished at hearing bad news when they know bad news is coming.  It can be as simple as saying, “We’ve got some bad news to cover,” followed by an explanation.  Often the executive has ways to mitigate an issue…and can when given the opportunity.

One time a subordinate sent out an email with a CFO’s name on it.  The CFO had not seen or approved the email.  When this was uncovered, we went to the CFO, explained the situation, let him get a little red-faced, and then knew we would live for another day when he said, “All we can do now is backfill, so how do we do that?”

• Share bad news privately – inexperienced project managers will clearly identify an issue on a report or status and leave it at that.  This presumes the busy executive has time to read the report!  Make a point of sharing bad news privately

Execs are people, too.  Sometimes they just need the privacy to drop an f-bomb and then get their game face on.

• Always get to the executive first – bad news travels like wildfire.  Executives want to help, and feel betrayed if they hear about issues from a peer or in a status meeting.

When an executive feels they are the last to hear bad news, trust erodes quickly.  Executives may not like bad news, but they appreciate knowing about it first.

Simple?  In a blog, it is simple.  In the real world it is much more complicated and gets into the personalities involved:

• Is the executive available (or traveling?)
• Bad news on Fridays can be tough.
• When is the “big meeting” (whether related or not)

Our rule of thumb is the bigger the risk, the quicker we must act.

Data Center Migrations

As companies emerge from the great recession, data center spending appears to again by in vogue.

While we believe colos and clouds deserve serious attention, many companies are expanding existing data centers to address the needs of business and/or systems growth.

Ok not THAT kind of migration

The first, and arguably most important, step in any migration effort is understanding the inventory. It’s our experience few companies have this airtight.

The inventory must include hardware (model, serial number) and also applications, databases, circuits and interdependencies. Any special power (including outlets) and cooling needs (a blade enclosure brings a significant heat source.) Network and SAN connections must be understood.

The company then decides how to perform the migration (most companies decide this first, and we believe in understanding the requirements first). Is it a “lift and shift,” a P2V (physical to virtualize) effort, or a fork lift upgrade? How the data will be migrated is a key consideration, with attention paid to how quickly data can be moved.

Most companies allow development machines to be “moved” anytime, although consideration must be given to the development machine usage. For example, a US based company moving a development environment during the US day is preferable when using a India development team.

The network team is a key player in any migration. How the network is extended between locations will have a direct impact on how challenging the migration will be….especially if IP addresses must change. Any administrator will tell you changing IP addresses is an easy task; however any “bad” application coding techniques will quickly be exposed.

Arguably the easiest migration is establishing a mirror environment, migrating data and testing applications in a “cocoon” segregated from the current production environment. This type of migration reduces downtime, and if clustering approaches are used downtime is virtually eliminated. Migrations are also an opportunity to use disaster recovery environments (provided they are well tested in advance.) The target environment can be an upgraded processing environment, and vendors often facilitate with an asset swap minimizing licensing and tax ramifications.

Perhaps the most challenging migrations are lift and shifts. In these migrations, a (backed up) environment is brought down, moved, and then restored. The issue is “falling back” is a practical challenge. Once the machines are moved, it’s very rare they are ever moved back…instead, they are “fixed in place.”

It’s important as much testing and verification as possible is performed before moving systems. All electrical must be confirmed, cables/fiber tested and labeled, door widths and heights reviewed, physical security access confirmed, loading dock heights confirmed, etc. When you find that one loading dock door that’s “always open” locked…how do you get it unlocked at 2AM? And when you are moving distances, it is prudent to check weather and road closures whenever possible BEFORE starting the migration. Some equipment is so large, professional movers are needed.

In a physical move, the application is brought down, backups performed, database brought down, followed by the server being brought down. Once at the target location, the server is brought up by the system administrators, database is confirmed by the DBA team, and then the application is prepared for user verification BEFORE productive use. At best, this is a multi-hour event. In a virtual migration, data is first migrated, and so the live environment is established and tested. The network team can then repoint DNS so the new environment. The application should be quiesced in the original environment so nobody can continue entering transactions.

Migrations can take time and often take place in the wee hours. It’s our experience, especially when physically moving older equipment; some equipment will not immediately come back (~1%). This often requires an extended recovery time, and necessitates “shifts” of support people. Technicians need to know this in advance and plan for it; to many, it is a badge of honor to work 72 hours straight.

In a short piece like this all options for migrations can’t be covered. Migrations are projects like any other, and require good planning and execution.

Project Planning 101

A number of years ago, my 11 year-old son came to me and asked me to sign his homework. Being an engaged parent, I decided to actually look at his work. To my surprise, he had been asked by his teacher to complete a project plan for his upcoming math project.

I looked at his paper and realized it asked for the following:

• A list of materials (“resources”)


• Steps to complete the project (“tasks”)


• A draft drawing (“milestone”)


• Due date (“deliverable”)


• His name (“project manager”)


• Parent signature (“signoff”)

I immediately asked when he found the time to go to Microsoft Project training

and how he managed to connect to Project Server so he could allocate enterprise resources to his project plan. He turned to me and asked whether I had been “inhaling” (this was when Bill Clinton was running for president) and then proceeded to tell me that he simply wrote down what he needed to complete his math homework. When asked how long it took, he told me 5 or 10 minutes, but that included keeping his lines straight.

While I poke fun, I wanted to share the importance of starting simple. Many technical managers become overwhelmed when asked to do project plan. They don’t understand how to use Microsoft Project and believe it is the key to building a project plan. Understanding your project and articulating what needs to be done (tasks) and the people required (resources) are the building blocks to a robust project plan. These can be documented in a Word document or and Excel spreadsheet. Think about logical sections of your project and you will start to develop phases. Each phase should have at least one deliverable. Document this deliverable and it becomes a milestone. Printing these documents and drawing lines between tasks and phases will create dependencies. Now you have what you need to use a project management tool, such as Microsoft Project.

People believe project management tools are all about building project plans. They force a their project plan into the tool, rather than using the tools to help them predict deliverable dates, resource levels, conflicts, budget issues, and project risks. A well designed and implemented project plan is a living document used to clarify project roles and responsibilities for the project manager, participants, and management. If done correctly, it can save time and provide real-time views of project status.

So, when starting any project, take the time to create a simple plan. Be comfortable and confident in the tools used to create the plan. Remember, the content is much more important than the presentation. If the tool you have selected is not a project management tool, then spend the time insuring your plan is comprehensive and robust before transferring the data to a project management tool. Use the project management tool to help you estimate key project metrics, and you will be successful.