Why measuring success by time and cost destroys projects

In the dawn of the electronic data processing (EDP) era – I’m showing my age here – measuring project success was simple.

It was a miracle just to get the hardware and COBOL programs working. Success in those days was just delivering something that worked and did not crash too often in the middle of the night. However, the projects were always late and always cost far more than anticipated.

Then, as EDP turned into IT, adding in project disciplines and tools borrowed from construction project management – critical path analysis, earned value, work breakdown etc. – was seen to be the solution. These additions ensured tight control over time and cost and all those newly minted project managers could then be measured and rewarded appropriately.

For example,

Finish ‘on time’ and ‘on-budget’ and you, Sir Project Manager are a hero.
Meanwhile we, the overstretched and overstressed business executives, can get on with our real work of operational management, confident that if the wheels fall off the project schedule and budget, we can fire you and hire another hero.

Over the last 2 decades measuring project success by time, cost and specification, the so-called ‘iron triangle’, has become the standard, unchallenged way we do things. Although over time, it has become obvious that such limited measures can lead to unintended negative consequences. So there have been ineffective attempts to add in some additional ‘soft’ measures:

User satisfaction
Business KPIs
Team ‘happiness’, etc.

Indeed, if you Google ‘measuring project success’, you will get 135 million results, most of which state that measuring project success is really, really hard.

Actually, measuring project success is easy

When you measure project success by time and cost, you are measuring the success of your project by the inputs used, not by whether you are getting something valuable as the outputof the project.

This is a bit like, and as dumb as, measuring the success of building your new family home by whether you used all the bricks and timber calculated. Intuitively, you know that the true personal measures of success for your new home will be:

having a comfortable and welcoming place for your family and friends,
with space for the kids to hang-out with their friends,
which has room for you and your partner to work from home,
where perhaps you can care for an elderly parent while allowing them their own space and privacy,
and all of this achieved within the time and cost you determine to be personally acceptable for your new home.

Time and cost therefore are not absolutes – they are meaningful only relative to the value delivered for the project.

The problem with measuring project by time and cost

The problem with measuring project success by the inputs is that you will achieve what you measure. Without fail, all the inputs of time and cost will be used up. But that does not guarantee your project will produce something of value.

If you measure only by the inputs used, rather than focusing on first defining and then tracking the output value as it is delivered through your project’s life, then you and your project are embarking on a long journey without a map and compass to keep you heading in the right direction. As the Cheshire Cat said to Alice, "If you don’t know where you are going then any road will take you there."

It will matter not one jot that your project is delivered on-time and budget, if it is ill-conceived, miss-conceived or just plain dumb.

If your project does not deliver an output that is valuable, it is a failure. No ifs, no buts, no exceptions.

And the reverse is true. If your organization achieves the valuable results and improvements, for the resources invested, within a time and cost that is acceptable to you, then your project is successful, even though it may be over-time or budget.

So how should you measure project success? It’s simple – by value

You should define and then measure your project by the value delivered.

People mistakenly think value is an absolute, but it is not. Value is defined relative to what is important to you and your organization. What is valuable to one organization implementing a CRM platform might be completely different to what is valuable to another.

Here too, we are using the term ‘value’ in its broadest sense – to include the desired outcomes, benefits and quantified $value, together with the project’s strategic contribution, meeting the reasons for doing the project and avoidance of any downsides of not doing the project.

To measure project success, you need to be able to measure three things:

1. Did you get the desired outcomes?

Desired outcomes are a series of statements that answer precisely, specifically and measurably:

What is your project intending to achieve?
What does the future business-as-usual look like when everything is working ‘just right’?

Success is measured simply by ‘yes or no’ and ‘do we or don’t we’.

2. Did you get the benefits?

Benefits are the desirable consequences delivered by achieving the desired outcomes. They fall into six categories, starting with an external focus and finishing with an internal focus. Benefits are defined for each desired outcome by answering these questions:

Customers — How will this outcome benefit our customers?
Competitors — How will this outcome make our company more competitive?
Capabilities — How will this outcome improve our company’s capability to perform and deliver?
People/productivity — How will this outcome make our staff more productive?
Financials — What financial benefits will this outcome deliver?
Risks — What risk exposures will this outcome reduce, eliminate, or avoid?

Success is again measured by ‘yes or no’ and ‘do we or don’t we’.

3. Did you get the $value

Here we now define ‘value’ in its narrowest sense – as measured in money terms. To measure value, we build a financial model which takes input drivers and predicts how much will be banked by the changes in the drivers. For example:

How much will a reduction in error rate reduce cost of materials purchased?
How much will an increase in ‘net promoter score’ increase revenue?

$value can vary greatly, and for reasons quite beyond the project’s or the steering committee’s control. When measuring and tracking value and variances, we need to know both the quantum and the reasons these differ from our predictions in order to take the right corrective action – whether to speed up, slow down, or even cancel the project.

When you measure project success by output value…

A funny thing happens. You increase your success at getting the results and value you intend, for the time and cost input to the project.

It’s that simple