Made To Measure: Getting Your Head Above Water In A Sea Of Metrics

With even the most traditional businesses embracing analytics, an explosion in data volume has left many decision makers swamped by information. It’s hard to drive the right actions when you’re drowning in a sea of useless metrics. Defining these metrics and how they get used can be complicated, expensive and often political. It can help to take a step back and start with three basic questions:

1. Who cares about this metric?
2. Why do they care (if the metric changes)?
3. What action will they take (if it does change)?

Before spending time reviewing existing metrics or developing new ones, get clarity on these points. If you can’t find anyone who can explain how a given metric will drive action, it’s probably not a metric worth measuring. It can also be helpful to understand what type of metric you are dealing with. One useful approach splits metrics into three categories; outcome, processand balancing.

Outcome metrics measure an end point, either directly or by proxy. For example, a mail order company may use revenue as an outcome, or number of orders delivered as a proxy. It’s usually best to measure directly when you can and avoid proxies unless it’s not practical to measure directly; for example, if it’s too expensive, too complex or there’s too much of a lag for the metric to be useful. Outcome metrics measure the things that are bottom line; tracking these is essential as changes in these metrics can signal success or failure for the business. On the other hand, because outcome metrics are usually dependent on many underlying factors, it can be hard to determine the right action to take looking at these metrics alone.

Process metrics measure some element of the processes that drive our outcomes. For example, in our mail order analogy, time from order received to package dispatch could be a useful process metric to track. As there are fewer contributing factors than an outcome metric, it’s often easier to determine the right action to take to affect these processes (and indirectly, our outcomes). Don’t forget that these metrics are secondary to our outcome metrics; don’t optimise and incentivise around process to the detriment of outcome.

Finally, there are balancing metrics. These are typically things that might affect the long term health or sustainability of your business. For example, in our mail order analogy, these might include customer satisfaction (which moves slowly but can have a major effect on sustainability of a business) or staff turnover (keeping this low will reduce training costs and maintain productivity). Whilst outcome is key, many businesses do not take these balancing metrics into account; such short sightedness can lead to negative long term outcomes.

Why does all this matter? When you measure something, you implicitly place value in it. By doing this, you draw attention to it and create ‘soft’ incentives to make sure the metric moves in a certain way. It’s a bit like Heisenberg’s uncertainty principle; by measuring something, you change it! By measuring the wrong things, you change the wrong things and distract from the things that really matter; your outcomes. It’s OK to measure process and balancing metrics but it’s important to state them explicitly as such and ensure you are investing your attention appropriately.

In summary; be intentional about what you are measuring and clear about what your outcomes are. Use process and balancing metrics to keep you on track and sustainable in the long term. Decide what matters, measure what matters, take action on what matters. A successful business is made to measure.

-- Colm