Is Your Work Resulting in Better Decisions? How You Can Make Sure

Earlier this year we reflected on how important it is in analytics to start with the right question and to choose the right metrics for your organisation. Say you have indeed done so and have worked your way up the analytics value chain. You should now have some powerful insights to land in the organisation. At this point, surely all that remains is to communicate your findings to the appropriate audience and the work is done? Maybe, but often that is not the case.

After all, the goal of analytics is not to crunch numbers, uncover insights, or even to make effective recommendations to an organisation. Fundamentally, the aim of analytics is to use data to improve decision making, whether those decisions exist to benefit an individual, a corporation, or a society. In other words, until those better decisions are actually made and enacted, the value of analysts' work is unrealised.

So how do you know better decisions are being made based on your painstaking work? In the case of analytically minded business leaders, you can simply ask them! With their cognitive tool set they should be able to reflect on how your work helped them make a particular decision – or not. However, in cases where your stakeholder is lacking the ability, you might have to try a different validation approach.

One effective way is to stop sharing your work with this team or person. Particularly in the case where you are providing reports and dashboards as opposed to fully fledged insights and recommendations, this is a highly effective way of determining whether your work is actually being used. If your stakeholder complains they can no longer do their job because they cannot make decisions unaided, congratulations!

The gold standard would be experimentation through a randomised controlled trial (RCT). If you have two comparable stakeholders (e.g. sales teams) you could choose one at random and provide them with your latest and greatest insights, while supplying the other team with generic information. Hopefully, you will notice an improvement in the success of the former based on the insights you uncovered.

In some situations, either approach will not be ideal. In reality, there are multiple factors that can limit your choices, such as fragile relationships or a lack of comparable teams for testing. However, this does not mean you are out of options. You will often be able to embed someone from your team with your stakeholder, so that you can develop a first hand understanding of their decisions. Analytical ethnography!

Whatever route you choose, what matters is that you know if your work is helping the organisation make better decisions. This is easier said than done, given that people are not always great at taking advice and that analysts can find it comforting to ignore what happens with their work once it has been published. Yet, without this focus on impact, we humans will never realise the full potential of data and analytics.

– Ryan