Evidence Based Management: Making Better Decisions at Work

Introduction

Most management decisions are made with evidence of some kind. Someone has a view, someone else has a dashboard, and a decision comes out. The question is not whether some evidence is used (some always is), but whether the evidence used is any good, whether there is more than one kind of it, and whether anybody checked.

Evidence based management is an attempt to make that process more deliberate. Several years ago James (and Jane) spoke to Rob Briner, for the World of Work podcast about what it involves and why so few organisations do it. Rob helped launch the Center for Evidence-Based Management (CEBMa), which aims to do for management what evidence-based practice has done for medicine and policing.

This piece summarises that conversation. You can still find that podcast episode wherever you get your podcasts.

What it actually means

The purpose is unglamorous: to make better informed decisions.

Everybody uses data and information when they decide things. They often do it badly, because there are predictable barriers in the way. Evidence based practice is one of several techniques that have evolved to help. The drive towards a more evidence based approach to things began in medicine around thirty years ago, though the underlying idea of using better information is not new at all.

Briner describes three differences between evidence based practice and what most of us normally do.

1. The approach to evidence is conscientious, explicit and judicious

Conscientious means you actually try. Explicit means you write it down, or record it somehow, rather than keeping it in your head. Judicious is the one that does the most work: it means making a judgement about the quality of what you have.

This is the part people find counter-intuitive. In evidence based practice you deliberately do not use all the information available to you. You use only the best available, on the grounds that a great deal of what surrounds us is poor quality and using it will not improve the decision. Think of all the “slop” that exists in the world now. We really don’t want to use that to help make our decisions.

2. It draws on multiple sources

For management and HR there are four:

• Your own professional expertise as a practitioner.
• Organisational data — what is actually happening in your organisation.
• The scientific literature relevant to the decision in front of you.
• Stakeholder perceptions and concerns — employees, managers, sometimes wider society.

Most people use one, or perhaps two: their own experience, maybe with some organisational data attached. Using several does two things. It lets you cross-check. You can ask: “am I getting the same picture from different directions?”. Perhaps more importantly, it also lets you contextualise. Good evidence from one source still may not apply in your situation, and the other sources are how you find that out.

3. It takes a structured, stepped approach

This means always starting with the question: “what is the problem, or the opportunity?” Only once you have a clear view of the problem or opportunity should you move into finding a solution. It is a familiar failure: an organisation adopts an intervention before anyone has established what it is for.

It is also stepped in a narrower sense: ask a question, acquire evidence, critically appraise its quality, and so on. There is a technique to it, but the three differences above matter more than the procedure.

Why so little of this happens

Asked how widely it has been adopted, Briner is candid: not very. There is more interest and more discussion than there was, and more universities and business schools teach it. In the UK the CIPD has explicitly adopted evidence based practice as part of its professional standards. But visible activity inside organisations remains thin.

The interesting part is his account of why, because it is not the obvious one.

Evidence based practice is hard, but not because it is intellectually demanding. It is hard because of cognitive biases, because of organisational politics, and above all because of incentives.

People are not incentivised to be evidence based. They are incentivised to do things.

That single observation explains a great deal. If your reward comes from launching the initiative, running the programme and spending the budget, then stopping to ask whether it works is at best a detour and at worst career-limiting. Briner is emphatic that this is not an individual failing. This finding is a property of the system people are working in, and blaming practitioners misses the point.

Two other barriers are worth naming. The first is ordinary human decision making: we like quick, intuitive, spontaneous choices, and for most of daily life that works extremely well. It is specifically bad for important, complex decisions, where confirmation bias leads us to look for evidence supporting what we already intended to do (see system 1 and system 2 thinking). The second is management fads. The new and attractive thing that everyone is adopting, the current buzz-word, tends to draw us in and their existence tends to switch off critical analysis altogether.

The evaluation problem

There is a self-reinforcing loop here that anyone who has commissioned training will recognise.

Most training is never properly evaluated. If nothing is evaluated, it does not much matter what you run, as long as people enjoy it. Positive reaction sheets keep the budget and the programme alive, and enjoyment is a poor measure of whether anything was learned. This is an exhausting truth in our experience in this field. And, of course, it’s one no one really wants to face into.

Briner offers an example from his own profession rather than pointing at anyone else’s. Standing at the front and lecturing is not generally regarded as an effective way for people to learn, yet the lecture remains the standard unit of university teaching. Teaching is assessed largely through student evaluations, which are known to be biased and which mostly measure how much students liked the lecturer. The relationship between those evaluations and actual learning is, at best, around zero. It might even be negative, since the courses people enjoy most are sometimes the ones on which they learn least.

He is clear that he would find it genuinely difficult to be an evidence based educator, and that this is a problem of systems and structures rather than of individual will.

Where to start

The practical advice is deliberately modest: do not try to do this for every decision.

Where the stakes are high, people adopt something close to evidence based practice without being told to. Briner’s own example is choosing where to eat in an unfamiliar city — you check several sources, you know each has its biases, you ask people whose judgement you trust, and you end up running a reasonable version of the process without calling it one.

Medicine and aviation are often held up as the models, and the reason is not really that their outcomes are easy to measure. It is that the cost of a mistake is very high and the benefit of a well-informed decision is obvious. Where people cannot see that the outcome matters much, they don’t bother.

So the sensible move is to pick the decisions where the consequences are real: a restructure, a selection process, a large investment in development, and apply the discipline there. Ask what the problem actually is. Gather more than one kind of evidence. Say out loud how good each kind is. Write it down.

That is a long way short of transforming your organisation, and it is considerably better than what usually happens.

Listen to the full conversation

This article is drawn from our World of Work podcast episode with Rob Briner on evidence based management and evidence based HR. If the topic interests you, the full conversation goes further into decision training, critical thinking and the role of data analytics than we have here.