A “5 Whys” conversation about measurement in health care
I’ve written this blog post as a “5 Whys” conversation. The 5 Whys is a learning tool from Lean that helps identify the real reasons (root-cause) why current practices are the way they are, or why a particular outcome or defect occurred, or why a particular plan for improvement is expected to work. It helps move beyond platitudinous answers and surface thinking, by continually digging for deeper, more thoughtful reasoning.
1. WHY do (should) we measure processes and outcomes in health care?
Because as “everyone” knows – you can’t manage what you don’t measure.
2. WHY can’t you manage what you don’t measure?
Because without measurement you really only have opinion and anecdote to depend on to tell you if your performance is getting better, staying the same – or getting worse.
3. WHY is opinion or anecdote insufficient?
Because, when you rely on opinion or anecdote when making decisions, you don’t have a way to judge the relative (or absolute) importance of the information you hear. This can lead you down all kinds of decision-making rabbit holes because people are highly vulnerable to several kinds of thinking problems or biases. For example, it is well known that people’s decision-making is highly influenced by information that stands out because it is easily remembered or has been heard repeatedly (accessibility heuristic) – as vivid anecdotes often are. While the information in the opinion or anecdote may be true, the importance of information gets over-weighted in the decision-maker’s mind, relative to other information. In management, this could lead to misguided strategic or tactical decisions that end up having less impact on the desired outcomes than expected. Having information based on measurement, not simply stories, is one ingredient in helping to overcome our human thinking traps and make better decisions.
4. WHY is measurement just “one ingredient” in making better decisions?
Measures don’t answer our questions on their own – they are just a (potentially) more objective source of information to use. We are still subject to thinking traps even when using measures. The other needed ingredient is that we establish and use our measurements in a way that helps us to learn. To learn, we need two more things: (1) some idea(s) – some theory – that describes what is and how we think things can be done differently to achieve the outcomes we desire; and (2) some predictions – hypotheses – that we can test in order to learn how well our theory helps us understand what is important and where we might need to add to our knowledge base. As W. Edwards Deming (one of the granddaddies of quality improvement) said: “….without theory there is no learning”.
HOLD ON A MINUTE–you had my attention when you were talking about practical things–now you’re talking about THEORY and HYPOTHESIS TESTING?
5. WHY are theory and hypothesis important to me? (Isn’t THEORY only for university professors who have nothing better to do with their time?)
Deming also said: “Knowledge is theory. We should be thankful if action of management is based on theory. … Information is not knowledge. The world is drowning in information but is slow in acquisition of knowledge. There is no substitute for knowledge.” (Deming. The New Economics for Industry, Government, Education. 1993)
If measures (our “information”) are not selected and used within a framework of theory (i.e., an understanding of the ‘current state’, the desired ‘future state’ and what we believe will help us get there) then they don’t help us – rather they drown us in extra effort. Hypotheses are important because they force us to actually use our theory to predict what our next course of action should be. Learning happens when we do something – based on what our knowledge of our work processes tells us should help us get to the future state – and it either turns out as we predicted or it doesn’t. Learning is actually deepest when our prediction was wrong and the action didn’t help in the way we thought it would. This is because predictions that turn out to be wrong tell us that we have more to learn (improve our theory).
OK – so remind me how this relates to measurement in health care again?
We started this conversation with “Why…measure… in health care?”
After five WHYs we’ve got to the root of it: we measure in order to help us learn. And as every good lean leader knows, management is ALL about learning and helping those we lead to learn.
If we find ourselves measuring something “because someone told us that’s what we should measure” – if we aren’t clear on how to use the measure to help us learn and move our work processes closer to desired future state – we should get that clarity as quickly as possible or stop that measurement work.
For anyone interested in more on this topic I encourage you to read a recent open access online publication in the journal BMJ Quality and Safety called “More quality measures versus measuring what matters: a call for balance and parsimony”.
I want to hear from you. Submit a comment — I promise I’ll respond, whether you agree with me or not. This is a topic we need to talk about more in health care.