WHY, oh WHY (oh why, oh why, oh why) do we need to measure?

2012-11-13 Why oh why

A “5 Whys” conversation about measurement in health care

I’ve written this blog post as a “5 Whys” conversation. The 5 Whys is a learning tool from Lean that helps identify the real reasons (root-cause) why current practices are the way they are, or why a particular outcome or defect occurred, or why a particular plan for improvement is expected to work.  It helps move beyond platitudinous answers and surface thinking, by continually digging for deeper, more thoughtful reasoning.

1.    WHY do (should) we measure processes and outcomes in health care?

Because as “everyone” knows – you can’t manage what you don’t measure.

2.    WHY can’t you manage what you don’t measure?

Because without measurement you really only have opinion and anecdote to depend on to tell you if your performance is getting better, staying the same – or getting worse.

3.    WHY is opinion or anecdote insufficient?

Because, when you rely on opinion or anecdote when making decisions, you don’t have a way to judge the relative (or absolute) importance of the information you hear.  This can lead you down all kinds of decision-making rabbit holes because people are highly vulnerable to several kinds of thinking problems or biases.  For example, it is well known that people’s decision-making is highly influenced by information that stands out because it is easily remembered or has been heard repeatedly (accessibility heuristic) – as vivid anecdotes often are.  While the information in the opinion or anecdote may be true, the importance of information gets over-weighted in the decision-maker’s mind, relative to other information.  In management, this could lead to misguided strategic or tactical decisions that end up having less impact on the desired outcomes than expected.   Having information based on measurement, not simply stories, is one ingredient in helping to overcome our human thinking traps and make better decisions.

4.    WHY is measurement just “one ingredient” in making better decisions?

Measures don’t answer our questions on their own – they are just a (potentially) more objective source of information to use.  We are still subject to thinking traps even when using measures.  The other needed ingredient is that we establish and use our measurements in a way that helps us to learn. To learn, we need two more things:  (1) some idea(s) –  some theory – that describes what is and how we think things can be done differently to achieve the outcomes we desire; and (2) some predictions – hypotheses – that we can test in order to learn how well our theory helps us understand  what is important and where we might need to add to our knowledge base.    As W. Edwards Deming (one of the granddaddies of quality improvement) said:  “….without theory there is no learning”.

HOLD ON A MINUTE–you had my attention when you were talking about practical things–now you’re talking about THEORY and HYPOTHESIS TESTING?

5.    WHY are theory and hypothesis important to me?   (Isn’t THEORY only for university professors who have nothing better to do with their time?)

Deming also said:  “Knowledge is theory. We should be thankful if action of management is based   on theory. … Information is not knowledge. The world is drowning in information but is slow in acquisition of knowledge. There is no substitute for knowledge.” (Deming. The New Economics for Industry, Government, Education. 1993)

If measures (our “information”) are not selected and used within a framework of theory (i.e., an understanding of the ‘current state’, the desired ‘future state’ and what we believe will help us get there) then they don’t help us – rather they drown us in extra effort.  Hypotheses are important because they force us to actually use our theory to predict what our next course of action should be.  Learning happens when we do something – based on what our knowledge of our work processes tells us should help us get to the future state – and it either turns out as we predicted or it doesn’t.  Learning is actually deepest when our prediction was wrong and the action didn’t help in the way we thought it would.  This is because predictions that turn out to be wrong tell us that we have more to learn (improve our theory).

OK – so remind me how this relates to measurement in health care again?

We started this conversation with “Why…measure… in health care?”

After five WHYs we’ve got to the root of it: we measure in order to help us learn.  And as every good lean leader knows, management is ALL about learning and helping those we lead to learn.

If we find ourselves measuring something “because someone told us that’s what we should measure” – if we aren’t clear on how to use the measure to help us learn and move our work processes closer to  desired future state – we should get that clarity as quickly as possible or stop that measurement work.

For anyone interested in more on this topic I encourage you to read a recent open access online publication in the journal BMJ Quality and Safety called “More quality measures versus measuring what matters: a call for balance and parsimony”.

I want to hear from you. Submit a comment — I promise I’ll respond, whether you agree with me or not.  This is a topic we need to talk about more in health care.

What did you think of this post? Click below and let us know.

, , , , , ,

11 Responses to “WHY, oh WHY (oh why, oh why, oh why) do we need to measure?”

  1. Anonymous
    November 23, 2012 at 9:44 am #

    Thanks for the great overview on the importance of measurement. But, we must remember the system needs balance to move forward. The essence of process improvement is the PDCA cycle. More time on the “p & d” planning and doing part of the cycle is essential. Measurement and understanding are essential elements while “checking “ the significance of the results (both intended and unintended) and feeds back into the final round of “action” in the cycle. As with most systems, process improvement is best made while the cycle continues in motion. We need to be careful to not get overly focused on any one particular component of the process.

    • Gary Teare
      Gary Teare
      November 23, 2012 at 10:14 am #

      Hi Dan. You’re absolutely correct – measurement is only one ingredient in the PDSA cycle of learning. I didn’t focus on the whole cycle because the purpose of my blog posts in Qreview are aimed primarily at issues of measurement. This seems to be a particularly weak part of the learning cycle in health care.
      Thanks for reading and responding with a comment!

  2. Anonymous
    November 22, 2012 at 10:49 am #

    I think what is important about measurement is to keep it simple. It is also important to understand the different purposes of measurement. Measurement for research is important and brings all kinds of goof information on how we’re doing or what is going well or not going well. It’s down side is its lag. What we need more in healthcare transformation is measurement for improvement. Look at a problem, hypothesize what you think is happening and begin measuring using simple tools and constructs. Don’t worry about datasets and that peer-reviewed journal you want to publish in. Get some quick, almost perfect data that will help you respond quickly and make improvements in real time that will benefit patients. We need to say this is “good enough” and adjust.instead of dithering and making it perfect. Keep it simple and improve my patient experience sooner.

    • Gary Teare
      Gary Teare
      November 22, 2012 at 1:46 pm #

      Hi there Anonymous,
      Thanks for your post. You hit the nail on the head – there are a variety of purposes for measurement and we should only make it as complex as it NEEDs to be for the purpose intended. You’ve raised a measurement topic that was subject of one of my earlier blog posts on Qreview titled “KISS (Keep it simple, stupid) and tell”

  3. sherry stackhouse
    November 16, 2012 at 8:52 pm #

    “After five WHYs we’ve got to the root of it: we measure in order to help us learn.” Thank you Gary! Working in a community hospital, it is often difficult to defend, “Why do you spend all that time chart reviewing and giving us feedback”. Very little resources are given to staff to measure anything. I have shared your comments with our staff this week. We are making great strides in many areas, particularly sepsi, in large part to the immediate feedback staff receive on a case to case basis. Keep the good stuff coming!

    • Gary Teare
      Gary Teare
      November 22, 2012 at 2:06 pm #

      Hi Sherry,
      Thanks for your comment – and sorry for delay in getting your comment approved and up on the blog and giving my response to it.
      Keep up the great work on measuring and getting rapid feedback to your staff – you are helping them reflect on their practices/processes and learn. Engaging them in the selection of what to measure and involving them in building the measurement into their work is a next step you may want to take (if you’ve not already) – as that will put their learning on steroids!
      Cheers!

  4. Bruce Harries
    November 16, 2012 at 1:31 pm #

    Hi Gary,
    Thanks for this piece which provides detail of the important link between theory, prediction and learning.

    Here is a thought from a Deming Institute meeting that has stuck with me.

    “If you claim to know that only what can be measured can be known, then you are involved in a contradiction: for this very thing that you claim to know, namely that only the measurable is knowable, cannot be known by measurement.”
    Dr. John Edleman

    Would you consider revising the first answer and second question?

    Regards,
    Bruce

    • Gary Teare
      Gary Teare
      November 16, 2012 at 10:19 pm #

      Hi Bruce,
      Nice to hear from you! Thanks for your quote from the Deming Institute. I fully agree that measurement isn’t the only way of knowing. In fact when I talk about anecdote and opinion in the blog post I’m careful to say that stories can convey truth – and certainly stories are the most powerful way to convey knowledge and are useful to knowing – and – management.
      The specific point I was making in the first answer and second question was that management (of processes) needs measurement. Stories – or other forms of non-measurement information are also valuable and useful – but insufficient in and of themselves for management. (I’m also not advocating that measurement needs to be complex and involve advanced statistics – though for some purposes these are important too) So – while I still stand by what I wrote, I appreciate your bringing forth the clarification that we shouldn’t denigrate the knowing (knowledge, learning) that comes by other means other than measurement!
      Cheers,
      Gary

  5. Gary Teare
    Gary Teare
    November 14, 2012 at 9:52 am #

    Thanks for your comment Carolyn. Yours is a great example of where health care could greatly improve if it paid better attention to information that ALREADY EXISTS (like stats about heart disease in women).
    BTW – your myheartsisters.org web site is a great resource about women and heart disease.
    Best wishes,
    Gary

  6. Carolyn Thomas
    November 14, 2012 at 9:29 am #

    Excellent overview here – thanks for this, Gary.

    “People are highly vulnerable to several kinds of thinking problems or biases”. That’s just what Dr. Jerome Groopman says in his book “How Doctors Think”.

    One of the most disturbing examples of ‘thinking bias’ was a 2005 survey of U.S. physicians undertaken by the American Heart Association, who wanted to know how many doctors were aware that heart disease kills more women than men each year (a stat that’s been true since 1984, by the way, so hardly a news flash).

    The survey results were astonishing: only 8% of family physicians were aware of this fact. But even more appalling: only 17% of cardiologists were. CARDIOLOGISTS! This is their business. This is all they do.

    This ingrained belief that heart disease is essentially a man’s problem may help to explain why, according to research published in the New England Journal of Medicine, women heart patients are seven times more likely to be misdiagnosed and sent home from the E.R. compared to our male counterparts. This may help to explain why women’s cardiac outcomes are worse than men’s. This may help to explain why my E.R. doctor took one look at me and, despite my textbook ‘Hollywood Heart Attack” symptoms, pronounced: “You’re in the right demographic for acid reflux!” This may also help to explain why the type of heart attack I survived is one that doctors still call “the widowmaker”. (They’re not calling it the “widowermaker” now, are they?)

    I’m very hopeful that things are changing, as more women are now being included in cardiac research (sadly, a very recent development). But then maybe my hopefulness is just an example of optimistic opinion… 😉

Leave a Reply to Gary Teare Click here to cancel reply.

Share your opinions and questions about this post. We welcome different points of view; but ask that you be respectful and constructive.  Your comments will be moderated before they appear on the site. Your Email address will be kept private.