What hope, QI? We have an important legacy to live up to. Make no excuses!

I’m fortunate. Training as an Improvement Advisor (IA) with the Institute for Healthcare Improvement was a privilege. I hold the time learning from inspiring teachers as the beacon in all I do - an enlightenment. More so for my deepening learning and appreciation of Dr. W. Edwards Deming.

Creating momentum for a movement in the bastion of healthcare that holds on to outdated approaches to management and improvement is one of the challenges I accepted as part of being an IA. My own development focuses on transformation:

“The first step is transformation of the individual. This transformation is discontinuous. It comes from understanding of the system of profound knowledge. The individual, once transformed, will perceive new meaning to his life, to events, to numbers, to interactions between people.”
Dr. W. Edwards Deming

And then transforming leadership, teams and organisations.

To p-value or not to p-value, that is the question

When I see data presented sub-optimally, it upsets me.

What frustrates me more is seeing outputs that don’t achieve the standards we were taught. How can we collectively maintain the momentum and bring our peers to the same level if we undermine it ourselves? How can we be part of the transformation?

I confess. I’m classically trained in medicine and enumerative statistical studies. I’ve interpreted, presented and published in the paradigm of statistics that relies on the p-value. There is a place for this, but not in QI and related domains that deserve analytical and predictive statistics.

I had a day of reckoning when I was taught about analytical statistics, that based on the teachings of Walter Shewhart and W. Edwards Deming. The power of time-series data in the form of Statistical Process Control (SPC) charts being the most powerful and paradigm-shifting for me.

If it’s Quality Improvement (QI), it shouldn’t be enumerative. Show me a Pareto chart, a histogram or SPC chart. P-values figure less in my paradigm nowadays, I have better tools for statistical analysis.

If you do QI, do it properly, especially if you should know better!
When the experts can’t get it right, is it OK? We all can make mistakes, yet the question becomes is this a systemic issue rather than an incidental error?

What went wrong?

Recently, I was excited to see an article including a quality improvement perspective published. I had high hopes.

The published article presented an important problem to be solved. And, it included quality improvement perspective in the title.

What did I find in the article that was QI? Not a lot.

Three tables, the main outcome with only three subgroups.

Descriptive statistics.

P-values!

There was mention of QI tools in the article (Cause-and-Effect Diagrams, Pareto analysis, Plan-Do-Study-Act cycles) and use of the IHI Value Improvement approach (don’t get me started on this, just another faddish name for Quality Improvement - see my article here for more on this).

A measurement plan is described, listing outcome, process and balancing measures. Disappointingly, the default staff and patient satisfaction were the balancing measures. Run charts were described as displayed in a central location, but not within the publication.

With none presented in the publication, perhaps all could be saved by the online Supplementary Information? Perhaps saved from the wasteland of faux QI publications (although a waste not including the QI tools in the actual publication)?!

Nope. Not again?!

No Cause-and-Effect Diagrams or Pareto analysis available there, either.

Statistical Process Control to the rescue

We have an SPC chart.

Oh, hold on...........we don’t.

The Control Limits are missing, so we only have a Central Line (assuming that is what it is, as there is no label).

Next up, it appears there has been re-phasing as the Centre Line for Percent Readmission has reduced twice. Big problem is there is no rationale for the re-phasing. It does not appear there have been accepted statistical criteria achieved to justify the re-phasing. There might have been one downward shift at some point, but the chart is not robust as presented to confirm this.

Good point, though: there’s lot of annotation relating to the PDSAs performed. At least they didn't term it PDCA!

At least we have Run Charts

Whilst Run Charts are a step up from line charts, using SPC charts are the optimal choice (especially if trained in their production and use). Two Run Charts are presented on patient engagement and depression screening in the supplementary information. There is hope, yet.

Argh! Spoke too soon. These aren’t even Run Charts. There is no median (it might be a target) line. Should we even get into targets?!

Analytical prowess

One of the identified strengths of the study was the large sample size. However, no statistical power calculation was reported to reassure those looking at the analysis that the study was suitably powered.

Now I’m not sure there’s QI perspective or even robust enumerative statistics here?

Doris’ Journey

I just happen to be enjoying Doris Quinn’s book about her experiences with Dr Deming. Some pertinent quotes:

“Not understanding how to look at data over time can be a serious problem for organizations.”

“Dr. Deming said that training was one of the most important things senior management should invest in.”

As IAs, or similar, it is our duty to understand the most appropriate ways to look at data, which also involves training others. We are talking about transformation, so we need to be tough on ourselves and others to live up to the expectations.

The QI 'Cha Cha' Dance

One step forward, two steps back - cha cha cha - with personnel trained to high levels, but disregarding the tools diligently taught to them and diluted by enumerative statistics when reporting a QI project.

Is it any wonder organisations struggle to truly embed QI beyond projects? Those who have the knowledge to develop and share the knowledge fail to live up to their capability, and the capacity dwindles.

Have the knowledge? Use it!

Start with your own outputs, show the benefits to your team, leadership and beyond. Spread across your organisation sharing best practices for QI, data, analysis and visualisation.

As well as improving how we do Quality to avoid the 'Cha Cha' dance, let's embrace learning from other realms. One of my favourite resources for data visualisation is Storytelling with Data by Cole Nussbaumer Knaflic, and the follow-up books. The website is also fantastic.

References

I'm not telling. Quality Improvement perspective article.

Knaflic CN. Storytelling with Data: A Data Visualization Guide for Business Professionals. Hoboken (NJ): Wiley; 2015.

Knaflic CN. Storytelling with Data: Let’s Practice! Hoboken (NJ): Wiley; 2019.

Knaflic CN. Storytelling with You: Plan, Create, and Deliver a Stellar Presentation. Hoboken (NJ): Wiley; 2022.

Knaflic CN. Storytelling with Data: Before and After – Practical Makeovers for Powerful Data Stories. Hoboken (NJ): Wiley; 2025.

Quinn, Doris. A Journey for Quality. A Healthcare Professional’s Travels With Dr Deming.