Webinar Notes: Analyzing Data Over Time for Quality Improvement

The Canadian Foundation for Healthcare Improvement (CFHI) is offering three webinars on Measurement for Quality Improvement. These are my notes from the third webinar: Analyzing Data Over Time for Quality Improvement: A Focus on Control Charts (I missed the second webinar and haven’t had time to go watch the archived video yet)

Speaker: Melanie Rathgeber

• recall from previous webinar on run charts
  • four rules for analyzing run charts from BMJ Quality & Safety 20(1), 46-51 (thought that webinar only talked about two)
  • can use the median from baseline OR you can use the median of all the points on your run chart (i.e., update median every time you a data point)

Control Charts

“Xbar chart for a paired xbar and R chart” by DanielPenfield – Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons.

• look like run chart
• has:
  • centre line (mean)
  • upper control limit (mean +3 SD)
  • lower control limit (mean – 3SD)
• can’t make these in Excel (need stats software that calculates the limit for you)
  • QI Macros Excel add on or QI Charts Excel add on allow you to do control charts
• not the same as confidence intervals
• no “p-value” in control charts – that answers a different question
• probability that we misinterpret conclusions (less than ~11% – depends on specific chart, but quite low)

• why would you use a control chart instead of a run chart
• if doing a QI project and want to know if we see improvement, we can use run charts
• control charts allow us to dig deeper (e.g., bigger QI project, lots of data points, running healthcare operations over time)
• control chart allows us to see variation
  • sometimes we have intended variation (e.g., we chose to do different things for some patients than others)
  • unintended variation is where we want to see an improvement
    • common cause variation (e.g., if you weigh yourself every day, you’ll see variation due to hydration level, etc.)
    • special cause variation (e.g., if you step on the scale after the Christmas holidays and you see you’ve gained weight)
      • special cause variation can be either desirable or undesirable
    • control charts are a bit more sensitive, pick up variation more quickly
  • things when you want to use a control chart instead of a run chart
    1. evidence of special cause variation (e.g., improvement)
    2. different sample size for each time period
    3. is system stable and predictable (e.g., data always within the control lines; if it’s not, remove special causes before trying to do improvement. when you do improvement projects, you want to have a stable system first)
    4. what will be the result next month?
    5. what are the sources of variation? (e.g., instead of time on x-axis, could have different departments and see if things vary by department)
  • analyzing a control chart
    • have to tell the software what type of data you have
      • e.g., percent data – P chart; count data – C chart; count data as a rate – U chart, averages (with subgroup size >1) – X bar and S; individual data or data that comes precalculated – I chart)
    • create control chart once you have about 12 data points
    • should have about 20 data points before establishing control limits and doing the analysis
    • finding special causes:
      • single point outside of control lines
      • shift: 8 or more consecutive points above or below centre
      • trend: 6 consecutive points increasing or decreasing
      • two of 3 points near a control limit
      • 15 consecutive point close to centre line
    • revise the limits as required – e.g., analyze data based on the initial state, but if you see an improvement where the data is shifted down, you’d then recalculate the limits to this new state (as you may want to see further improvement projects)

Speaker: Using Data with Healthcare Performance Improvement Initiatives

• healthcare fits the classic definition of a system: “a group of interacting, interrelated, or interdependent elements forming a complex whole” (IDC, 2012)
• even top-performing health care organizations are unlikely to acheive high quality in every aspect of their performance
• complexity of healthcare demands a robust approach to measuring and improving quality
• analytics allows us to see:
  • is our system in control?
  • is variation random or due to a special cause?
  • can we predict future results?
• many QI initiatives begin with a good idea, but they aren’t evaluated
• many QI methods are used in healthcare: PDSA (little experiments), Lean (reduce waste/non-value added activities, increase value added activities), Six Sigma (reduce variation/deviation in processes – define, measure, analyze, improve, control), Total Quality Management
• how do you decide which method to apply?
• how do you know how to analyze the data you get when using these different QI methods?
• linking data and analytics with QI methods
  • define problem and desire outcomes
  • determine appropriate metrics and indicators
  • build appropriate analytic tools
  • disseminate to stakeholderes

You can view a recording of this session ($69 for one session or $250 for all 3) on the CFHI website (not up yet, as the workshop was today, but should be up soon. ¹Disclosure: I have received a grant from the the CFHI in the past and am currently an “Improvement Coach” on one of their projects. I don’t receive anything for blogging about their workshop – I’ve just found them useful and wanted to share what I’m learning and I wanted to be transparent that I’ve received funding from them for projects.