Webinar Notes: “Issues in Qualitative Analysis and Reporting”

Presenter: Christine Frank, PhD, CE
Hosted by: Canadian Evaluation Society
Date: 18 Nov 2013

  • qualitative data are “words, actions, interactions, images”
  • objective” exploring in-depth, finding out why….”
  • research method:
    • qual (explore & define concepts & issues) –> quant (measure, generalize, test theory) –> qual (ask why?)
  • triangulation:
    • methods
    • data sources (different times, situations, roles, perspectives)
    • investigator (data collection & analysis)
    • theory
  • trustworthiness (Lincoln & Guba, 1986)
    • truth value: credibility
    • applicability: transferrability
    • consistency: dependability (how systematically did you collect your data? your competence as an evaluator)
    • neutrality: confirmability (would someone else get similar findings?)
  • how is a coding structure created?
    • code references in sources into categories (themes)
    • emergent/open coding: maximizes value of qual over quant; does not restrict thinking to pre-conceived notions, can use first to establish codes for a team; use after broad category coding)
    • pre-determined: ensures focus on key research question; tends to be faster; greatly assists team analysis; very often start with coding by interview question and then emergent coding from there
    • Types of codes:
      • in vivo codes: participants’ actual words
      • researcher-framed codes: may be from literature or program theory; may be term that best captures breadth of a category (e.g., other health problems)
    • Challenges:
      • too many codes – does not help reduce the data, you forget what the codes mean
      • choosing how much text to include in a reference
      • contradictory statements (could be one person contradicting self (might be code “mixed feelings”, but could be different participants or different stakeholder groups disagreeing – that’s good to surface)
      • moderator feeds ideas to participants as they moderate
      • cannot make sense of data
    • NVivo software can do “reliability check” – but time consuming (evaluators often under very tight timelines compared with academics!)
    • how can you quantify qualitative data?
      • she resists it strongly – typically convinces people that it’s the wrong thing to do; reasons include:
        • nature of qual data: people don’t talk in countable discreet ideas (if you want countable, you’d do a survey or other quant method – may do a two phase – qualitative to identify concepts, then survey to test theory); purpose of qual is to go deeper!; ideas evolved as people speak (how do you count that?)
        • coding is subjective
        • interviewer/moderator variances; timing issues
        • with focus groups – people may just nod to what other said, may agree/disagree but not express, may be unequal numbers in groups; one person may dominate or make several points
        • sample not representative (usually small & non-random)
      • possible approaches to limited quantification
        • she reports themes in order of frequency, but explains in report what that means (and does not provide numbers because of above reasons)
        • create a scale for categorization responses (still depends on analyst’s judgement)
        • report presence/non-presence of themes in sub-groups
        • include survey component in focus group or community meetings (still non-random, small sample)
        • for survey with open-ended questions, many researchers code & report frequencies
        • systems such as Trochim’s concept mapping



This entry was posted in data analysis, evaluation tools, webinar notes and tagged , , , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *