Presenter: Christine Frank, PhD, CE
Hosted by: Canadian Evaluation Society
Date: 18 Nov 2013
- qualitative data are “words, actions, interactions, images”
- objective” exploring in-depth, finding out why….”
- research method:
- qual (explore & define concepts & issues) –> quant (measure, generalize, test theory) –> qual (ask why?)
- triangulation:
- methods
- data sources (different times, situations, roles, perspectives)
- investigator (data collection & analysis)
- theory
- trustworthiness (Lincoln & Guba, 1986)
- truth value: credibility
- applicability: transferrability
- consistency: dependability (how systematically did you collect your data? your competence as an evaluator)
- neutrality: confirmability (would someone else get similar findings?)
- how is a coding structure created?
- code references in sources into categories (themes)
- emergent/open coding: maximizes value of qual over quant; does not restrict thinking to pre-conceived notions, can use first to establish codes for a team; use after broad category coding)
- pre-determined: ensures focus on key research question; tends to be faster; greatly assists team analysis; very often start with coding by interview question and then emergent coding from there
- Types of codes:
- in vivo codes: participants’ actual words
- researcher-framed codes: may be from literature or program theory; may be term that best captures breadth of a category (e.g., other health problems)
- Challenges:
- too many codes – does not help reduce the data, you forget what the codes mean
- choosing how much text to include in a reference
- contradictory statements (could be one person contradicting self (might be code “mixed feelings”, but could be different participants or different stakeholder groups disagreeing – that’s good to surface)
- moderator feeds ideas to participants as they moderate
- cannot make sense of data
- NVivo software can do “reliability check” – but time consuming (evaluators often under very tight timelines compared with academics!)
- how can you quantify qualitative data?
- she resists it strongly – typically convinces people that it’s the wrong thing to do; reasons include:
- nature of qual data: people don’t talk in countable discreet ideas (if you want countable, you’d do a survey or other quant method – may do a two phase – qualitative to identify concepts, then survey to test theory); purpose of qual is to go deeper!; ideas evolved as people speak (how do you count that?)
- coding is subjective
- interviewer/moderator variances; timing issues
- with focus groups – people may just nod to what other said, may agree/disagree but not express, may be unequal numbers in groups; one person may dominate or make several points
- sample not representative (usually small & non-random)
- possible approaches to limited quantification
- she reports themes in order of frequency, but explains in report what that means (and does not provide numbers because of above reasons)
- create a scale for categorization responses (still depends on analyst’s judgement)
- report presence/non-presence of themes in sub-groups
- include survey component in focus group or community meetings (still non-random, small sample)
- for survey with open-ended questions, many researchers code & report frequencies
- systems such as Trochim’s concept mapping
- she resists it strongly – typically convinces people that it’s the wrong thing to do; reasons include:
