Mixed Methods Research in Practice: Decision Making about Life Support in Intensive Care Units Douglas B. White, MD, MAS Assistant Professor of Medicine

Mixed Methods Research in Practice:Decision Making about Life Support in

Intensive Care Units

Douglas B. White, MD, MAS

Assistant Professor of Medicine

Division of Pulmonary and Critical Care Medicine

Investigator, UCSF Program in Medical Ethics

Overview

What is qualitative research?

What is mixed methods research?

Criteria to evaluate the quality of mixed methods research

Examples of models of mixed methods research

Previously healthy 71-year man admitted to the ICU with a large stroke. He develops severe pneumonia w/ resp failure, sepsis and renal failure.

Aphasic, R hemiparesis APACHE II: 35; In-hospital mortality 70% Significant functional impairment Patient decisionally incapacitated

Should life support be continued?

Surrogate decision-making No clear “right” medical answer Preference-sensitive decision

Why study communication of prognosis?

1. Patients/Families have: A right to know

• autonomy & informed DM

A need to know• Prognostic info affects treatment choices

2. Prognostic misunderstandings are common

Lloyd CB. Crit Care Med 2004; 32:649-54

Prognostic Information Changes Patients Decisions about Life Support

I Shouldn't Have Had To Beg for a PrognosisWith all the conflicting reports on his health, I didn't know if he was holding steady or dying.

Aug. 22, 2005 issue - I was once a stalker. My victims—yes, there were several—were high on the social scale, but they were not celebrities. They were doctors.…

What causes physician-family discordance about prognosis?

No discussion about prognosis?

Poor MD communication skills?

Low health literacy/numeracy?

Undo optimism by families?

Lack of trust in physicians?

Different attitudes about whether clinicians can predict the future?

Different attitudes about what determines a patient’s prognosis

Little empirical research about mechanisms

The Structure-Process-Outcome Paradigm: Prognosis Communication in the ICU

Process of care:- # prognosis discussions- Process/Content of discussion

Outcome MD-family agreementre: prognosis

Family characteristics:- literacy/numeracy- optimism- depression-trust in physician- Explanatory models of future telling

Physiciancharacteristics:-Demographics-Skills - Attitudes

Goals of Doing Qualitative Research and the Alternatives

Goals of Research Develop hypotheses Develop framework for

understanding phenomenon

Understand multiple perspectives

Understand why patients or clinicians do what they do

Alternative to Research Make up hypothesis Use pre-existing framework

or conceptual model Use expert or clinical

perspective Guess why they do what

they do

Qualitative Methodologies

Grounded theory Content analysis Ethnography Naturalistic inquiry Discourse analysis Phenomenology

Unifying trait:Inductive reasoning “Moving from detailed facts to

general principles”

Deductive reasoning: “Reasoning from general

principles to the particular”

Ex.: Beta-blocker use in AMI

Questions begging for a qualitative approach

Why do patients not take their medications?

How do people make end-of-life decisions?

What are the barriers to implementing semi-recumbency to prevent pneumonia in ICUs?

A Quantitative or Qualitative Design?

Quantitative Establish incidence

and prevalence Measure risks and

frequency of events Determine treatment

effectiveness

Qualitative Describe phenomenon Understand thinking,

behavior Describe “why”

interventions do or don’t work

What is Mixed Methods Research?

“Research that includes both QUAL and QUANT data collection and analysis in parallel form (in which two types of data are collected and analyzed) or in sequential form (in which one type of data provides a basis for collection of another type of data).

Tashakkori and Teddlie; 2003, Handbook of mixed methods in social and behavioral research

Why Use a Mixed Methods Approach?

THE reason: the research question demands it. When neither method is sufficient to capture the

trends and details of a subject. Guide quantitative instrument development. Triangulate findings on 1 topic from 2

methodologic approaches. Comprehensive evaluation of an intervention.

NIH Guidelines for Rigor in Mixed Methods Research

1) What is the rationale for mixing methods? Why valuable for the study aims?

• To identify questions relevant for instrument design• To gain a comprehensive understanding of the

phenomenon• To triangulate findings (to enhance claims of

validity)

NIH. Qualitative Methods in Health . 1999 Creswell JW. Ann Fam Med. 2004; 2:7-12.


2) Which method takes priority? Depends on goal

• Develop instrument: QUAL informs QUANT goal of reliable and valid instrument.

• Develop theory: QUAL predominates; QUANT can provide useful ancillary data.

• Explain phenomenon: Equal weight to QUAL and QUANT inquiries.



3) How are the methods implemented? Concurrent vs Sequential

• Concurrent: simultaneous QUAL and QUANT data collection

• Sequential: One method serves as foundation for subsequent method.

– Qualitative data from semistructured interviews identify relevant domains for quantitative instrument.



4) When are the data integrated?• During data collection:

– Ex. 1: QUAL data examined, then transformed into QUANT instrument.

– Ex. 2: QUAL coding of audiotapes transformed into quantitative group of predictors to determine association with outcome.

• During synthesis of results: – Ex. 1: Investigators use their expertise to synthesize data

and draw inferences/conclusionsa.


Mixed Methods Designs: 1. Instrument Development

Create quantitative tools informed by qualitative methods

Pursue QUAL research to ensure comprehensiveness of instrument.

QUAL QUANT RESULTS

Assessing the Quality of Mixed Methods Research

1) Rationale for mixed methods: To inductively develop an instrument

2) Priority: Qualitative

3) Implementation: Sequential

4) Integration: During data collection


Mixed Methods Designs: 2. Explanatory Model

Qualitative data and analysis are used to explain quantitative data (e.g., patterns in the data, or the meaning of outliers)

QUAL

QUANT RESULTS

Example: Physician-family discordance about prognosis:

Misunderstanding vs Different Belief?

Background: Small qualitative studies in other domains of healthcare suggest that discordance might arise from different explanatory models of future telling.

Aim: To determine whether different belief systems contribute to MD-family discordance about prognosis.

Design: Prospective cohort study

Subjects: 175 surrogate decision makers for ICU patients with >50% chance mortality.



MEASUREMENTS:Quantitative data:

Surrogate’s estimate of patient’s prognosis for hospital survival

Surrogate’s estimate of what the physician thinks is the prognosis.

Qualitative data: Semi-structured interview: “I notice that your estimate is different from/the same as

what you think the doctor thinks. Can you tell me a little about why?”

0% chance of survival

100% chance

of survival

1.What do you think are the chances that the patient will survive this hospitalization if the current treatment plan is continued? Place a mark on the line…

0% chance of survival

100% chance of

survival

Outcome Measure- Prognostic Discordance

What do you think the doctor thinks are the chances that the patient will survive this hospitalization if the current treatment plan is continued? Place a mark on the line…



QUALITATIVE ANALYSIS: Interviews were audiotaped and transcribed Coding by multidisciplinary team Grounded theory methods

Performed open coding- line-by-line (5 transcripts) Multiple investigator meetings axial coding (grouping

individual codes into organizing framework) Coded 5 more transcriptsRevised codebook Reliability: kappa > 0.60 for all codes



Quantitative Results:

35% (50/143) held a different belief about the prognosis than their guess of the MD’s estimate.

Mean difference (absolute)=15%; (range 10-100%)



Qualitative Results:

Disagreement

PATIENT•Physical strength•Emotional strength“He’s a fighter”•Past history of “beating odds”

FAMILY•Miracles•Family presence can alter outcome•Conscious optimism•Need to disbelieve

PHYSICIAN•Overly pessimistic•Lack faith•Don’t know patient•Can’t predict future


1) Rationale for mixed methods: To explain an observed quantitative relationship

2) Priority: Qualitative

3) Implementation: Concurrent

4) Integration: During data collection and analysis


Mixed Methods Designs: 3. Triangulation Model

Pursue QUANT and QUAL methods in parallel Research: Study subject using more than one way of

knowing; more than one viewpoint Enhance comprehensiveness of findings.

QUAL QUANT

RESULTS

CONCLUSIONS

RESULTS

SYNTHESIS


1) Rationale for mixed methods: To gain a comprehensive understanding

2) Priority: Equal

3) Implementation: Concurrent

4) Integration: Analysis phase


Mixed Methods Designs: Data Transformation Model

Initial qualitative analysis of data to develop codes, then using descriptive statistics to describe how often they are observed

Transform QUAL findings into quantative and use

conventional statistical methods.

QUAL

QUANT(outcome)

RESULTS

QUANT (predictor)

Transform data

Example of Data Transformation Model: Emotional Support during ICU Family Conferences

QUAL: To understand how physicians emotionally support family members making EOL decisions for ICU patients.

QUANT: To determine whether specific physician behaviors are associated with higher family satisfaction.

Example: Emotional Support and Family Satisfaction during EOL Decision Making

Design:Design: Prospective multi-center, study of 51 Prospective multi-center, study of 51 audiotaped physician-family discussions about EOL audiotaped physician-family discussions about EOL decision-makingdecision-making

Setting:Setting: Adult ICUs of 4 Seattle-area hospitals Adult ICUs of 4 Seattle-area hospitals

Measurements: Measurements: QUAL: Transcripts inductively coded using grounded QUAL: Transcripts inductively coded using grounded

theory methodstheory methods QUANT: validated instrument assessing family QUANT: validated instrument assessing family

satisfaction with communicationsatisfaction with communication

Curtis JR. J Crit Care; 2000

Example: Emotional Support and Family Satisfaction during EOL Decision Making

Qualitative Results:Qualitative Results: Support for end-of-life decisionsSupport for end-of-life decisions Addresses family issues Addresses patient as a person Nonabandonment All is being done Reassures about comfort Support for hopes Acknowledges emotions expressed Acknowledges the complexity Offers assistanceOffers assistance Being direct about dyingBeing direct about dying Elicits questionsElicits questions ListensListens

Curtis JR. J Crit Care; 2000

Emotional Support Associated with Higher Family Satisfaction

Higher family satisfaction when MDs:• Expressed empathy (p <0.05)

– “I imagine it must be hard for you to see your father this way”• Assured nonabandonment (p<0.05):

– “Even if we stop life support, we won’t stop taking care of her.”

• Assured comfort (p<0.05): – “We’ll make sure she’s not SOB and not in pain.”

• Expressed support for family’s decisions (p<0.05)– “I fully support this plan and this is what I would do if it were

my son in the bed.”

Stapleton, Crit Care Med, 2006White DB. Under review 2007

The Effect of a Proactive Communication Strategy on Bereavement Outcomes

Design:Design: Multi-center, randomized controlled trial in 22 Multi-center, randomized controlled trial in 22 Adult ICUsAdult ICUs

Subjects:Subjects: 126 family members of patients whom MD 126 family members of patients whom MD thought would not survivethought would not survive

InterventionIntervention: : MDs trained to do the following:MDs trained to do the following:VValue the attitudes/decisions of the familyalue the attitudes/decisions of the familyAAcknowledge family members’ emotions cknowledge family members’ emotions LListenistenUUnderstand the patient as a personnderstand the patient as a personEElicit questions from the familylicit questions from the family

Measurements:Measurements: PTSD symptoms at 90 days assessed by IES PTSD symptoms at 90 days assessed by IES (range 0-75)(range 0-75)

Results:Results: Intervention group had significantly lower PTSD Intervention group had significantly lower PTSD symptom scores compared to control group (27 vs 39; p=0.02)symptom scores compared to control group (27 vs 39; p=0.02)

Lautrette A. NEJM 2007 ;356: 469-78

Conclusions

Mixed methods studies are particularly advantageous for some, but certainly not all, topic areas

“Grounds” our work, so that we ask the important questions, avoid omitting insights, and have realistic hypotheses.

Increases the potential that research will be more easily translated into practice

Documents

Mixed Methods Research in Practice: Decision Making about Life Support in Intensive Care Units Douglas B. White, MD, MAS Assistant Professor of Medicine