MISSING DATA

DSBS Meeting 28 May 2009

Kristian Windfeld, Genmab

ICH E9 (1998) on missing values

Try to avoid – by design (2.3)

Frequency and type must be documented in the CTR (5.2)

Imputation techniques (LOCF, …, complex math. models…) (5.2.1)

Are considered protocol violations (5.2.2)

Assess pattern of occurrence among treatment groups (5.2.2)

Source of bias (5.3)

Pre-specify methods for handling in protocol (5.3)

Analyze sensitivity of trial results to missing value handling method (5.3)

CPMP PtC on missing data (2001)

Effect of missing values on analysis & interpretation:

Power/variability (smaller sample size => smaller power but plausibly less variability => more power)

Bias – unless not related to unobserved real value (which cannot be verified)

CPMP PtC on missing data (2001)

Handling of missing data

Complete case analysis

Exploratory trials or as supportive analysis

Imputation

LOCF (”widely used”…”likely to be accepted if measurements constant over time”…”may provide an acceptably conservative approach”)

best/worst case imputation

multiple imputation

mixed models

CPMP PtC on missing data (2001)

Recommendations

Avoid missing data

Pre-specify handling method in statistical section of protocol with justification

Approach must be conservative

Approach may be updated in a SAP if unforeseen problems occur

Analyze missing values (number, timing, pattern, reason)!

Analyze sensitivity of results to missing value handling method

=> Everybody continued to do ITT with LOCF…

CHMP: Recommendation for revision of PtC on missing data (2007)

In many MAAs the handling of missing data is poor

Little or no discussion on missing data pattern…

Only one sensitivity analysis… or none with no justification

Misconception that LOCF is necessary and sufficient…

Need for cautionary note on mixed models and multiple imputation methods – their use still controversal

New EMEA draft guideline

GUIDELINE ON MISSING DATA IN CONFIRMATORY CLINICAL TRIALS (CPMP/EWP/1776/99 Rev. 1)

Released for consultation 23 april – 31 oct 2009

New EMEA draft guideline

Not much news… but some additional points:

Missing values violate ITT principle (collect data regardsless of protocol compliance)

Missing value taxonomy (MCAR, MAR, MNAR) – but since unverifiable => use conservative method

Use methods not assuming MCAR/MAR, ”pattern mixture, selection, and shared parameter models” mentioned

Specific suggestions for sensitivity analyses

Example: Missing ACR20 values from RA trial

Missing ValueHandling

Placebo 700 mg

N ACR20 N ACR20700mg/placebo

OR

Complete caseanalysis(no imputation) 46 13% 47 60% 10

0 (non-response)Imputation 55 11% 57 49% 7.81 (response)Imputation

55 27% 57 67% 5.5

Is non-response imputation conservative?

Key message

We need to start taking missing values seriously!

Imagine a news headline…

”Licensing application for obliviximab rejected because of the lack of analysis of the missing data…”

Do you want to be responsible?

Discussion questions

Why don’t we take the guidelines seriously when it comes to missing value handling?

How many sensitivity analyses are needed and should be planned?

When is the missing values influence so great that trial results become non-interpretable?

Is the potentially inflated precision resulting from single imputation methods a problem and if so, how may this be addressed?