Spontaneous reports and electronic healthcare records for safety signal detection – yin and

yang

Presenter: Alexandra Păcurariu, MSc Pharm, a.pacurariu@erasmusm.nl

Authors: Alexandra C.Pacurariu1,2, Sabine M. Straus,1,2 Gianluca Trifirò,1,3 Martijn J. Schuemie1, Rosa Gini4, Ron Herings5, Giampiero Mazzaglia6, Gino Picelli7, Lorenza Scotti8, Lars Pedersen9, Peter Arlett10, Johan van der Lei1, Miriam C. Sturkenboom1, Preciosa M.Coloma1

1 Department of Medical Informatics, Erasmus University Medical Center, Rotterdam, the Netherlands; 2 Dutch Medicines Evaluation Board, Utrecht, Netherlands; 3 Department of Clinical and Experimental Medicine, University of Messina, Messina, Italy; 4 Agenzia Regionale di Sanità della Toscana, Florence, Italy; 5 PHARMO Institute, Utrecht, Netherlands; 6 Società Italiana di Medicina Generale, Florence, Italy; 7 Pedianet-Società Servizi Telematici SRL, Padova, Italy; 8 Department of Statistics, Università di Milano-Bicocca, Milan, Italy; 9 Department of Clinical Epidemiology, Aarhus University Hospital, Aarhus, Denmark ; 10 European Medicines Agency, London United Kingdom

Caveat!

§  Sparse  clinical  data  

§  Small  and  biased  popula2on  sample  due  to  underrepor2ng,  selec2ve  repor2ng  

§  No  real  denominator  ,  no  real  risk  es2mates  

§  Cover  a  wide  range  of  drugs  

§  Specifically  designed  for  ADR  collec2on  à  suspected  causality      

Spontaneous reporting systems vs. electronic healthcare records

§  More  detailed  clinical  data  

§  We  can  calculate  risk  es2mates    

§  Limited  number  of  drugs  

§  Not  specifically  designed  for  ADR  collec2on,  not  all  events  are  ADRs    

§  Eudravigilance  system  was  created  by  European  Medicines  Agency  (EMA)  in  2001  and  comprise  spontaneous  individual  case  safety  reports  of  ADRs  related  to  drugs  which  are  marketed  across  European  Economic  Area  (EEA).    

EU-­‐ADR  system  is  an  aggregate  of  eight  established  electronic  healthcare  record  databases  from  four  European  countries  (Denmark,  Italy,  the  Netherlands  and  United  Kingdom)  constructed  with  the  aim  of  early  detec2ng  drug  safety  signals.    

   §  To  inves2gate  how  a  SRS  and  an  EHR-­‐based  signal  detec2on  system  can  be  used  complementarily      §  To  iden2fy  specific  scenarios  where  they  can  provide  added  value  to  each  other.          Hypothesis  The    The  SRS  systems  are  beQer  in  detec2ng  rare  ADRs  with  a  high  likelihood  to  be  drug  induced  while  EHR  systems  will  be  beQer  in  detec2ng  events  with  a  moderate-­‐high  background  incidence.    

Objective

Methods §  The  2  databases  were  treated  as  independent  systems      

§  Period  -­‐  2000  to  2010  -­‐  cumula2ve  data  at  the  end  of  the  period  is  analysed    

 §   5  events  with  different  e2ologies  and  background  incidence  rate-­‐  acute  myocardial  infarc2on,  

bullous  erup2on,  acute  pancrea22s,  hip  fracture  and  upper  GI  bleeding      

§   We  did  not  restrict  to  common  drugs    

§   2  detec2on  methods  –propor2onal  repor2ng  ra2o  (PRR)  in  EudraVigilance  and  Longitudinal  GPS  (Bayesian  method)  in  EU-­‐ADR  

§  Benchmark  against  which  to  measure  performance-­‐  a  list  of  “known  ADRs”  with  evidence  in  the  scien2fic  literature**  

   

**  at  least  3  case  reports  of  higher  level  evidence        

Results §  The  total  number  of  drug-­‐event  associa2ons  inves2gated  for  all  events  was  5,049    §  1,490  poten2al  signals  were  flagged  in  either  EudraVigilance  or  EU-­‐ADR    §  Upon  signal  verifica2on,  the  ra2o  of  posi2ve  to  nega2ve  associa2ons  varied  from  1:6  for  pancrea22s  to  1:19  

for  hip  fracture.  

§  The  number  of  “known=true”  SIGNALS  in  the  reference  set    

Contribution of each system to signal identification (% of ‘known ADRs’ detected)

n=total number of true associations in the dataset; found in neither= the association was not highlighted as a signal in any of the screened databases during the signal detection process

Hypothesis

The  SRS  systems  are  beQer  in  detec2ng  rare  ADRs  with  a  high  likelihood  to  be  drug  induced  while  EHR  systems  will  be  beQer  in  detec2ng  events  with  a  moderate-­‐high  background  incidence.    

Correlation between background incidence of events and number of detected signals

0  

10  

20  

30  

40  

50  

60  

0.5   0.75   1   1.25   1.5   1.75   2   2.25   2.5  

Percen

tage  of  u

nilaterally  iden

2fied

 signals  (%)  

Background  incidence  of  the  events  of  interest  (log)  Eudravigilance    

Acute  myocardial    infarc2on    

Bullous    erup2on  

Hip    fracture  

Upper    gastrointes2nal    bleeding    

Acute  Pancrea22s  

The background incidences of the events, estimated from EU-ADR data, pooled across all databases are (per 100,000 person-years): bullous eruption=4.2, pancreatitis=21.4, upper GI bleeding=82.2, hip fractures=117.7, acute myocardial infarction =153.7. Identified signals refer to signals proven to be known ADRs; R= Spearman’s correlation coefficient

R=-­‐1,  P<0.  01*

Correlation between background incidence of events and number of detected signals

0  

10  

20  

30  

40  

50  

60  

0.5   0.75   1   1.25   1.5   1.75   2   2.25   2.5  

Percen

tage  of  u

nilaterally  iden

2fied

 signals  (%)  

Background  incidence  of  the  events  of  interest  (log)  

Eudravigilance    

EUADR    

Acute  myocardial    infarc2on    

Bullous    erup2on  

Hip    fracture  

Upper    gastrointes2nal    bleeding    

Acute  Pancrea22s  

The background incidences of the events, estimated from EU-ADR data, pooled across all databases are (per 100,000 person-years): bullous eruption=4.2, pancreatitis=21.4, upper GI bleeding=82.2, hip fractures=117.7, acute myocardial infarction =153.7. Identified signals refer to signals proven to be known ADRs; R= Spearman’s correlation coefficient

R=0.7,  P=0.18

R=-­‐1,  P<0.  01*

The costs associated with identifying potential signals The burden associated with screening any data source for signals depends on the number of signals that require further assessment or investigation and the workload involved in each of these investigations.

Workload per signal is highly variableà we used number of signals that needs to be investigated as a proxy for workload- number needed to detect (NND) It is more ‘costly’ to detect safety signals in EU-ADR than in EudraVigilance, with a median NND across all events of 7 versus 5.

§  Overall,  SRS  are  beQer  suited  to  detect  signals  than  EHR,  especially  for  certain  type  of  events  (rare  and  with  a  high  aQributable  drug  risk).    

§  Use  of  EHR  might  be  jus2fiable  in  some  situa2ons  where  SRS  perform  poorly,  provided  that  the  addi2onal  costs  can  be  taken  into  account.      

§  SRS  and  EHR-­‐based  signal  detec2on  systems  can  be  complementary,  the  value  of  one  to  the  other  varying  across  events,  as  a  func2on  of  the  background  incidence  of  event.    

Conclusions

Thank you for your attention