36
How to Organize Data How to Organize Data Collection For Collection For Registers Registers on ART on ART What Data and Why? What Data and Why? Istanbul. March 21, 2009 Istanbul. March 21, 2009 David Adamson, MD David Adamson, MD Director, Fertility Physicians of Northern Director, Fertility Physicians of Northern California California Clinical Professor, Stanford University Clinical Professor, Stanford University Associate Clinical Professor, UCSF Associate Clinical Professor, UCSF International Committee Monitoring ART International Committee Monitoring ART

How to Organize Data Collection For Registers on ART

Embed Size (px)

DESCRIPTION

How to Organize Data Collection For Registers on ART. What Data and Why? Istanbul. March 21, 2009 David Adamson, MD Director, Fertility Physicians of Northern California Clinical Professor, Stanford University Associate Clinical Professor, UCSF International Committee Monitoring ART. - PowerPoint PPT Presentation

Citation preview

Page 1: How to Organize Data Collection For Registers on ART

How to Organize Data How to Organize Data Collection For RegistersCollection For Registers

on ARTon ARTWhat Data and Why?What Data and Why?

Istanbul. March 21, 2009Istanbul. March 21, 2009

David Adamson, MDDavid Adamson, MDDirector, Fertility Physicians of Northern CaliforniaDirector, Fertility Physicians of Northern California

Clinical Professor, Stanford UniversityClinical Professor, Stanford University

Associate Clinical Professor, UCSFAssociate Clinical Professor, UCSF

International Committee Monitoring ARTInternational Committee Monitoring ART

Page 2: How to Organize Data Collection For Registers on ART

What Do We Need to What Do We Need to Know?Know?

1.1.Country/RegionCountry/Region

2.2.ART ClinicART Clinic

3.3.Patient demographicsPatient demographics

4.4.TreatmentTreatment

5.5.OutcomeOutcome

Page 3: How to Organize Data Collection For Registers on ART

Country/RegionCountry/Region

Why does it matter?Why does it matter?1.1. RaceRace2.2. Socioeconomic statusSocioeconomic status3.3. Cultural/religious uniquenessCultural/religious uniqueness4.4. Political agendaPolitical agenda5.5. HealthcareHealthcare

1.1. QualityQuality2.2. AccessAccess

6.6. Submission to ICMARTSubmission to ICMART

Page 4: How to Organize Data Collection For Registers on ART

ART ClinicART Clinic

Why does it matter?Why does it matter?

• Public vs. privatePublic vs. private

• Academic vs. non-Academic vs. non-academicacademic

• Large vs. smallLarge vs. small

• Urban vs. ruralUrban vs. rural

• Types of services Types of services providedprovided

Page 5: How to Organize Data Collection For Registers on ART

ART Patient ART Patient DemographicsDemographics

Why does it matter?Why does it matter?• Types of patients using ARTTypes of patients using ART• Factors that affect Factors that affect

pregnancy ratespregnancy rates• Types of interventions in Types of interventions in

different types of patientsdifferent types of patients• Different outcomes in Different outcomes in

different types of patientsdifferent types of patients

Page 6: How to Organize Data Collection For Registers on ART

ART Treatment & ART Treatment & OutcomesOutcomes

Why does it matter?Why does it matter?• Need to know which Need to know which

interventions are successfulinterventions are successful• Need to link intervention Need to link intervention

with patient type and with patient type and outcome to determine outcome to determine benefit of interventionbenefit of intervention

• Link with complicationsLink with complications

Page 7: How to Organize Data Collection For Registers on ART

What is Success?*What is Success?*

Percentage= Percentage=

NumeratorNumeratorDenominatorDenominator

*ICMART Glossary does not include specific measures of *ICMART Glossary does not include specific measures of “success”, which should take into consideration the “success”, which should take into consideration the wellbeing of babies as well as their mothers.wellbeing of babies as well as their mothers.

Page 8: How to Organize Data Collection For Registers on ART

Possible NumeratorsPossible Numerators1.1. Number of oocytesNumber of oocytes2.2. Number of 2 pronuclear zygotesNumber of 2 pronuclear zygotes3.3. Number of healthy embryosNumber of healthy embryos4.4. Number of embyros transferredNumber of embyros transferred5.5. Number of embryos cyropreservedNumber of embryos cyropreserved6.6. Chemical pregnancyChemical pregnancy7.7. Clinical pregnancyClinical pregnancy8.8. Fetal heartbeatFetal heartbeat9.9. Viable (ongoing) pregnancyViable (ongoing) pregnancy10.10.Third trimester pregnancyThird trimester pregnancy11.11.Term pregnancyTerm pregnancy12.12.Live birth(s)Live birth(s)13.13.Healthy child(ren)Healthy child(ren)14.14.Healthy child(ren) including frozen Healthy child(ren) including frozen

embryosembryos15.15.Healthy adult(s)Healthy adult(s)

Page 9: How to Organize Data Collection For Registers on ART

Possible NumeratorsPossible Numerators

The numerator that is chosen The numerator that is chosen usually depends on the interest usually depends on the interest of the person doing the of the person doing the choosing:choosing:

• PatientPatient• Infertility specialistInfertility specialist• ScientistScientist• ObstetricianObstetrician• PediatricianPediatrician• Policy-makerPolicy-maker

Page 10: How to Organize Data Collection For Registers on ART

Most appropriate Most appropriate numerator:numerator: Healthy Healthy

SingletonSingleton

•Best for patient and familyBest for patient and family

•Best for childBest for child

•Best for obstetricianBest for obstetrician

•Best for pediatricianBest for pediatrician

•Best for policy-makerBest for policy-maker

•? Best for infertility ? Best for infertility physicianphysician

Page 11: How to Organize Data Collection For Registers on ART

Problems With Healthy Problems With Healthy Singleton as Measure of Singleton as Measure of

OutcomeOutcome

•Who collects the data?Who collects the data?

•What is definition of What is definition of healthy?healthy?

•Who evaluates baby?Who evaluates baby?

•When is evaluation done?When is evaluation done?

•How long is the follow-up?How long is the follow-up?

•What about healthy What about healthy multiples?multiples?

Page 12: How to Organize Data Collection For Registers on ART

Possible DenominatorsPossible Denominators

• ART is not a single procedure1. Decision to pursue ART2. Ovarian stimulation (“Intention to treat”)3. Egg retrieval4. Fertilization5. Embryo transfer6. Implantation (chemical pregnancy)7. Fetal development (clinical vs. heartbeat)8. Viable pregnancy (>20 wks vs. > 26 wks)9. Live-birth delivery10. Patient (i.e. more than one cycle, +

frozen)

Page 13: How to Organize Data Collection For Registers on ART

Problems Using “Patient” Problems Using “Patient” as Denominatoras Denominator

• Women do different numbers of cyclesWomen do different numbers of cycles

• Some have frozen embryo cycles, some notSome have frozen embryo cycles, some not

• Variation in decision to pursue additional Variation in decision to pursue additional cyclescycles

• Different populations in each round of cyclesDifferent populations in each round of cycles

• Cycles performed differentCycles performed different– YearsYears– IntervalsIntervals– ClinicsClinics

• Important laboratory data may be overlookedImportant laboratory data may be overlooked

Page 14: How to Organize Data Collection For Registers on ART

USA DefinitionUSA Definition

Pregnancy Rate=Pregnancy Rate=

Live Births per number of Live Births per number of ovarian stimulation ovarian stimulation procedures initiatedprocedures initiated

Page 15: How to Organize Data Collection For Registers on ART

USA DatasetUSA DatasetClinicClinic

Clinic name, address, Clinic name, address, numbernumberName of laboratory used*Name of laboratory used*SART member ?SART member ?Services for single women ?Services for single women ?Donor egg available ?Donor egg available ?Donor egg sharing ?Donor egg sharing ?Total number of ART cyclesTotal number of ART cycles

Page 16: How to Organize Data Collection For Registers on ART

USA DatasetUSA DatasetPatient InformationPatient Information

Ethnicity, birth date, addressEthnicity, birth date, addressU.S. ResidentU.S. ResidentPrior pregnancy historyPrior pregnancy historySterilizationSterilizationMonths of infertilityMonths of infertilityPrior ART cyclesPrior ART cyclesFSH/E2 levelsFSH/E2 levels

Page 17: How to Organize Data Collection For Registers on ART

USA DatasetUSA Dataset

ART Cycle InformationART Cycle Information

•Reason(s) for ARTReason(s) for ART•Cycle start dateCycle start date•Suppression with GnRHaSuppression with GnRHa•Stimulation drugs & dosageStimulation drugs & dosage•Intended treatment proceduresIntended treatment procedures

–Fresh/frozenFresh/frozen–Patient/donor eggs/surrogacyPatient/donor eggs/surrogacy–IVF, GIFT, ZIFT,TETIVF, GIFT, ZIFT,TET–Cycle for embryo banking, researchCycle for embryo banking, research

Page 18: How to Organize Data Collection For Registers on ART

USA DatasetUSA Dataset

ART Cycle InformationART Cycle Information•Did cycle occur as Did cycle occur as intended?intended?

–Cancelled, date, reasonCancelled, date, reason–ComplicationsComplications–HospitalizationHospitalization

•Date of oocyte retrievalDate of oocyte retrieval•Number of oocytesNumber of oocytes•Semen source, collection Semen source, collection methodmethod

Page 19: How to Organize Data Collection For Registers on ART

USA DatasetUSA Dataset

ART Cycle InformationART Cycle Information

•Use of ICSI, hatchingUse of ICSI, hatching•Transfer attempt & dateTransfer attempt & date•Number of fresh Number of fresh embryos transferred, embryos transferred, cryopreservedcryopreserved•Number of thawed Number of thawed embryos transferred, re-embryos transferred, re-frozenfrozen

Page 20: How to Organize Data Collection For Registers on ART

USA DatasetUSA Dataset

ART Cycle Outcome ART Cycle Outcome InformationInformation

•OutcomeOutcome–Not pregnantNot pregnant–Type of pregnancyType of pregnancy

•UltrasoundUltrasound–DateDate–Number of fetal heartsNumber of fetal hearts

•Induced reductionInduced reduction

Page 21: How to Organize Data Collection For Registers on ART

USA DatasetUSA Dataset

ART Cycle Outcome ART Cycle Outcome InformationInformation

• Outcome of pregnancyOutcome of pregnancy

• Source of informationSource of information

• Number of infants bornNumber of infants born

• Birth weightBirth weight

• Neonatal Neonatal morbidity/mortalitymorbidity/mortality

Page 22: How to Organize Data Collection For Registers on ART

USA DatasetUSA DatasetART Cycle Outcome ART Cycle Outcome

InformationInformation Birth DefectsBirth Defects

• For each infantFor each infant– Pregnancy terminationPregnancy termination

• SpontaneousSpontaneous• ElectiveElective• Induced reduction (multifetal reduction)Induced reduction (multifetal reduction)

– StillbirthStillbirth– Live birthLive birth

• Categorization of defectCategorization of defect• Ascertainment issues and biasAscertainment issues and bias• Birth certificates unreliable*Birth certificates unreliable*• Later death and diagnosis ?Later death and diagnosis ?

*Gore et al. J Reprod Med *Gore et al. J Reprod Med 2002;47:297-3022002;47:297-302

Page 23: How to Organize Data Collection For Registers on ART

ART OutcomesART Outcomes

Birth DefectsBirth Defects

Confidentiality issues (e.g., HIPAA)Confidentiality issues (e.g., HIPAA)Linkage to other registries (e.g. birth registry)Linkage to other registries (e.g. birth registry)Definition inconsistencyDefinition inconsistencyData poor qualityData poor qualityMethodology weaknessesMethodology weaknessesAscertainment biasAscertainment biasLack of controlsLack of controlsStatistics complicatedStatistics complicatedWide confidence intervalsWide confidence intervalsStatistical vs. clinical significanceStatistical vs. clinical significance

Page 24: How to Organize Data Collection For Registers on ART

SummarySummaryWhat Data and Why?What Data and Why?

•Many factors influence what Many factors influence what data should be collecteddata should be collected•Need system to collect good Need system to collect good data (GIGO!)data (GIGO!)•STANDARDIZATIONSTANDARDIZATION

–Definitions (ICMART/WHO Definitions (ICMART/WHO glossary)glossary)–CollectionCollection–AnalysisAnalysis

Page 25: How to Organize Data Collection For Registers on ART

The Fertility Clinic Success The Fertility Clinic Success Rate And Certification Act Rate And Certification Act (FCSRCA) of 1992 (Wyden (FCSRCA) of 1992 (Wyden Law)Law)• Passed with support of SART and ASRMPassed with support of SART and ASRM

• Required:Required:– Annual reporting clinic-specific success ratesAnnual reporting clinic-specific success rates– Listing of clinics that do not reportListing of clinics that do not report– Development of model program for Development of model program for

certification of embryo laboratoriescertification of embryo laboratories

– Promulgation of criteria and procedures for Promulgation of criteria and procedures for approval of accreditation programs to approval of accreditation programs to inspect and certify labsinspect and certify labs

• First report published under the law in 1997 for First report published under the law in 1997 for 1995 cycles1995 cycles

Page 26: How to Organize Data Collection For Registers on ART

FCSRCAFCSRCAClinic-SART-CDC Clinic-SART-CDC

RelationshipRelationship• CDC contracts with SART to obtain annually a copy of CDC contracts with SART to obtain annually a copy of SART databaseSART database

• Clinics submit data to SARTClinics submit data to SART

• 380 of 400 (95%) clinics are SART members380 of 400 (95%) clinics are SART members

• All clinics that submit data to this CDC-supported SART All clinics that submit data to this CDC-supported SART system in compliance with FCSRCAsystem in compliance with FCSRCA

• SART keeps database of clinics (openings and closings)SART keeps database of clinics (openings and closings)

• Clinics must notify SART of personnel, address etc. Clinics must notify SART of personnel, address etc. changeschanges

• Requirements published in the Federal Register Requirements published in the Federal Register 2000;65:53310-53316.2000;65:53310-53316.

Page 27: How to Organize Data Collection For Registers on ART

FCSRCAFCSRCAData CollectionData Collection

• SART distributes Clinical Outcomes Reporting System SART distributes Clinical Outcomes Reporting System and instructions (SART-CORS) annually to clinicsand instructions (SART-CORS) annually to clinics

• Clinics abstract data from clinic records Jan 1-Dec 31Clinics abstract data from clinic records Jan 1-Dec 31

• Data entered using SART softwareData entered using SART software

• ART cycle starts withART cycle starts with– ovarian stimulatory drugs, orovarian stimulatory drugs, or

– ovarian monitoring with the intent of having embryos ovarian monitoring with the intent of having embryos transferredtransferred

• Data file organized with one record per cycleData file organized with one record per cycle

• Multiple cycles from one patient Multiple cycles from one patient notnot linked (HIPAA) linked (HIPAA)

Page 28: How to Organize Data Collection For Registers on ART

FCSRCAFCSRCAData Quality During CollectionData Quality During Collection

• Medical Director responsibleMedical Director responsible– Submit data on time (one year following year Submit data on time (one year following year

being reported)being reported)– Verify by signature that data are accurateVerify by signature that data are accurate– All patient identifiers removed before All patient identifiers removed before

submission to SARTsubmission to SART

• Prospective data submissionProspective data submission– Started with Year 2000 dataStarted with Year 2000 data– Within 3 days of cycle start (i.e. before outcome Within 3 days of cycle start (i.e. before outcome

known)known)– Demographic dataDemographic data– Internet basedInternet based

Page 29: How to Organize Data Collection For Registers on ART

FCSRCAFCSRCAData Quality During CollectionData Quality During Collection

• SART compiles data and submits to CDCSART compiles data and submits to CDC

– CDC CDC cannotcannot identify individual patients identify individual patients

• SART & CDC review, resolve inconsistenciesSART & CDC review, resolve inconsistencies

• SART compiles and submits SART compiles and submits final, correctedfinal, corrected dataset to CDCdataset to CDC

• Individual cycle data compiled and analyzedIndividual cycle data compiled and analyzed

– Report clinic-specific resultsReport clinic-specific results

– Aggregated for national reportAggregated for national report

Page 30: How to Organize Data Collection For Registers on ART

FCSRCAFCSRCAData Quality ValidationData Quality Validation

• Sample of 8-10% of clinics chosen for on-Sample of 8-10% of clinics chosen for on-site validationsite validation

• Randomly, Randomly, andand

• Pre-selected variablesPre-selected variables

– 50 randomly selected charts reviewed by 2-50 randomly selected charts reviewed by 2-person teamperson team

– CDC representative attends some visits as CDC representative attends some visits as observerobserver

– Error rates calculated Error rates calculated

– SART & CDC review findingsSART & CDC review findings

Page 31: How to Organize Data Collection For Registers on ART

FCSRCAFCSRCAData Quality ValidationData Quality Validation

• Data validation processData validation process

– Primarily educationalPrimarily educational

– Identify problem areas in data-collection processIdentify problem areas in data-collection process

– Correct general data collection problemsCorrect general data collection problems

– Requires individual clinics to correct problemsRequires individual clinics to correct problems

– Funded by government as of 1999Funded by government as of 1999

• Validation results to dateValidation results to date

– All error rates for all clinics acceptableAll error rates for all clinics acceptable

– Majority of errors minorMajority of errors minor

– Errors have minimal impact on success ratesErrors have minimal impact on success rates

Page 32: How to Organize Data Collection For Registers on ART

FCSRCAFCSRCAData Analysis and Data Analysis and

PublicationPublication • CDC primarily responsibleCDC primarily responsible

• Report co-authored by ASRM, SART, RESOLVEReport co-authored by ASRM, SART, RESOLVE

• Results published hard copy and websiteResults published hard copy and website

– National reportNational report

– Individual fertility clinic tablesIndividual fertility clinic tables

– Appendices of associated informationAppendices of associated information

• Detailed internal analysis and statistical Detailed internal analysis and statistical validationvalidation

• CDC editorial staff for final format and proofingCDC editorial staff for final format and proofing

• Publication by CDCPublication by CDC

Page 33: How to Organize Data Collection For Registers on ART

Additional Recent AdvancesAdditional Recent Advancesin USA Registryin USA Registry

• Model program for certification of Model program for certification of laboratorieslaboratories

• Publish scientific articles based on Publish scientific articles based on registry data (SART & CDC)registry data (SART & CDC)

• Obtain highest possible Obtain highest possible Confidentiality Status (308D): limits Confidentiality Status (308D): limits data access to CDC & SARTdata access to CDC & SART

• Bridge to SART CORS from clinic IT Bridge to SART CORS from clinic IT systemssystems

Page 34: How to Organize Data Collection For Registers on ART

What Have We What Have We Learned?Learned?• ART surveillance system is not staticART surveillance system is not static• ChallengesChallenges

– Data collectionData collection•Large number of clinics, coordination difficultLarge number of clinics, coordination difficult•Deadlines necessaryDeadlines necessary• Individual clinic variabilityIndividual clinic variability•Standardization of definitions and practice Standardization of definitions and practice

difficultdifficult

– Data presentationData presentation•Focus groupsFocus groups•Other feedbackOther feedback•Simplification and explanationSimplification and explanation

Page 35: How to Organize Data Collection For Registers on ART

The FutureThe Future• Surveillance system will continue to evolveSurveillance system will continue to evolve• Clarification of definitions and ambiguities (ICMART)Clarification of definitions and ambiguities (ICMART)• Collect better data on outcomes and confounding Collect better data on outcomes and confounding

variablesvariables• Stop collecting non-useful variablesStop collecting non-useful variables• Link ART cycles in same patient=cumulative live Link ART cycles in same patient=cumulative live

birthsbirths• Prospective reporting, validation, analysis, audit, Prospective reporting, validation, analysis, audit,

investigation, sanctionsinvestigation, sanctions• Focus onFocus on

– AccuracyAccuracy– Appropriate context for use of dataAppropriate context for use of data

• Fairness to clinicsFairness to clinics• Clarity to patientsClarity to patients

• Continued collaboration ASRM, SART, Clinics, Continued collaboration ASRM, SART, Clinics, RESOLVE, CDCRESOLVE, CDC

Page 36: How to Organize Data Collection For Registers on ART

THANKTHANK

YOU!YOU!