SDTM Implementation - Best Practices

Pantaleo Nacci, Head Statistical Safety & Epidemiology/PV

Biometristi Italiani Associati (BIAS) Seminar:

CDISC, SDTM and ADaM: Moving from theory to practice

SAS Institute - Milano, 14 March 2014

SDTM Implementation - Best Practices

The story so far

Benefits of implementing SDTM

Basic requirements

Legacy data conversion: a missed opportunity?

The way forward

Conclusions

Agenda

The CDR Project

In 2010 Novartis Vaccines (NVx) initiated a very ambitious project to pool together all legacy studies for which data are electronically available in-house, as well as all new EDC trials, within the framework of a Clinical Data Repository (CDR)

A slightly adapted variant of CDISC SDTM 3.1.2 was selected as common standard for final storage, analysis and reporting, which allows storing of study metadata as well

All legacy study data are being remapped to this standard in two phases, and their metadata retrieved (mostly from protocols) and entered, for a total of around 400 studies

| BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices3

The CDR Project (2)

All EDC panels have been redesigned to collect data in a format compliant with CDASH standards, which allows easy remapping to SDTM

After a careful and lengthy selection process among three candidates, the solution chosen as the basis of the CDR has been SAS Drug Development (SDD) as a hosted environment

The re-mapping is done using custom-written SAS programs rather than SAS Clinical Data Integration (CDI), as initially planned


The CDR Project (3)

(Re-)coding of adverse events, concomitant medication, medical history, etc. is performed in Oracle Thesaurus Management System (TMS)

Our home-grown Standard Reporting Software suite of validated SAS programs was adapted to run in the new SDD environment AND to use the new CDISC data structures

Last but not least, all our existing processes were migrated to the new platform, and several new ones mapped and documented


Two years later...

More than two years after the CDR went live, all new clinical studies are now running completely in it• Data get pushed automatically into the system at regular intervals

• SDTM data are then obtained directly from CDASH dumps nightly, so that the latest version is always available online

• Deviations from the NVx standard are dealt with using a set of study-specific ‘rules’, and approval by a technical committee is necessary

Standard Reporting Software has finally been tested on real data, necessary fixes have been applied and new enhancements are actively being worked on as we speak

Special techniques are being developed to speed up operations in SDD


Two years later... (2)

Phase 2 of the Legacy Data Conversion sub-project has started, and data from 191 legacy studies, dating back as far as 1992, are now available for use, of course in addition to those from CDR-native ones

Utilities to create and maintain complete and ongoing dynamic ‘oceans’ have been fully validated, and jobs to maintain them are scheduled to run nightly

Other utilities to create static ‘data poolings’ have been validated too, and are available for use• Several data poolings have now been created and released for

analysis


CDR - The Full Picture


Clinical Data Repository -

SAS Drug Development

PV Safety database

Serology Lab

Submission Data sets

Scanned CRFs (ICRF)

Serology database

SAEs

eCRFs – EDCAdapter

TMS Coding (AE, MEDHX, CMED)

SAE coding *

•TGROUPS

•Lab norms

•Randomization

•Exclusions

•Stat. Populations Stat. Analysis

Periodic Updates

PV Safety reports

Clintrial

CDISC Data setsExt. Labs

CRO data, eDiaries

Rapid PoolingData Mining

Signal Detection.

Rapid PoolingData Mining

Signal Detection.

CTMS

Phasing out

Future wish:SAE ->EDC-> CDR-> PV DB

* Ideally all coding in 1 system

Monitor listings

Pooled analysis

Paper CRFs

The story so far


Basic requirements


The way forward

Conclusions

Agenda

Regulatory Requirements

It was in October 2007, while attending the PhUSE annual conference in Lisbon, that I heard for the first time about JANUS from an FDA speaker

The message was clear even then: the need to adopt non-proprietary data standards was not a question of ‘if’, but just of ‘when’

It took longer than expected, but on 6th February 2014 FDA officially published several guidance documents for electronic regulatory submissions, with a 90-day window for public comment• Various CDISC data standards (CDASH, SDTM, SEND, ADaM) are

specifically mentioned as required

10 | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

Effects on Submission Reviews

Even after adopting CDISC standard more pitfalls might be present in (therapeutic) areas not well characterized

E.g., given the degree of latitude still left to the single companies by the gaps in the current standards as regards vaccines data, an FDA reviewer might be faced with widely differing structures (and possibly codelists) representing the same data but coming from different companies• E.g., in vaccine trials solicited AEs can be stored in AE, CE, FA, etc.

This might result in longer times for review, even with the help of JANUS or similar systems


Evolution of the NVx Data StandardsHow an unstable reference can ultimately be a liability

It is now almost exactly two years since the first native CDISC study data entered the CDR

Only once that milestone was reached it was finally possible to fully test the reporting software, which had been developed using mock data• As expected, many issues, small and not so small, surfaced

The next slide show an overview of the changes applied to the initial production version of the NVx SDTM standard until June 2012• A few further changes, mostly adoption of new domains, have been

applied since then


Evolution of the NVx Data Standards (2)Adapting to changing requirements is a must


Evolution of the NVx Data Standards (3)How a pilot phase can save a lot of time later

In practice, these are a few examples:• Information which in the old data structures was contained in one

variable is now spread over several ones, e.g.:

- EDTTEST = ‘HI_B_BRISBANE08_CC’

- LBCAT=‘IMMUNOLOGY’

- LBMETHOD=‘HEMAGGLUTINATION INHIBITION’

- LBTEST=‘INFLUENZA B BRISBANE/2008 CC AB ‘• Re-assaying of samples, only identifiable using the date when an

assay was run, was not accommodated for until a new TSTDTC variable was added in SUPPLB


Usage of SDTM Data

As mentioned earlier, two data ‘oceans’, inherently dynamic, are maintained within it and have been used multiple times to create static ‘data poolings’• Complete ocean: contains all available data from studies which are not

active anymore; data for newly completed studies are added to those of existing ones

• Ongoing ocean: contains all available data for studies not yet completed, and is recreated from scratch every night

• All AE/MH verbatims can be updated to the latest MedDRA version in one go

These ‘data poolings’ are used to support submissions, but also papers, etc., i.e., whenever there is the possible need to go back to a known status to perform additional analyses


The story so far


Basic requirements


The way forward

Conclusions

Agenda

Basic Requirements - General

• The scope of such projects must be well defined at the beginning, so that all necessary functions are involved from early on

• Proper resourcing levels are needed from the beginning, and all interested systems must be identified and properly covered

• Think boldly when looking at the big picture and drawing your long-term plans, but don’t be ashamed to take only the steps you can afford at any given time

• It is extremely dangerous to have all knowledge on a certain topic concentrated in only one or two persons, and it can also create unnecessary bottlenecks at the wrong time

• A target reference framework must be clearly identified early on

• While it is possible to implement CDISC structures from scratch just looking at the documentation, choosing the right system will make it easier to implement, e.g., the inevitable tweaks linked to the peculiarities in your legacy data


Basic Requirements – Legacy Data

• The criteria to include/exclude studies and the overall timelines should be well understood and shared by top management to avoid creating excessive or plainly wrong expectations

• Putting study data together takes a lot of time, patience and attention to detail, even when most of the electronic data are already more or less in the right format

• Finding actual evidence about old studies can be a challenge (!)

• Original documentation (at least annotated CRF and protocol/CSR, including all amendments) must be available for all studies before the coding starts, to avoid misinterpreting the data

• Metadata are important as much as data themselves, so, e.g., make absolutely sure your T-domains are correct

• If at all possible plan for a pilot phase, it will most probably save a lot of time and grief later


The story so far


Basic requirements


The way forward

Conclusions

Agenda

What About All Those Studies in the Archive?

We decided that data from all new studies would be collected and stored according to CDISC standards, but this still left us with data from over 400 studies lying in our archives gathering dust

In this case the skill set required for the remapping was different, needing both good knowledge of the new CDISC standards and a lot of experience with the old one(s)

As described earlier, the first step was to decide which studies to convert first


Interpreting Old Data

A data inspection step was then undertaken, with the aim to identify all the legacy datasets and variables used

In our case we could further split the studies in pre- and post-Clintrial adoption:• Pre-Clintrial ones were managed by several different CROs in the

first half of the 90’s, so that today existing knowledge is extremely limited and fragmentary

• Post-Clintrial ones are on the contrary based on relatively minor variations of the current internal standard, which is known and understood by most members of the programming group

Study data were either re-extracted again from Clintrial or retrieved from our electronic archives


Necessary Tools

We correctly identified the need to retrieve all protocols, protocol amendments, (annotated) CRFs and CSRs from the archives, but underestimated the difficulty of getting them, especially the very old ones

The initial programming approach, agreed with our Quality partners, was based on random sampling of results, but a radical change to double-programming was requested well into the process • This caused a sudden shortage of experienced programmers


Remapping Your Existing DataAnother source of possible issues for your SDTM implementation

The LDC Phase 1 exercise, restricted to 153 high priority studies, took considerably longer than planned, but why?• Not all studies had available protocol and CRF at the time

programming started, forcing the team to a dangerous guessing game

• The NVD data standard was still evolving rapidly, so that some important assumptions proved grossly wrong when new studies were designed and run in the new system, sometimes requiring extensive rewriting of almost validated programs

• One domain in particular proved a nightmare, LB/SUPPLB, as several old test codes, as well as free text descriptions, were not included in the new so-called lab metadata tables


Remapping Your Existing Data (2)Still something you want to do?

Even with all these issues and delays, this is still one of the core business reasons why NVx started the CDR project

Safety signaling across vaccines, therapeutic areas and even single vaccine components can now be performed using data in the oceans • This is meaningful only for projects where all data are already

available in SDTM


The story so far


Basic requirements


The way forward

Conclusions

Agenda

Future DevelopmentsWhat’s next on the CDR plate

The current implementation of CDR is based on SDD 3.5, which does not exactly shine for speed• A new, revised version of SDD (4.3) has been out for a while, and we

have been getting a feel for it for some time

The set of available CDISC standards currently implemented is restricted to CDASH 1.1 and SDTM 3.1.2• Two main tasks have been identified here, upgrading to SDTM

3.1.3/3.2 and adopting ADaM 2.1

The Standard Reporting Software we use is still a revamped version of very old programs ported to SDD+SDTM• We are in the process of redesigning the reporting system from

scratch, embedding in it the creation of ADaM datasets


Future Developments (2)What’s next on the CDR plate

Until now we have remapped only part of the available legacy data• Lessons learned from the Phase 1 have been put to good use to re-

engineer the process

• Many things have changed to the better in the CDR environment:

- The reference standard is much more stable

- OpenCDISC consistency checks are readily available to check the progress all along

- Basic documentation has been retrieved for all remaining studies (no more guessing games!)

- Programmers know better both sets of data structures


What is Happening on the SDTM FrontLately CDISC has been very active too!

SDTM 3.1 was released on 14 July 2004, SDTM 3.1.1 on 26 August 2005 and SDTM 3.1.2 on 12 November 2008

After a hiatus of almost four years, during which more and more companies have adopted it, SDTM 3.1.3 has been finally released on 16 July 2012, followed by an utterly unexpected version 3.2 on 26 November 2013

• According to their previous roadmap, we should have seen SDTM 3.1.4 last July, and 3.1.5 roughly one year later

• An updated roadmap has been published in late February

Stated long-term aim is to have two updates per year, à la MedDRA


CDISC Planned Activities and ReleasesWhat we can expect for the next year


The story so far


Basic requirements


The way forward

Conclusions

Agenda

Conclusions

The old SAS server is still in service, but we are reducing the number of both business processes and users using it

We have plenty of exciting developments on our plate

CDR is a long-term project, which is evolving and will take years to reach a stable status

More and more activities are now moved completely to CDR

Safety signaling using all available data for selected vaccines/products is now possible, and by the end of 2014 this should be possible for most of our major products currently on the market


Conclusions (2)

Using capabilities already built in SAS we should be able to create necessary documentation for the submission data (e.g., define.xml) with much less effort than in the past, and with a higher quality

In the meantime SDTM is moving again

Changes to existing processes have always a cost, and resources are not always a viable alternative to time

It will be crucial to understand how much change is introduced by each new version, and calculate the business impact in each company scenario• It might make sense to skip some ‘minor’ versions


Useful Links

CDISC: • http://www.cdisc.org

• Technical Plan Project Schedule

SAS Clinical Standards Toolkit• http://support.sas.com/rnd/base/cdisc/cst/

FDA/PhUSE Computational Science Symposium 2014 • http://www.phuse.eu/css


Useful Links (2)

FDA:• FDA-2014-D-0092 - Study Data Technical Conformance Guide and

Data Standards Catalog

• FDA-2012-D-0097 - Guidance on Electronic Submissions: Standardized Study Data

• FDA-2014-D-0085 - Guidance on Submissions in Electronic Format--Submissions under the Federal Food, Drug, and Cosmetic Act


Questions?


Documents

SDTM Implementation - Best Practices