ISPE Quality Metrics Program –

An Update

Diane Hagerty, Genentech / Roche

Vice President, Global Quality Systems & Processes

17 September 2014

• ISPE Quality Metrics Program

• Overview

• Quality Metrics Industry Pilot

• Quality Performance Metrics

• Status

• Next Steps

2

ISPE Quality Metrics Program

To produce a “white paper” acceptable to industry on quality metrics,

which could be reportable to the FDA to support a risk-based inspection

program included in Sections 704 to 706 of US FDASIA.

The “white paper” should:

• Identify proposals targeted for discussion with the FDA and potentially with other

agencies.

• Justify why certain metrics are proposed, considered or rejected

• Justify the choice of site based and/or product-based metrics

• Comment on the relevance of and/or relationship to data already provided to

regulatory agencies, e.g., Field Alert Reports

• Be published on ISPE website

FIRST GOAL

G U I D I N G P R I N C I P L E S

• Clearly defined to allow consistent reporting across sites

• Objective and Meaningful

• Easy to Capture

• Easy to Report

• “Normalized” based on factors such as process differences

and technical complexity

Drive acceptable, not unwanted behaviors

ISPE Quality Metrics Program

W H I T E PA P E R

R E C O M M E N D AT I O N S

ISPE Quality Metrics Program

• Conduct a Pilot to ”flesh out” the definitions

and approach

• Start with site metrics, move to product

metrics later

>9,000 reportable sites (by FEI #)

>100,000 reportable product (by Application #)

April 16th 2014 ISPE

announces Pilot Program

• Partner with McKinsey for operational support

• Confidentiality of data

• Allow blinded comparison with industry ‘average’ and

technology platform peers

• Responsive to FDA’s recommendation for industry to

initiate activities

Quality Metrics Industry Pilot

6

Primary Objectives for the Pilot’s Initial

Phase

• Test a set of quality performance metrics based on

industry and FDA input to date

• Harmonize a set of metric definitions

• Test feasibility of data collection across different

companies

• Explore industry practices in quality culture and use

of process capability

• Inform continued industry input to the FDA

7

Input from Industry and FDA (Brookings Meeting, May 2014)

http://www.brookings.edu/events/2014/05/01-measuring-pharmaceutical-quality, The Brookings

Institution

8

Technology Specific metrics

Quantitative metrics

Media fill (for sterile aseptic sites) failures B

Environmental monitoring (for sterile aseptic sites) B

Lot acceptance rate B

Complaints rate (total B and critical

Confirmed OOS rate B

US recall events (total and by class) B

Stability Failure rate B

Invalidated (unconfirmed) OOS rate B

Right first time (Rework / Reprocessing) rate B

APQR reviews completed on time

Recurring deviations rate

CAPA effectiveness rate

Additional survey-based metrics

Process capability

Quality culture

M E A S U R I N G Q U A L I T Y P E R F O R M A N C E

Quality Metrics Industry Pilot Consensus Industry Metrics

B - metric proposed in Brookings meeting - Product- and site-based metric Metric set represents both leading and lagging indicators

Rationale for Choice of Metrics

• Quantitative metrics represent a balance:

• both leading and lagging indicators,

• measures of site, product and quality system performance

• Specific metrics for high risk operation – aseptic

filling

• Survey quality culture and process capability

• Culture obviously important, measurement not easy

• Process capability useful for identifying continual

improvement opportunities. No consistent view on

measures (CpK, PpK, etc).

10

June 2nd

ISPE-FDA

CGMP

Conference

Workshop

to explain

and launch

pilot

approach

Formal

check-in

with FDA

Timeline – Quality Metrics Industry Pilot

Pilot: Wave 1

▪ Confirm pilot participants

▪ Launch data collection

▪ Complete data collection,

clean/validate data

▪ Share initial findings and

correlations

▪ Individual report-outs to all

pilot participants

▪ Issue ISPE report on Wave 1

▪ Plan for Wave 2

June 2014 - Mar. 2015 2015+

Potenial

Wave 2

▪ Participate in Brookings

meeting to discuss FDA

direction

▪ Finalize metrics and

definitions

▪ Prepare data collection

templates and surveys

▪ Outline analysis

methodology

▪ Announce pilot details

Finalize metrics

and timelines

April-June 2014

11

Current Status – Quality Metrics Industry

Pilot

• 16 companies confirmed participation in Wave 1

• 38 sites

• Good spread of:

Technologies

Businesses

Regions

• Process established to gather input from participants

• Continuing informal interactions with FDA

• Regular updates on ISPE web site

12

Overview of the Sample: 38 Participating

Sites/Technologies1

15

7 5

3

8

API Bio DS Other Sterile Solids

17

9 10

3

Contract Cons. health

Gx Rx

19

12

3 4

NA Asia LA EMEA

By technology By business model

By region By company size

4

Small Large

34

1 If a site has more than 1 technology we count the number of separate templates they will fill, usually 1 per technology

• Influence the output from the Pilot program (choice of

metrics, definitions, data collection) based on actual

experience

• Have a blinded comparison to the overall industry

average and among technology platform peers

• Gain a head-start on enhancing internal procedures for

metric collection along with a set of metric definitions

• Develop insight into implications for metric

implementation

• Facilitate change management dialogue with internal

stakeholders

Quality Metrics Industry Pilot BENEFITS OF PARTICIPATING

14

• Sub teams continue to work on Quality

Culture Index and Process Capability

• Readout to Wave 1 participants 1Q 2015

• Input from participants to ISPE Report

• Discuss with FDA

• ISPE Report in 1Q 2015

• Inform Wave 2

15

Wave 1 of Pilot Program

Next Steps

Next Steps

• Continue to expand participation

• Assess a potentially adjusted metric set based on

learnings from Wave 1 and aligned with FDA

technical regulatory agenda

• Incorporate input from sub-teams working on

Quality Culture and Process Capability

• More focus on current data collection and analysis

• Continue to expand cooperation with other groups

and associations

Wave 2 of Pilot Program

16

In Summary – Quality Metrics Industry

Pilot

• Still enrolling for Wave 1

• Valuable insights already being

gained

• Excellent support and interest

from industry

• Ongoing dialogue with FDA and

other groups

• Follow us on the ISPE web site

• Send queries to

ispe_pilot@mckinsey.com

17

ISPE QUALITY METRICS CORE TEAM

MEMBERS

• Diane Hagerty (Team Leader) and Matt Pearson, Genentech/Roche

• Laura Cannon and Lorraine McClain, Teva Pharmaceuticals

• Michael Davidson, Pfizer

• Mairead Goetz and Lorraine Thompson, Novartis

• Peggy Speight, Bristol Myers Squibb

• Nuala Calnan, Dublin Institute of Technology

• Reem Malki, Mylan

• Paul Weninger, Perrigo

• Katy George, Vanya Telpis, Lorenzo Positano, Paul Rutten, McKinsey

• Chris Potter, ISPE Advisor

• Mary Storch, ISPE Regulatory Director