Risk Management and the Agile Approach...- Detailed - Specific ISO 14971 3.1: Mfr "shall establish, document and maintain throughout the life-cycle an ongoing process“ for analyzing,

3/31/2019

1

Risk Management and the Agile Approach

Brian Shoemaker, Ph.D.

ShoeBar Associates

SDMD Global 2019, 11.-14. February 2019

Who I am

SDMD Global 2019 11.-14. Feb. 2019

2

� Originally an analytical chemist

� 15 y in clinical diagnostics (immunoassay): analytical support → assay development → instrument software

validation

� 6 y as SW quality manager (5 in clinical trial related

SW)

� 13 y as independent validation consultant to FDA-

regulated companies – mostly medical device

� Active in: software validation, Part 11 evaluation,

software quality systems, auditing, training

3/31/2019

2

Acknowledgement

SDMD Global 2019 11.-14. Feb. 2019

3

Special thanks to Pat Baird, who has provided valuable material for this presentation, not only from his own workshops on risk management, but also by furnishing background on the planned updates to ISO 14971.

Thesis

SDMD Global 2019 11.-14. Feb. 2019

4

In the changing world, medical device risk management more than ever needs the flexibility of an Agile approach.

3/31/2019

3

Risk Management and Agile

SDMD Global 2019 11.-14. Feb. 2019

5

Public attention is focusing on risk management – in all fields

Risk work is inherently iterative

62304 requires us to "tailor" our approach

Reduce risk: decouple, reduce complexity

ISO 14971: broadened definitions; more risk/benefit emphasis

Flexibility of Agile: better risk management

Safety Incidents Make News (1)

SDMD Global 2019 11.-14. Feb. 2019

6

July 10, 2006:

concrete ceiling

panel in one of the

tunnels leading to

Boston Logan

Airport collapsed,

killing an auto

passenger and

seriously injuring

the driver. The

tunnel was closed

for almost a year.

http://archive.boston.com/news/specials/big_dig_ceiling_collapse/

3/31/2019

4


SDMD Global 2019 11.-14. Feb. 2019

7

The same blowout that caused the explosion also caused a massive offshore

oil spill in the Gulf of Mexico, considered the largest accidental marine oil spill

in the world, and the largest environmental disaster in U.S. history.

April 20, 2010: explosion

and fire aboard the

Deepwater Horizon offshore

drilling rig resulted in the

sinking of the Deepwater

Horizon and the deaths of

11 workers; 17 others were

injured.

https://slate.com/technology/2016/09/bp-is-to-blame-for-deepwater-horizon-but-its-mistake-was-actually-years-of-small-mistakes.html


SDMD Global 2019 11.-14. Feb. 2019

8

May 12, 2015: Amtrak

Northeast Regional train

from Washington, D.C.

bound for New York City

derailed and wrecked on

the Northeast Corridor in

the Port Richmond

neighborhood of

Philadelphia, Pennsylvania.

Of 238 passengers and 5

crew on board, 8 were

killed and over 200 injured,

11 critically.

https://www.nytimes.com/2017/05/12/us/amtrak-derailment-crash-philadelphia.html

3/31/2019

5


SDMD Global 2019 11.-14. Feb. 2019

9

September 13, 2018: Excessive pressure in natural gas lines owned by

Columbia Gas caused a series of explosions and fires to occur in as many

as 40 homes, with over 80 individual fires, in the Merrimack Valley,

Massachusetts towns of Lawrence, Andover, and North Andover.

https://www.usatoday.com/Feature/money/2018/09/15/massachusetts-explosions-

nisource-columbia-gas-linked-previous-blasts/1302447002/

Medical Devices Aren’t Exempt

SDMD Global 2019 11.-14. Feb. 2019

10

Headline:

Medical Devices

Harm Patients Worldwide As Governments

Fail On Safety

November 25, 2018: A global investigation reveals the rising human

toll of lax controls and testing standards pushed by a booming industry.

https://www.icij.org/investigations/implant-files/medical-devices-harm-patients-worldwide-as-governments-fail-on-safety/

3/31/2019

6


SDMD Global 2019 11.-14. Feb. 2019

11







Risk Management Is Central

SDMD Global 2019 11.-14. Feb. 2019

12

Quality System: ISO 13485 / 21 CFR 820

IEC 60601-1Medical Elect Eqpt – Basic Safety, Essential Performance

IEC 62304Medical Device SW

- Lifecycle

ISO 14971Med Dev Risk Management

IEC 62366Med Dev –

Application of Usability

Engineering

Risk Mgmt

3/31/2019

7

Risk Management in Brief

SDMD Global 2019 11.-14. Feb. 2019

13

1. What are you trying to do?

2. What can go wrong?

3. What are you going to do about it?

4. Did it work?

Source: Pat Baird; "Incorporating Risk Management into (software) Design & Development"; IQPC's 18th Software Design for Medical Devices; 10/27/2014

TIR 80002-1: Iterate!

SDMD Global 2019 11.-14. Feb. 2019

14

IEC 62304SW lifecyclerisk based

ISO 14971Med dev risk mgmt

IEC TIR80002-1SW risk mgmt

SW concerns, examples

… Iteration of RISK MANAGEMENT activities and coordination with SYSTEM design activities throughout the software LIFE-CYCLE … is advantageous to address any further HAZARDS as soon as they become apparent. This implies iteration within the implementation of the RISK CONTROL measure.

3/31/2019

8

Refine both features and mitigations

SDMD Global 2019 11.-14. Feb. 2019

15

Requirements

Requirements Hazards

Requirements

+ Mitigations

Early in project

- Preliminary

- High-level

- Approximate

Late in project

- Refined

- Detailed

- Specific

ISO 14971 §3.1: Mfr "shall

establish, document and

maintain throughout the

life-cycle an ongoing

process“ for analyzing,

evaluating, and controlling

risks.

Include RM in Emerging Documentation

SDMD Global 2019 11.-14. Feb. 2019

16

SRS

•Feature 1

•Feature 2

•Feature 3

•Feature 4

•Feature 5

•Feature 6

•Feature 7

V&V

SDS

Product

Hazards & Mitigations

3/31/2019

9

RM Lasts Through the Product Life

SDMD Global 2019 11.-14. Feb. 2019

17

Intended

Use Inherent risks, irrespective of design

Concept

Rqmts

Design

CodeTest

Demo

Design decisions introduce new hazards, new risk controls

RM matures, doesn’t freeze

Postmarket: customers use and misuse product!

Continue monitoring even after sales/service stop


SDMD Global 2019 11.-14. Feb. 2019

18







3/31/2019

10

62304: Manage RISK in Development

SDMD Global 2019 11.-14. Feb. 2019

19

Activities within ISO 13485 / 14971Customer

Needs

Customer

Needs Satisfied

SYSTEM development ACTIVITIES (including RISK MANAGEMENT)

7 Software RISK MANAGEMENT

8 Software CONFIGURATION MANAGEMENT

9 Software problem resolution

5.1SW

DevelPlanning

5.2SW

RqmtsAnalysis

5.3SW

Architectdesign

5.4SW

Detaileddesign

5.5SW Unit

Implem& verif

5.6SW

Integrn,Int Tstg

5.7 SW

SystemTesting

5.8

SWRelease

62304: Manage Risk in Maintenance

SDMD Global 2019 11.-14. Feb. 2019

20

Activities within ISO 13485 / 14971Maintenance

Request

Request

Satisfied

SYSTEM maintenance ACTIVITIES (including RISK MANAGEMENT)

7 Software RISK MANAGEMENT

8 Software CONFIGURATION MANAGEMENT

9 Software problem resolution

6.1Estab

SW MaintPlan

6.2Prob &

modificnanalysis

5.3SW

Architectdesign

5.4SW

Detaileddesign

5.5SW Unit

Implem& verif

5.6SW

Integrn,Int Tstg

5.7 SW

SystemTesting

5.8SW

Release

6.3 Modification Implementation

3/31/2019

11

Discussion: Examples?

SDMD Global 2019 11.-14. Feb. 2019

21

Class A: No injury or damage to health is possible

Class B: Non-serious injury is possible

Class C: Death or serious injury is possible

A risk management process – compliant with ISO 14971

- is required throughout development and maintenance

… BUT the type and extent of required activities /

documentation depends on the safety classification.

That is, you must tailor your use of 62304 based on

safety classification.

Agile must be tailored, too

SDMD Global 2019 11.-14. Feb. 2019

22

Credit: Ahmed Sidky, “The Agile Mindset”, available at http://www.softed.com/assets/Uploads/Resources/Agile/The-Agile-Mindset-Ahmed-Sidky.pdf

3/31/2019

12

Risk Management Fits in Agile Stories

SDMD Global 2019 11.-14. Feb. 2019

23

What to build

Estimate

Architecture

Risk Plans

Test Approach

QA Approach

Agile Story

All these evolve as a side-effect when the voices of Customer and Engineering bring a Story to maturity.

How about this?

7th SDMD Europe 21.-24. Feb. 2017

24

Hazard Mitigation:“As a caregiver,I want to ensure that therapy will stop if short, open circuit, or high impedance is detected,to avoid harming the patient.”

A “negative story” is perfectly valid for the development backlog.

3/31/2019

13

Risk Mgmt Fits in Agile Process

SDMD Global 2019 11.-14. Feb. 2019

25

DR Deploy

Each iteration has design, dev, test, demo ( )

Each demo an incremental design review

Consider risk issues in each demo / design review


SDMD Global 2019 11.-14. Feb. 2019

26







3/31/2019

14

Much to learn from system failures

SDMD Global 2019 11.-14. Feb. 2019

27

Key issues:• Complex systems: unlike linear (e.g.

assembly line), many different parts can interact, often in unpredictable ways. (Ex: nuclear power plants, chemical plants, modern autos, computerized stock trading systems, social media)

• Tight coupling: failure or change in one part of a system directly affects other parts, often quickly. (nuclear power plants, modern [computerized] auto systems)

The Danger Zone

SDMD Global 2019 11.-14. Feb. 2019

28

Complexity

Tig

ht

Co

up

ling

Nuclear power plants

Marine Transport

Assembly line production

Post offices

Dams

Chemical plants

R&D firms

Universities

Redrawn from Clearfield & Tilcsik, Meltdown: Why Our Systems Fail and What We Can Do About It

3/31/2019

15

Move Out of the Danger Zone

SDMD Global 2019 11.-14. Feb. 2019

29

• Break the system into manageable parts (cook turkey

and stuffing in portions, not all at once)

• Provide more visibility into internals of the system

(indicators, meaningful error messages)

• Conduct "premortem" analysis - note the similarity to

FTA

• Make major changes in small increments

• Record accidents and near-misses, and assess them for

root causes (ASRS; MAUDE)

• Have diverse, cross-functional teams making decisions

Taken from Clearfield & Tilcsik, Meltdown: Why Our Systems Fail and What We Can Do About It


SDMD Global 2019 11.-14. Feb. 2019

30







3/31/2019

16

ISO 14971 – updates coming

SDMD Global 2019 11.-14. Feb. 2019

31

Changes are a response to

• lower risk tolerance

• media coverage of incidents

• attention to security and privacy

• some cases of fraud

Jos van Vroonhoven, “Risk Management – Revision of ISO 14971 and Critical Issues for the Future,” presented at

AAMI/BSI International Conference on Medical Device Standards and Regulations, June 19-20, London UK

Planned Updates

SDMD Global 2019 11.-14. Feb. 2019

32

• Definitions: HARM changing; BENEFIT added

• REASONABLY FORESEEABLE MISUSE distinguished from

USE ERROR

• New Clause 2 on normative references (per ISO-IEC Directives)

• Corrections of minor (editorial) errors and inaccuracies

• More emphasis on benefits and benefit-risk balance

• Explanation of risks related to usability, security and privacy

• Clarification of requirements for overall residual risk evaluation

• Clarification of requirements for post-production phase

• Additional guidance (in companion report ISO TR 24971)

Jos van Vroonhoven, Risk Management – Revision of ISO 14971 and Critical Issues for the Future, presented at

AAMI/BSI International Conference on Medical Device Standards and Regulations, June 19-20, London UK

3/31/2019

17


SDMD Global 2019 11.-14. Feb. 2019

33







Risk – part of the Agile Mindset

SDMD Global 2019 11.-14. Feb. 2019

34

NOT this:

But this:

3/31/2019

18

Tailor to consider risks

SDMD Global 2019 11.-14. Feb. 2019

35

Credit: Ahmed Sidky, “The Agile Mindset”

With risk

management!

New Risk Information All the Time

SDMD Global 2019 11.-14. Feb. 2019

36

3/31/2019

19

References


37

Pat Baird; "Incorporating Risk Management into (software) Design & Development"; presented at IQPC's 18th Software Design for Medical Devices; 27-Oct-2014.

Chris Clearfield and András Tilcsik, Meltdown: Why Our Systems Fail and What We Can Do About It, New York, Penguin Press, 2018.

Ahmed Sidky, “The Agile Mindset”, available at http://www.softed.com/assets/Uploads/Resources/Agile/The-Agile-Mindset-Ahmed-Sidky.pdf

Jos van Vroonhoven, “Risk Management – Revision of ISO 14971 and Critical Issues for the Future,” presented at AAMI/BSI International Conference on Medical Device Standards and Regulations, June 19-20 2018, London UK

ANSI/AAMI/IEC 62304:2006, Medical device software – Software life cycle processes, 17-Jul-2006 (with ed. 1 revisions, Jun-2015).

ANSI/AAMI/IEC TIR80002-1:2009, Medical device software - Part 1: Guidance on the application of ISO 14971 to medical device software, 26-Oct-2009.

Contact information


38

Brian Shoemaker, Ph.D.

Principal Consultant, ShoeBar Associates

199 Needham St, Dedham MA 02026 USA

+1 781-929-5927

[email protected]

http://www.shoebarassoc.com

Documents

Risk Management and the Agile Approach...- Detailed - Specific ISO 14971 3.1: Mfr "shall establish, document and maintain throughout the life-cycle an ongoing process“ for analyzing,