Managing Tough Decisions With Decision Analysis INFORMS Southern California Chapter Meeting California State University, Northridge Phil Beccue

Managing Tough DecisionsWith Decision Analysis

INFORMS Southern California Chapter Meeting

California State University, Northridge

Phil Beccue

Decision Sciences INFORMS CSUN 2

What makes resource allocation decisions difficult?

Many needs,limited resources

A

B

CYr

Timing/Staging complexities

?

Risk/Uncertainty

Competing Objectives

Wall Street

Patient Benefit

Organizationimage/publicperception

Novelproducts

Long-termFinancial

value

Dispersed Information/Multiple Stakeholders/

Differing Opinions Projects that won’t die


Many companies use informal approaches when allocating resources which may not be effective

R I S K

Trust Us Approach

Ostrich Approach

DO NOT

DISTURB

Locked Door Approach

Squeeeak

Squeaky Wheel Approach

Dominant Personality Approach

FOR SALE

Washington Monument Approach

Fair Share Approach


Decision Analysis (DA) addresses the challenges of real-world decisions

• Provides a method to decompose complex problems

– Broad set of alternatives

– Specific agreement (up front) on criteria

– Explicit accounting of uncertainty

• Offers a set of tools and processes to bring clarity to the best course of action

• Based on foundational axioms of utility theory

• Is a prescriptive approach to decision-making applied to important, real-world problems


The Decision Analysis process provides a guide to think systematically about R&D decisions

• Decision analysis is a rigorous, transparent, quantitative approach for balancing the difficult tradeoffs inherent in R&D decisions.

• A formal process provides a common language for thinking and communicating about decisions within a multidisciplinary team.

StructuringModeling and Data Collection

EvaluationCommunication

and Integration

ActionActionDecision Problem


A case example illustrates the application of DA to a tough decision for a drug company

• Phase 2 trials for Leapogen to address an unmet medical need (jumping ability) are nearly complete

• High efficacy shown in a few of the 17 major potential clinical settings

• Management had differing views on the best way to proceed to develop and commercialize Leapogen

• Senior management review meeting in 2 months

• We started by carefully crafting a decision statement to keep the team focused:

How should we develop and commercialize Leapogen for the athletic jumping indication?


We took a comprehensive look at 8 strategies and included all significant issues• 8 well-defined Strategies• Approved Label

– Narrow– Medium– Broad

• Probability of Tech Success• Probability of Reg Approval• Launch Timing• Price• R&D Costs• S&M Costs• COGS

• Patient Population by Indication– Treated patients– Patient growth rate– Disease Severity– Therapeutic Penetration– Competition– Market Share

• Marketing Focus


Ph 2

Ph 3

None

Ph 2

Ph 3

INV IND

None

Ph 2

Ph 3

INV IND

None

Ph 2

Ph 3

INV IND

None

Ph 2

Ph 2/3

Ph 3

INV IND

None

Ph 2

Ph 3

INV IND

None

Ph 2

Ph 3

INV IND

None

Strategy 1 $128M

Strategy 2 $140M

Strategy 3 $ 82M

Strategy 4 $ 40M

Strategy 5 $120M

Strategy 6 $ 103M

Strategy 7 $ 25M

Strategy 8 $180M

A strategy table narrowed the feasible alternatives to 8 clearly defined development strategies

Strategy Name Dev Cost

ClinicalSetting A

(Basketball)

Clinical Clinical Clinical Clinical Clinical ClinicalSetting B(Soccer)

Setting C(Volleyball)

Setting D(Long Jump)

Setting E(High Jump)

Setting F(Hurdles)

Setting G(Rugby)


We carefully defined 3 label outcomes for each strategy by specifying the clinical setting included

Narrow Medium BroadS1 adefmn cdefmnopq bcdefghijklmnopqS2 adefmn defmnopq bcdefghijklmnopqS3 adefmn defmnopq bcdefghijklmnopqS4 adef adefmnoq adefmnopqS5 def cdef cdefmnS6 defmn cdefmnopq abcdefghijklmnS7 mn mno mnoqS8 adefmn adefmnopq abcdefghijklmnopq

Label

Str

ate

gy

KEYa – basketballb – soccerc – volleyballd – long jumpe – high jumpf – hurdlesg – rugbyh – balleti – gymnasticsj – figure skatingk – …


The decision tree specified the scenarios to be analyzed for each strategy

The model calculated NPV for over 4 million scenarios!

Low

Total NPV Nom

Total NPV High

Total NPV

Low

Nom

High

DiseaseSeverity

Low

Nom

High

TherapeuticPenetration

Region 1

Region 2

Region 3

Region 4

MarketShare

Early

Late

MarketingFocus

Narrow

Medium

Broad

CompetitionTiming

Yes

Label

No

Pay R&D Costs

Yes

RegulatoryApproval

No

Pay R&D Costs

Yes

TechnicalSuccess P3

No

Pay R&D Costs

S1

S2

S3

S4

S5

S6

S7

S8

TechnicalSuccess P2

LeapogenStrategy


An influence diagram defined the key input requirements to compute NPV

DecisionUncertaintyValue

COGSDemand

NPV

Cost

Revenue

S&MCost

DevCost

Price

PatientsTreated

Dose perpatient per

cycle

Marketshare

Competition

Today'sPts receiving

treatment

Cyclesper patient

COGSper gm

GrowthRate

Futuretrends for treatment

Patientpopulation

Pts treatedwith anytherapy

TechnicalSuccess

Label

LeapogenStrategy

MarketingFocus


We specified uncertain inputs using probability distributions

10% of area10% of areaPro

bab

ility

500 1000 2000

Clinical Disease Severity Patients TreatedSetting Lo Nom High Lo Nom High

A 3% 18% 30% 3,200 4,400 7,500 B 3% 18% 30% 10,500 20,600 28,000 C 10% 15% 30% 1,500 4,800 7,000 D 5% 8% 10% 6,000 10,000 12,000 B 60% 80% 90% 6,000 9,000 12,000 E 20% 40% 80% 4,000 5,600 7,800 F 1% 5% 8% 13,000 19,000 25,000 G 10% 20% 35% 15,000 17,100 34,000 H 6% 9% 12% 45,000 66,000 90,000


Technical success probabilities were assessed through structured conversations and compared to industry benchmark data

Approved

Not Approved

Success

Approval

Failure

Success

P3

Failure

Success

P2

Failure

P1

.60

.91

.73

.46

New, Active Substances (NAS)CMR Int’l Data, 1997

Launch


The probabilities of technical and regulatory success varied by strategy

Narrow Medium Broad 1 30% 100% 90% 5% 2 30% 100% 95% 5% 3 35% 100% 95% 5% 4 25% 0% 100% 95% 5 30% 100% 30% 0% 6 15% 100% 90% 10% 7 40% 100% 90% 0% 8 20% 100% 95% 0%

Strategy

ProbabilityTechnicalSuccess

Probability ofRegulatory Success

Given Label


0%

10%

20%

30%

40%

50%

-100 -50 0 50 100

Expected NPV ($M)

Pro

bab

ility

of

Tec

hn

ical

Su

cces

s

The top strategy (#4) provides over $150M additional value than the status quo (#8) strategy

4

8

3

6

7

2

5

1

Size of bubble is proportional to expected peak sales.


NPV ($M)

Cu

mu

lative

Pro

ba

bility

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-100 -50 0 50 100 150 200 250 300 350

The expected NPV for Strategy 4 is $10M, including all technical and commercial risks

There is a 30% chance of launch

There is only a 10% chance of getting an NPV greater than $100M

Probability-weighted average = $10 million


Assuming technical success, peak year revenue could vary from $100 to $250 M, with expected sales of $150M

Peak Sales ($M)

Pro

ba

bili

ty D

en

sity

75 100 125 150 175 200 225 250 275 300 325

ExpectedValue


Competition

Peak Share Basketball

Label

Severity of Disease Long Jump

Peak Share High Jump

Peak Share Long Jump

Severity of Disease Long Jump

Peak Share Soccer

Severity of Disease High Jump

Peak Share Volleyball

Peak Share Hurdles

Severity of Disease Volleyball

Peak Share Gymnastics

Peak Share Ballet

0 50 100 150 200 250 300 350 400

Peak Sales ($M)

The key drivers of risk for Strategy 4 are Competition, Peak Share for Basketball, and Label

Assumes technical and regulatory success

The base value ($270 MM) is calculated by setting all uncertain inputs to their base case.

Each bar in the tornado diagram shows the impact on commercial value of moving one uncertain input across its range of uncertainty while holding all other inputs to the base case.


Revenues for Strategy 4 are uncertain, and key contributors of value are clinical settings A, D, and E

0

50

100

150

200

250

2000 2002 2004 2006 2008 2010 2012 2014 2016

Su

m o

f R

even

ues

($M

)

-

50

100

150

200

1

Strategy 4

Pea

k S

ales

($M

)

A C D E F M N O P Q


Risk/return tradeoffs for each strategy were made explicit

Peak Sales ($M)

Pro

bab

ility

Strategy 8

Strategy 4

Strategy 7

0 50 100 150 200 250 300 350


Strategy 3 will provide positive value if the chance of success exceeds 50%

-80

-60

-40

-20

0

20

40

60

80

100

0% 20% 40% 60% 80% 100%

Probability of Technical Success for Strategy 3

Ex

p N

PV

($

M)

NominalProbability = 25%


The optimal marketing focus depends on the outcome of future uncertainties

Region 1

Region 2

Region 3

Region 4

Early

MarketingFocus

Region 1

Region 2

Region 3

Region 4

Late

Narrow

CompetitionTiming

Region 1

Region 2

Region 3

Region 4

Early

Region 1

Region 2

Region 3

Region 4

Late

Medium

CompetitionTiming

Broad

S1

Label

S2

S3

S4

S5

S6

LeapogenStrategy

Bold line indicateshighest NPV path


Key insights from the strategic analysis

• Strategy 4 (a focused strategy) has best overall value

• Peak sales is $150 million

• Optimal marketing focus should be determined closer to launch

• High value indications (% of total value):

– Clinical setting A (62%)

– Clinical setting E (26%)

– Clinical setting D (12%)

• Chance of success = 30%

• Clinical setting G is not as critical as once thought


The initial DA had a significant impact at Amgen

• Jumping ability is a viable indication for Leapogen• Structuring the complex set of development options provided

direction and a clear development plan• Senior management agreed to follow the recommended

strategy of a focused program• The recommended strategy gave $150M additional value over

the status quo strategy• Key drivers were identified as needing further investigation;

senior management asked to update the decision analysis in the future

• A few months later, the team asked our group to translate the decision model into a forecasting tool for ongoing use

• We have performed additional strategic projects for Leapogen based on the initial work


It is critical to keep management informed at each step of the process

Action !Action !

Decision BoardDecision Board

Decision Analysis TeamDecision Analysis Team

Information ExpertsInformation Experts

Time

ProblemStructuring

Modeling/Data

Collection

Evaluation Communicationand

Integration


The benefits of using decision analysis should be weighed against the costs

• Documents the decision process

• Information collection is focused and efficient

• Helps resolve conflicts and debates

• Fewer “surprises” because uncertainty is explicitly considered

• Avoids common pitfalls in analyzing complex situations

– solving the wrong problem

– analyzing what is known rather than what is important

– getting lost in the process

• More time required of decision-makers

• More time for analysis team

• Possible discomfort with new process

• Reveals logic of decision

– confidential information

– lack of knowledge

– embarrassing motivations

Benefits Costs


The significant problems we face cannot

be solved at the same level of thinking

we were at when we created them.

- Albert Einstein