Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Eindhoven University of Technology
MASTER
NPD portfolio managementthe development of a reference model for Kalypso
De Mulder, M.L.W.
Award date:2011
Link to publication
DisclaimerThis document contains a student thesis (bachelor's or master's), as authored by a student at Eindhoven University of Technology. Studenttheses are made available in the TU/e repository upon obtaining the required degree. The grade received is not published on the documentas presented in the repository. The required complexity or quality of research of student theses may vary by program, and the requiredminimum study period may vary in duration.
General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain
Eindhoven, January 2011
BSc Mechanical Engineering
Student identity number 0597664
In fulfilment of the requirements for the degree of
Master of Science
in Innovation Management
Eindhoven University of Technology
First Supervisor: Prof. dr. F. Langerak, TU/e, IE & IS
Second Supervisor: dr. ir. I. Reymen, TU/e, IE & IS
Kalypso
First Supervisor: Laurens Broekhof
NPD Portfolio Management
The Development of a Reference Model
for Kalypso
by
Maarten De Mulder
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder I
Preface This master thesis is the result of my graduation project conducted at Kalypso. The
project is the conclusion of the master program Innovation Management at the
Eindhoven University of Technology (TU/e). The project was supervised by the
department of Industrial Engineering & Innovation Sciences (IE&IS).
For me, this journey has come to an end. I look back with great excitement on my
master thesis project and, in particular, the period spent at Kalypso. It has been a
period of great personal development both academically and professionally. None of
this would have been possible without the help of so many people.
First of all, I would like to thank my university supervisors, Prof. dr. Fred Langerak
and dr. Ir. Isabelle Reymen for their critique and support. Especially the guidance of
Fred Langerak has been very helpful throughout the project and the preliminary
literature study.
I am also very grateful to Laurens Broekhof, my supervisor at Kalypso, for his
guidance and fast response to my many questions. In addition, I want to thank Mick
Broekhof, he has been a very insightful source of knowledge and experience. From
the TU Delft I would like to thank Ir. Linda Kester for her very thorough critique and
comments.
Finally, I am grateful to everybody who supported me – intellectually, financially, and
mentally – during my graduation project and the rest of my study. These people
include my family, who had so much patience with me, my girlfriend and my great
friends, who were always there for distraction and memorable moments.
Maarten De Mulder
Amsterdam, January 2011
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder II
Management summary
Introduction Developing the “right” new products is critical to a firm’s success and is often cited as
the key to a sustainable competitive advantage (Chao and Kavadias, 2007). New
Product Development (NPD) portfolio management enables organizations to decide
which projects to pursue and which to abandon. There are currently various NPD
portfolio management methods, and each method has its own advantages and
disadvantages. Consequently, none of these methods are suited to be used as the
single and only method in an organization.
Goal Kalypso developed an NPD portfolio management model which is specifically
tailored to one of their clients. The model is designed around client specific metrics
and criteria which makes it difficult to implement the model in other organizations.
The goal of this master thesis is:
To develop a reference NPD portfolio management model that enables users to make
the best possible NPD portfolio decisions (1). This reference model should be
scientifically substantiated and practically applicable (2). The NPD portfolio
reference framework should enable Kalypso to implement NPD portfolio management
solutions at any client (3).
NPD portfolio management assessment tool Based on scientific literature, a list of criteria for NPD portfolio management is
composed (table 2.9, page 23). These criteria are subdivided into seven categories.
The list of criteria forms the basis of the assessment tool. An NPD portfolio
management method receives a score of 1 (poor) to 7 (excellent) for each criterion.
The overall score is calculated by the average of each criterion score. The assessment
tool is used to assess various NPD portfolio management methods e.g., financial
methods, scoring models and strategic methods. None of these methods received a
score high enough to be considered the single and only NPD portfolio management
method in an organization.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder III
Table I Average scores various NPD portfolio management models Assessment score Financial methods 3,09 Probabilistic financial methods 3,48 Strategic Methods 3,98 Checklist and Scoring methods 3,73 Mapping approaches or bubble diagrams 3,50 Poor 1 2 3 4 5 6 7 Excellent
The Original Kalypso Model In the original Kalypso model, projects are assessed individually on 3 value drivers;
strategic fit, financial and risk. Strategic fit, risk and, in the first two stages of the
development process financial, are assessed with qualitative criteria. In later stages
the financial assessment is based on a quantitative metric. Based on these 3 variables
a single score is calculated. This score enables users to rank and prioritize projects.
The original Kalypso model is assessed with the NPD portfolio management tool;
table II shows the results of this assessment.
Table II Average scores Kalypso model per category
1. General Criteria 5.71
2. Usability Criteria 5.17
3. Process Criteria 4.71
4. Balance Criteria 3.75
6. Financial Criteria 3.11
5. Risk Criteria 3.00
7. Strategic Criteria 2.75
Overall score 4.03 Poor 1 2 3 4 5 6 7 Excellent
The Kalypso model has a high score on several criteria such as: general, usability and
process criteria. E.g., the Kalypso model uses multiple types of methods for the
valuation of projects. However, the assessment of the various value drivers is flawed.
• Due to the lack of a clear strategy accompanied by well-defined goals and
objectives, it is very difficult to determine the strategic fit of individual
projects. Because of the absence of strategic buckets it is very problematic to
implement a strategy top-down.
• The quantitative financial metric is not suited for NPD portfolio management
decisions.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder IV
• The risk assessment is not sufficient; projects are assessed on a too limited
number of risks. The risks that are reviewed are not assessed on the three
dimensions of risk.
• The value of the charts produced by the original Kalypso model is limited due
to the lack of a clear definition of what balance is.
The Kalypso model v2 The assessment of the original Kalypso model provides the starting point for the
redesign. The basis of the model remains the same, the Kalypso model v2 uses the
same three variables (risk, strategic fit and financial) as the original model. However,
the determination and assessment of these variables is done in a more detailed
manner. Risks are now assessed on three dimensions (occurrence, impact and the
ability to influence the outcome). This assessment is based on the Risk Diagnosing
Methodology (RDM) and the risks used are based on the risk reference framework
and organization specific risks.
A strategy should be able to answer the questions proposed by Markides (1999):
1. Who should the organization target as customers?
2. What products or services should the organization offer the targeted
customers?
3. How, with what core competences and processes?
4. Why, what is the value proposition to the customers and its competitive
advantage?
5. What are the major threats in the environment?
Based on these questions, various strategic criteria can be formulated. In addition, the
Kalypso model v2 introduces strategic buckets. A strategic bucket is a collection of
NPD projects which are aligned with a particular innovation strategy or strategic
objective (Chao and Kavadias, 2007).
In the first two stages of the development process the financial variable is assessed
with qualitative criteria. In later stages the financial variable is based on the Expected
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder V
Commercial Value (ECV). This metric is able to incorporate probabilities,
development costs, launch costs and Net Present Value.
A project receives a score on each variable (financial, risk and strategic fit),
depending on the development stage of the project, a final score can be calculated.
The weight of each variable varies per development stage. The Kalypso model v2
provides a framework for determining the weight of each variable depending on the
development stage. The final NPD portfolio score is determined with:
!"# !"#$%"&'" !"#$% = !"!! ∗ !! + !" ∗ !! + !"! ∗ !! Where: ai = weight of Strategic Fit Score (SFS) 1 < ai <100 bi = weight of Risk Score (RS) 1 < bi <100 ci = weight of Financial Score (FS) 1 < ci <100 i = stage project is in ∈ (0,1,..,4)
The Kalypso model v2 is also assessed with the NPD portfolio assessment tool. The
results of the assessment are shown in table III.
Table III Average scores per category Original Kalypso
model Kalypso model v2
General Criteria 5,7 6,4 Process Criteria 4,7 5,4 Strategic Criteria 2,8 5,9 Financial Criteria 3,1 6,1 Balance Criteria 3,8 5,0 Risk Criteria 3,0 6,8 Usability Criteria 5,2 5,6
Overall score 4,0 5,9 Poor 1 2 3 4 5 6 7 Excellent
The Kalypso model v2 is a significant improvement over the original model. In all
categories the Kalypso model receives a higher score. Consequently, the Kalypso
model v2 enables users to make the best possible NPD portfolio decisions.
The Kalypso model v2 requires significant commitment from its users. Because many
people are involved in the process, organizations could be hesitant to implement the
Kalypso model v2. In addition, the Kalypso model v2 requires a significant amount of
data, this could limit the usability of the model.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder VI
Scientific Contribution This research contributes to the NPD portfolio management body of knowledge by
developing a model that combines the NPD portfolio management process developed
by Archer and Ghasemzadeh (1999), research conducted by Cooper et al., (1990,
1993, 1995, 1998, 1999, 2001, 2002, 2006, 2008) and the Risk Diagnosing
Methodology by Keizer et al. (2002). In addition an NPD portfolio management
method assessment tool is developed. This tool is an effective way to assess NPD
portfolio methods.
Limitations There are some limitations to this research. First, the Kalypso model v2 is not tested
in practice, it is a theoretical model. Second, the experts consulted for the
development and assessment of the Kalypso model v2 have a consulting and science
background. No practitioners were involved in the development of the model.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder VII
Table of Contents
Preface .................................................................................................................... I
Management summary .......................................................................................... II Introduction ......................................................................................................................................................... II Goal .......................................................................................................................................................................... II NPD portfolio management assessment tool ......................................................................................... II The Original Kalypso Model ........................................................................................................................ III The Kalypso model v2 ..................................................................................................................................... IV Scientific Contribution .................................................................................................................................... VI Limitations ........................................................................................................................................................... VI Chapter 1: Introduction ....................................................................................... 1 1.1 Problem description ............................................................................................................................. 1 1.2 Goal of Master Thesis ........................................................................................................................... 2 1.3 Scope ............................................................................................................................................................ 3 1.4 Approach .................................................................................................................................................... 3 1.5 Outline ......................................................................................................................................................... 4 Chapter 2: NPD Portfolio Management ............................................................... 6 2.1 What is NPD Portfolio Management? ............................................................................................ 6 2.2 NPD Portfolio Process .......................................................................................................................... 8 2.3 NPD portfolio management goals, propositions and criteria ........................................... 12 2.3.1 Goal One: Maximize the value of the portfolio .................................................................... 12 2.3.2 Goal Two: The portfolio must be aligned with the strategy .......................................... 15 2.3.3 Goal Three: The portfolio must be balanced ........................................................................ 17 2.3.4 Risk ......................................................................................................................................................... 20 2.3.5 Usage of NPD Portfolio Management Methods .................................................................. 21 2.3.6 Conclusion: Many criteria ............................................................................................................ 23
2.4 Portfolio methods ................................................................................................................................ 25 2.4.1 Financial methods ........................................................................................................................... 26 2.4.2 Strategic methods ............................................................................................................................ 27 2.4.3 Checklists & Scoring models ........................................................................................................ 28 2.4.4 Bubble diagrams .............................................................................................................................. 29 2.4.5 Conclusion: Which Method? ........................................................................................................ 30
Chapter 3: Description the Original Kalypso model ............................................ 33 3.1 Goal Kalypso model ............................................................................................................................. 33 3.2 Process ...................................................................................................................................................... 35 3.2.1 Input ...................................................................................................................................................... 36 3.2.2 Processing & Output ....................................................................................................................... 42 3.2.3 Charts and Diagrams ..................................................................................................................... 44
3.3 Usability ................................................................................................................................................... 45 3.4 Conclusion ............................................................................................................................................... 47 Chapter 4: Evaluation of the Original Kalypso Model ......................................... 48 4.1.1 General Criteria ................................................................................................................................ 48 4.1.2 Process criteria ................................................................................................................................. 50 4.1.3 Strategic criteria .............................................................................................................................. 51 4.1.4 Financial Criteria ............................................................................................................................. 53 4.1.5 Balance criteria ................................................................................................................................ 54 4.1.6 Risk Criteria ........................................................................................................................................ 54 4.1.7 Usability Criteria .............................................................................................................................. 55
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder VIII
4.2 Conclusion ............................................................................................................................................... 56 Chapter 5: Improving the Original Kalypso model .............................................. 58 5.1.1 Strategic criteria .............................................................................................................................. 58 5.1.2 Risk criteria ........................................................................................................................................ 64 5.1.3 Financial criteria ............................................................................................................................. 68 5.1.4 Balance criteria ................................................................................................................................ 72 5.1.5 Process criteria ................................................................................................................................. 73 5.1.6 Usability criteria .............................................................................................................................. 76 5.1.7 General criteria ................................................................................................................................. 78
5.2 Evaluation of the Kalypso Model v2 ............................................................................................. 80 5.2.1 General Criteria ................................................................................................................................ 81 5.2.2 Process Criteria ................................................................................................................................. 82 5.2.3 Strategic Criteria ............................................................................................................................. 83 5.2.4 Financial Criteria ............................................................................................................................. 84 5.2.5 Balance Criteria ................................................................................................................................ 85 5.2.6 Risk Criteria ........................................................................................................................................ 86 5.2.7 Usability Criteria .............................................................................................................................. 86
5.3 Conclusion ............................................................................................................................................... 87 Chapter 6: Conclusion ....................................................................................... 88 6.1 Scientific Implications ........................................................................................................................ 88 6.2 Scientific Limitations .......................................................................................................................... 88 6.3 Practical Implications ......................................................................................................................... 89 6.4 Practical Limitations ........................................................................................................................... 91 6.5 Recommendation for further research ....................................................................................... 92 References ........................................................................................................... 94
Appendices .......................................................................................................... 98 Appendix 1: Description of the different stages of the Stage Gate® process ........................ 98 Appendix 2: Description of different levels of PML maturity .................................................... 100 Appendix 3: Risk Reference tool including risk issues taken from literature and issues resulting from case studies ....................................................................................................................... 101 Appendix 4: Assessment of the five different NPD portfolio management methods ...... 105 Appendix 5: Overview of Kalypso model performance per criteria ranked by score ..... 108 Appendix 6: Decision rules for classification into risk classes .................................................. 110
List of Figures and Tables
Preface .................................................................................................................... I
Management summary .......................................................................................... II Table I Average scores various NPD portfolio management models ........................................ III Table II Average scores Kalypso model per category ...................................................................... III Table III Average scores per category ...................................................................................................... V Chapter 1: Introduction ....................................................................................... 1 Figure 1.1 Design master thesis project ................................................................................................... 3 Figure 1.2 Document outline ........................................................................................................................ 5 Chapter 2: NPD Portfolio Management ............................................................... 6 Table 2.1 Difficulties in NPD portfolio management .......................................................................... 7 Figure 2.1 NPD portfolio process ................................................................................................................ 8
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder IX
Table 2.2 Overview of NPD portfolio management process criteria ......................................... 12 Table 2.3 Requirements are for an ideal financial metric. ............................................................. 13 Table 2.4 An overview of criteria for achieving strategic alignment ........................................ 17 Table 2.5 Example of parameters used for X or Y-‐axis .................................................................... 18 Table 2.6 Criteria for NPD portfolio management for reaching balance in the portfolio . 20 Table 2.7 Overview of risk criteria ........................................................................................................... 21 Table 2.8 Overview of NPD portfolio management model usability criteria ......................... 23 Table 2.9 Overview of criteria for NPD portfolio management ................................................... 23 Exhibit 2.1 Determination of expected commercial value of project ........................................ 27 Table 2.10 Criteria used to Rank Projects ............................................................................................. 28 Figure 2.3 Risk-‐Reward Bubble diagram ............................................................................................... 30 Table 2.11 overview of different NPD portfolio methods with their assessment score, advantages and disadvantages .................................................................................................................. 31 Chapter 3: Description the Original Kalypso model ............................................ 33 Exhibit 3.1 implemented Stage Gate process ....................................................................................... 34 Exhibit 3.2 Kalypso model process diagram ........................................................................................ 35 Table 3.1 Overview of value categories ................................................................................................. 36 Table 3.2 Overview of value drivers ........................................................................................................ 37 Table 3.3 Qualitative criteria for assessing the probability of financial success ................. 38 Table 3.4 Criteria for determining level of strategic fit ................................................................... 39 Exhibit 3.3 Criteria for determining project risk level .................................................................... 40 Table 3.5 Distribution of points per value driver .............................................................................. 43 Table 3.6 Weighting of each value driver for each phase ............................................................... 43 Table 3.7 List of charts and diagrams the Kalypso Europe model produces ......................... 44 Exhibit 3.4 Responsibilities in the Kalypso Model ............................................................................ 45 Table 3.8 Role and Responsibilities ......................................................................................................... 46 Chapter 4: Evaluation of the Original Kalypso Model ......................................... 48 Table 4.1 Kalypso model general criteria evaluation ....................................................................... 48 Table 4.2 Kalypso model process criteria evaluation ...................................................................... 50 Table 4.3 Kalypso model strategic criteria evaluation .................................................................... 51 Table 4.4 Criteria for determining level of strategic fit ................................................................... 52 Table 4.5 Kalypso model financial criteria evaluation .................................................................... 53 Table 4.6 Kalypso model balance criteria evaluation ...................................................................... 54 Table 4.7 Kalypso model risk criteria assessment ............................................................................ 54 Table 4.8 Kalypso model usability criteria evaluation .................................................................... 55 Table 4.9 Average scores Kalypso model per category ................................................................... 56 Chapter 5: Improving the Original Kalypso model .............................................. 58 Figure 5.1 Stage Gate® process by Cooper ............................................................................................ 58 Figure 5.2 Strategy development .............................................................................................................. 59 Table 5.1 Various consideration for the development of strategic buckets ........................... 60 Figure 5.3 Example of cascading down strategic Buckets ............................................................. 61 Exhibit 5.1 Process of strategic buckets ................................................................................................. 62 Figure 5.4 Strategy process ......................................................................................................................... 63 Figure 5.5 Outline of risk diagnosing methodology .......................................................................... 64 Exhibit 5.2 Example of risk questionnaire (for two risks) ............................................................. 66 Exhibit 5.3 Example of a risk profile (for two risks) ........................................................................ 66 Figure 5.6 Risk process ................................................................................................................................. 67 Exhibit 5.4 Determination of expected commercial value of project ........................................ 68 Table 5.2 Example of distribution of risk .............................................................................................. 70 Figure 5.7 Financial process ........................................................................................................................ 71 Figure 5.7 Balance process .......................................................................................................................... 72 Figure 5.8 Kalypso model v2 process diagram ................................................................................... 74
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder X
Figure 5.9 Responsibilities in the Kalypso v2 model ....................................................................... 77 Table 5.2 Role and Responsibilities Kalypso model v2 ................................................................... 78 Exhibit 5.5 Weighting of the various variables depending on development phase ............ 79 Table 5.3 KE model general criteria evaluation ................................................................................. 81 Table 5.4 KE model process criteria evaluation ................................................................................. 82 Table 5.5 Kalypso model v2 strategic criteria evaluation .............................................................. 83 Table 5.6 Kalypso model v2 Financial criteria evaluation ............................................................. 84 Table 5.7 Kalypso model v2 Balance criteria evaluation ................................................................ 85 Table 5.8 Kalypso model v2 Risk criteria evaluation ....................................................................... 86 Table 5.9 Kalypso model v2 Usability criteria evaluation .............................................................. 86 Table 5.10 Average scores per category ................................................................................................ 87 Chapter 6: Conclusion ....................................................................................... 88 Table 6.1 Scientific Limitations .................................................................................................................. 89 Figure 6.1 OPMP3tm maturity levels ........................................................................................................ 90 Table 6.2 Practical Limitations .................................................................................................................. 92
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 1
Chapter 1: Introduction 1.1 Problem description New products are vital to the success and future prosperity of modern organizations
(Cooper et al., 1998). Organizations that fail to excel at developing new products will
invariably disappear or be gobbled up by the winners (Cooper et al., 1998). A vital
question in new product development is: How should organizations invest their R&D
and new product resources most effectively? The goal of New Product Development
(NPD) portfolio management is to answer this question. An NPD portfolio is a group
of NPD projects carried out under the sponsorship of a particular organization. It
often occurs that an organization can’t develop all its projects due to limited R&D
resources; they have to choose between projects. NPD portfolio management refers to
the selection and support of project investments or program investments that are
guided by the organization’s strategic plan and available resources (in PMBOK.
2000). Hence, NPD portfolio management is about resource allocation in the firm.
Which projects, out of often many opportunities should an organization fund, which
project should receive top priority? Cooper et al. (1993, 1998) found that failing to
conduct NPD portfolio management could lead to:
1. An overloaded NPD pipeline; there are too many project for the resource
available.
2. Consequently, the time to market of projects increases significantly because
resources are spread too thin.
3. When resources are spread too thin the quality of execution starts to suffer.
4. There will be a significant increase in failure rates.
5. Failing to conduct NPD portfolio management will result in a lack of “stellar
new product winners” because good projects are starved for resources.
6. Without NPD portfolio management the wrong projects are often selected
based on the erroneous criteria. Instead of decisions based on facts and
objective criteria decisions are based on politics, opinioneering and emotion.
7. An NPD portfolio composition with projects that are not aligned with the
corporate strategy.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 2
Consequently, a lot of research has been directed toward NPD portfolio management.
However, surveys have shown that the solutions provided by academic research have
found little use in practice (Shane and Ulrich, 2004). Scientific models often require a
lot of data: e.g., information on the financial results, probabilities of completion and
success for all projects. Practitioners often don’t have the required data and when
they do get results, practitioners doubt them due to lack of transparencies and
robustness (Loch et al., 2002). Managers perceive solutions provided by science to be
too difficult to understand and use (Cooper et al., 1999). Practitioners often use
multidimensional decision making tools or ranking methods. A drawback of these
tools is that they rely on an ad hoc list of dimensions and their decision makers often
manipulate the methods to generate desired outcomes instead of using them as true
decision support tools (Chao and Kavadias, 2007 and Loch and Kavadias, 2002).
These tools do help management “think trough” a problem but they lack theoretical or
empirical basis (Loch and Kavadias, 2002). Senior managers, R&D managers, and
project managers are forced to make resource allocation decisions based primarily on
intuition or heuristic rules (Chao and Kavadias, 2007). Hence there is the need for an
NPD management model that is both based on scientific research and is practically
applicable by organizations. Currently, there is not a single NPD portfolio method
which is able to fulfill these needs. This brings us to the goal of this master thesis
research project.
1.2 Goal of Master Thesis Kalypso developed an NPD portfolio management model that tried to overcome the
difficulties mentioned above. They developed an NPD portfolio management model
that is specifically tailored for one of their clients. The model is designed around
client specific metrics and criteria, which makes it difficult to implement the model in
other organizations. The goal of this master thesis is:
To develop a reference NPD portfolio management model that enables users to make
the best possible NPD portfolio decisions (1). This reference model should be
scientifically substantiated and practically applicable (2). The NPD portfolio
reference framework should enable Kalypso to implement NPD portfolio management
solutions at any client (3).
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 3
With supporting sub questions:
1. What is NPD portfolio management?
2. Which requirements are needed for an NPD portfolio management tool?
3. Which NPD portfolio management methods are there?
4. What are the advantages and disadvantages of these NPD portfolio
management methods?
5. How does the Kalypso NPD portfolio management model work?
6. What are the strengths and weaknesses of the Kalypso NPD portfolio
management model?
7. How can the Kalypso NPD portfolio management model be improved in order
to comply with the answer to question 2?
1.3 Scope NPD portfolio management can be conducted at various levels, from entire NPD
programs to specific projects. This research is limited to the NPD portfolio
management of individual projects.
1.4 Approach This master thesis project will be relevant to both scientists and practitioners by
contributing to both streams of knowledge. The design for this master thesis project is
shown in figure 1.1.
Figure 1.1 Design master thesis project
Literature
Original model developed by Kalypso
List Criteria for NPD Portfolio Management
Science Practice
!"#$%#&'
New model
Experts
!"#$%#&'
()*+%**),-
Knowledge of NPD Portfolio Management
('"'$,.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 4
Currently there is a large body of knowledge about NPD portfolio management. Both
scientist and practitioners contribute to this body of knowledge. Scientists contribute
by conducting research and publish their findings. Practitioners use and contribute to
this knowledge by applying it in real business environments. In this master thesis a
list of criteria for NPD portfolio management is composed. This list of criteria is the
result of a comprehensive review of the scientific literature concerning NPD portfolio
management. Kalypso developed an NPD portfolio management model (hereafter:
original model) for one of its clients based on their experience and knowledge of NPD
portfolio management. The list of criteria will be used to evaluate the original
Kalypso model and is conducted by the author and experts. This evaluation will give
insight into the strong and weak aspects of the original model. The new model is built
on the strong points of the original model while overcoming the weak. In addition, the
new model is based on scientific literature as well as on contributions by experts. In
the end, the new model will be evaluated based on the same criteria as the original
model. By comparing both evaluations it will become clear if the new model is an
improvement over the original model. If this is the case, the new model will
contribute to the existing body of knowledge by adding new knowledge about NPD
portfolio management models.
1.5 Outline The goal and supporting sub questions form the outline of this master thesis report. A
graphical depiction of the outline is shown in figure 1.2. In chapter two, the subject
of NPD portfolio management will be researched and answers to what NPD portfolio
management is and why it is so difficult will be sought. In the second part of chapter
two a comprehensive list of criteria for NPD portfolio management, found in the
literature, is composed. In addition, various existing NPD portfolio management
methods are discussed. These methods are assessed on the criteria found in the
previous part of this chapter. This assessment provides a good insight in the strong
and weak aspects of the various NPD portfolio management methods. What will
become clear is that none of these methods is suited to be used as a stand-alone NPD
portfolio management method. In chapter three the original Kalypso model, the
process and various in and outputs will be described. In addition, a running example
is added to clarify numerous aspects of the original Kalypso model. In chapter four
the original Kalypso model is evaluated on the criteria for NPD portfolio management
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 5
found in chapter two. This assessment is done together with Laurens Broekhof, a
Kalypso employee who was involved in the development of the current model. Each
score is explained and substantiated. Chapter four provides insight in the strong and
weak aspects of the original Kalypso model. These strong and weak aspects provide
an excellent starting point for the redesign in chapter five. This chapter describes in
detail the various improvements in the Kalypso model v2 over the original Kalypso
model. These improvements are validated with experts from Kalypso (Mick
Broekhof) and outside Kalypso (Linda Kester). Chapter five ends with an assessment
of the Kalypso model v2 on the criteria for NPD portfolio management found in
chapter two. This assessment answers the question if the Kalypso model v2 is better
suited for NPD portfolio management than the original Kalypso model and the other
models covered in this research. Chapter six starts with an overall conclusion
followed by a description of the consequences of this research for science and
practice. Chapter six ends with the limitations of this research, recommendations for
further research and usage of the Kalypso model v2.
Figure 1.2 Document outline
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Literature Review
Description of OriginalModel
Evaluation of OriginalModel
Development of NewModel
Conclusion
-What is NPD portfolio management?-Why is it so difficult?-What are the criteria for NPD Portfolio management?-What are the current NPD Portfolio management methods?-Evacuation of the current NPD portfolio management methods.
-Description of the goal of the original model.-Description of the process.-Description of the various inputs.-Description of the output.-Running example of original model.
-Evaluation of the original model based on the criteria from chapter 2.-Evaluation conducted by author and experts.-Conclusion of evaluation, what are the strong and weak points?
-Redesign of the original model prioritized by the results of chapter 4.-Evaluation of the new model based on the criteria from chapter 2.-Evaluation conducted by author and experts.-Conclusion of evaluation.
Chapter 1
-Statement of goal master thesis project-Statement of sub questions.-Definition of scope master thesis project.-Description of master thesis approach.-Description of master thesis outline.
Introduction, and Statementof Goal Master Thesis Project
-Is the master thesis goal attained?-Consequences for science and practitioners-Recommendations-Limitations-Further research
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 6
Chapter 2: NPD Portfolio Management
2.1 What is NPD Portfolio Management? A project portfolio is a group of projects carried out under the sponsorship of a
particular organization. It is possible that an organization has multiple projects that
meet all the criteria set, however it often occurs that an organization can’t continue
with all projects due to limited resources, they have to choose between projects. NPD
Portfolio management is the management of different NPD projects and NPD
programs (depending on the size of the NPD portfolio) in an NPD portfolio, it is
about deciding which projects to pursuit, continue, recycle or to abandon. It is very
important for a organization to have the right product development mix, failing to do
so could lead to loss of a organization’s competitive advantage (Chao and Kavadias,
2004, 2007). Such a mix could consist of long and short term, radical and incremental
and low and high-risk projects. Success requires a fundamental trade off: short-term
benefits accrued through incremental improvements versus long term benefits
achieved through radical or new to the world products and services (Tushman and
O’Reilly, 1996). NPD portfolio management enables organizations to make these
decisions. Cooper et al. (1999) define NPD portfolio management as a dynamic
decisions process, whereby a organization’s collection of active new product (and
R&D) projects is constantly updated and revised. In this process, new projects may be
accelerated, terminated, or de-prioritized; and resources are allocated and reallocated
to the active projects.
An NPD portfolio allows firms to operate and implement their innovation strategy
over time. This implies that the NPD portfolio problem entails a large component of
ambiguity and complexity, since the determinants of organizational success and their
interactions are rarely known (Loch and Tapper, 2002). Another difficulty of NPD
portfolio management is resource scarcity, an organization doesn’t have limitless
resources and therefore can’t pursue every opportunity. In many cases (76%)
organizations have too many projects for the resources available (Cooper and Edgett,
2006). This results in resources that are spread too thin over too many projects and
there is not the time or ability to do many of the key project activities proficiently
(Cooper et al., 2002). In multi-project environments, scarce resources render the
resource allocation decision a critical factor for success (Adler et al., 1995). It often
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 7
occurs that organizations develop multiple projects in closely related markets. When
these projects interact, synergies or incompatibilities can exist in their technical
aspects. Products may substitute or complement each other in different markets.
Interactions between success determinants play a critical role in the resource
allocation decision (Chao and Kavadias, 2007). Outcome uncertainty is another aspect
which makes NPD portfolio management difficult. Management faces uncertainty
along the dimensions of the potential market value and the technical output of any
given project (Chao and Kavadias, 2007, Pich et al., 2002). Decision makers must
allocate resources over time and NPD programs evolve over time. Not all projects
evolve in the same manner over time. It is often difficult to quantify the potential of
promising ideas or to precisely measure the risks involved (Chao and Kavadias,
2007). Table 2.1 summarizes the difficulties of NPD portfolio management.
Table 2.1 Difficulties in NPD portfolio management
• NPD portfolio problem entails a large component of ambiguity and complexity (Loch and Tapper, 2002) • NPD portfolio management is constrained by resource scarcity (Cooper and Edgett, 2006) • NPD portfolio management transcends multi-‐project environments (Adler et al., 1995) • NPD portfolio management deals with outcome uncertainty (Chao and Kavadias, 2007, Pich et al.,
2002) • Projects are interdependent (Chao and Kavadias, 2007, Pich et al., 2002) • Programs and projects often don’t evolve in the same manner (Pich et al., 2002) • Difficult to quantify promising ideas (Chao and Kavadias, 2007)
In remaining part of this chapter we will discuss NPD portfolio management in detail.
There are two important streams of literature. The first one is by Cooper et al. (1998,
1999) and the second one is by Archer and Ghasemzadeh (1999). Cooper et al. (1998,
1999) formulated three goals for NPD portfolio management. Archer and
Ghasemzadeh (1999) developed an NPD portfolio process model and formulated
various propositions for NPD portfolio management. What will become clear is that
the two streams of literature overlap but most of all complement each other. In this
chapter a list of criteria for NPD portfolio management will be developed, this list
will be the basis for the assessment of the Kalypso model in chapter 4. However, first
we will describe the NPD portfolio management process model developed by Archer
and Ghasemzadeh (1999). What will become clear is that goals of Cooper et al. (1998,
1999) and the propositions of Archer and Ghasemzadeh (1999) do not cover all
aspects involved in NPD portfolio management, these missing aspects will be covered
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 8
in paragraph 2.3.4. The rest of this chapter will provide insight in current NPD
portfolio methods.
2.2 NPD Portfolio Process We will start with the description of the NPD portfolio management process
developed by Archer and Ghasemzadeh (1999). The NPD portfolio management
process consists of various stages which progress from the initial broad strategy
considerations towards the final solution. The NPD portfolio management process is
shown in figure 2.1.
Figure 2.1 NPD portfolio process
Source: Archer and Ghasemzadeh (1999) page 211
The portfolio selection process proposed by Archer and Ghasemzadeh (1999, 2007) is
divided in three different phase; the preprocess phase, process phase and post process
phase. The first phase, preprocess phase, consists out of two parts. The first part is
methodology selection, this is a strategic process that should be done in advance of
any other activities in NPD portfolio selection. Methodology selection needs to be
done once for all time, with minor adjustments from time to time if other
methodology choices appear to be better matches for the tasks at hand before each
cycle of portfolio selection (Archer and Ghasemzadeh, 2007). It is critical that
common measures (e.g., NPV, scoring attributes, valuation of risk etc.) are chosen so
Strategy Development
Proposed Projects
PrescreeningIndividual
ProjectAnalysis
Screening PortfolioSelection
PortfolioAdjustment
ProjectDevelopment
ProjectEvaluation
PortfolioCompletion
MethodologySelection
ProjectDatabase
Guidelines Resource allocation
MainStages
Pre ProcessActivity
Pre ProcessActivity
Post Processstages
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 9
they can be calculated separately for each project under consideration, allowing an
equitable comparison of the projects (Archer and Ghasemzadeh, 1999). The second
part of the first phase is strategy development. The NPD Portfolio enables an
organization to implement its strategy, to do so, the strategy must be developed before
hand. Strategy can be used to develop a focused objective for an NPD portfolio and
the level of resources needed for its support (Cooper et al., 1998, 1999, Chao and
Kavadias, 2007).
The process phase consists out of 5 (main) stages. The first stage is prescreening. The
goal of this stage is to ensure that any project being considered for the portfolio fits
the strategy. Projects should be classified in advance of the portfolio selection,
according to criteria that can override other considerations (Archer and Ghasemzadeh,
2007).
The second stage of the process phase is the individual project analysis. Each project
is individually assessed on the common measures selected in the methodology
selection stage. The output of this stage is a common set of parameter estimates for
each project. Individual project analysis can be done with a Stage Gate® process
developed by Cooper (1990, 2008). The stage gate process is depicted in figure 2.2.
Figure 2.2 Stage Gate® process with funnel
Source: combination of Docherty (2006) and Cooper (2008) page 215 and 231
Stage 1 Stage 4Stage 3Stage 2 Stage 5Gate3
Gate2
Gate4
Gate5
Stage 0 Gate1
Post launchReview
$
Project i
2
1
3
12
8
14
11
13
11
9
166 5
1110
4
18
19
24
21
i
Discovery Scoping BuildBusiness
Case
Development Testing &Validation
Launch
Idea ScreenSecondScreen
Go ToDevelopment
Go ToTesting
Go To Launch
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 10
An effective Stage-Gate® system is essential to sound portfolio management (Cooper,
2008). The Stage-Gate® process is a conceptual and operational map for moving new
product projects form idea to launch and beyond. Numerous projects enter the funnel
but their number decreases at every step. A Stage Gate® process is a process with
different stages divided by gates. Each stage has the goal to gather information and
reduce risk of a project. The costs concerning a project increase significantly at each
consecutive stage. Teams are cross-functional and work parallel on activities in each
stage. After each stage, at the gate a go/kill/hold/recycle decision is made along with
an approved action plan for the next stage. These decisions are made based on
deliverables and criteria. Deliverables are visible, based on a standard menu for each
gate and are determined at the output of the previous gate. Every gate has criteria
against which the project is judged, these criteria are divided in must meet criteria and
should meet criteria. Must meet criteria are criteria that a project must meet, if they
are not met the project will be terminated. Should meet criteria are highly desirable
characteristics that are rated in a point-count scheme. Most best practice organizations
have adopted a Stage Gate® system (Griffin, 1997, Cooper, 2008). The different
stages and gates of the Stage Gate® process are described in appendix 1. Each project
should be assessed individually on its (financial) value (Archer and Ghasemzadeh,
1999). Consequently the portfolio decision process encompasses and/or overlaps a
number of decision-making processes within the Stage Gate® process. Information
and criteria used to review a single project at each gate can also be used to make
decision regarding the entire portfolio.
Screening is the next stage, the common set of parameter estimates are examined in
advance of the actual selection process in order to eliminate any projects or
interrelated families of projects that do not meet present criteria (e.g., NPV), except
for those projects that are mandatory or required to support other projects still being
considered (Archer and Ghasemzadeh, 2007).
In the following stage, Portfolio selection, interactions among various projects are
considered, including interdependencies, competition for resources and timing, with
the value of each project determined from a common set of parameters that were
estimated for each project in the previous state (individual project analysis) (Archer
and Ghasemzadeh, 1999). It often occurs that projects are interdependent (e.g. project
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 11
C can be undertaken when projects A and B are completed since their outputs feed
project C) (Chao and Kavadias, 2007). An NPD portfolio tool should take into
account these interdependencies. Many portfolio selection techniques do not consider
the time-dependent resource requirements of projects (Martino, 1995), and most
implicitly assume that all projects selected will start immediately. This does not fit the
reality of project management, where projects compete for limited resources. Projects
ought to be scheduled in such a way that they use resources as smoothly as possible in
time, and projects should be completed within some planned interval (Archer and
Ghasemzadeh, 1999). Hence an NPD portfolio method should take into account the
time dependent nature of project resource selection.
The last main stage is portfolio adjustment. The end result of this process is a
portfolio that meets the objectives of the organization optimally or near-optimally.
This stage provides an overall view, where the characteristics of projects, critical for
an optimized portfolio (e.g., risk, time-to-complete), can be represented. An important
aspect of this stage is achieving some form of balance (Archer and Ghasemzadeh,
2007).
Cooper and Kleinschmidt (1995) found that a high quality new product development
process is the most important factor for new product success. The NPD portfolio
management process shown in figure 2.2.1 enables decision makers to move logically
towards an integrated consideration of projects most likely to be selected, based on
sound theoretical models (Archer and Ghasemzadeh, 1999). The product development
process consists of many stages (see appendix 1), each time a project leaves one stage
to enter another one, the project should be re-evaluated at the same time as new
projects being considered for selection. Note that the NPD portfolio management
process by Archer and Ghasemzadeh (1999) has a lot of similarities with the Stage
Gate® process by Cooper (1990, 2008). Table 2.2 shows an overview of the criteria
for NPD portfolio management mentioned in this paragraph.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 12
Table 2.2 Overview of NPD portfolio management process criteria • The NPD portfolio management model should be divided in various stages (Archer and Ghasemzadeh,
1999) • The NPD portfolio methodology and common measures should be selected before the NPD portfolio
process starts (Archer and Ghasemzadeh, 1999) • The strategy should be defined before the NPD portfolio management process starts (Archer and
Ghasemzadeh, 1999) • Projects should be classified in advance of the before the NPD portfolio selection starts (Archer and
Ghasemzadeh, 1999) • Projects should be assessed individually, based on a Stage Gate® system (Archer and Ghasemzadeh,
1999) • Projects should be individually assesses based on a common set of parameters (Archer and
Ghasemzadeh, 1999, Cooper 1990, 2008) • The NPD portfolio model should consider the interdependencies between projects (Chao and Kavadias,
2007) • The NPD portfolio model should consider the time dependent resource requirements of projects
(Martino, 1995) • The NPD portfolio model should provide an overview of the NPD portfolio where the most important
project characteristics are represented (Archer and Ghasemzadeh, 1999)
2.3 NPD portfolio management goals, propositions and criteria Different authors have formulated various goals and criteria. In the next part we will
discuss the three goals proposed by Cooper et al. (1998, 1999) and various
propositions of Archer and Ghasemzadeh (1999). What will become clear is that both
streams complement and reinforce each other and form an excellent basis for NPD
portfolio management tool development.
2.3.1 Goal One: Maximize the value of the portfolio We will start with the description of the three goals proposed by Cooper et al., (1998,
1999). The first goal Cooper et al. (1998, 1999) formulated is to maximize the NPD
portfolio’s value against one or more objective (such as profitability, strategic,
acceptable risk, etc.). You can achieve this by selecting projects with the highest
value for an organization. The highest value is often determined with financial
metrics. Various financial metrics can be used to determine the value of a project. The
Net Present Value (NPV) method involves discounting expected cash flows at a
discount rate that is set to reflect the perceived riskiness of a project. The Return On
Investment (ROI) is the ratio of money gained (or lost) on an investment relative to
the amount of money invested. In the literature various requirements of financial
metrics are mentioned. Financial metrics should incorporate the cost of development
and launch (Cooper et al., 1998). In order to correctly compare projects one must
look at what the project yields and at what the project costs. Another reason is that
organizations don’t have limitless resources, each critical resources must be optimally
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 13
used. Consequently, a financial metric should therefore be constrained by a critical
resource (Cooper et al., 1998, 1999).
Most financial metrics disfavor advanced technology projects due to the long-term
payoff and high likelihood of failure (Chao and Kavadias, 2007). The ideal financial
metrics should have no bias towards high-risk projects. The NPD process is a staged
and flexible process (projects can be abandoned, paused, and stages could be done
over again, etc. (Cooper, 1990, 2008)) a financial metrics should take this into
account. These options can add value to a project (Newton et al., 2004). In order to
incorporate these options one must look at the probability of these options occurring
(Pennings and Lint, 1997, Cooper et al., 1998). Sunk costs are not relevant for the
decision process and should therefore not be included into the financial metric
(Cooper et al., 1998).
Remer and Nieto (1995) studied 25 different (financial) project evaluation techniques,
they tested each method by calculating the value of two projects (projects A and B
where project A is more valuable than project B). Remer and Nieto (1995) found that
methods that did not use the Minimum Acceptable Rate of Return (MARR) and Time
Value of Money (TVM) selected the wrong project (project B over A). As mentioned
before there are various financial metrics, these metrics differ in complexity and
applicability. It is important that the financial metric used is understandable plus
usable for practitioners and doesn’t appear to be a black box (Faulkner, 1996, Cooper
et al., 1999, Kester et al., 2009). An overview of the requirements for a financial
metric are shown in table 2.3.
Table 2.3 Requirements are for an ideal financial metric.
• The financial metric should incorporate the cost of development and launch (Cooper et al., 1998) • The financial metric should have no bias towards long term radical projects (Chao and Kavadia, 2007) • The financial metric should capture the flexibility of the development process (Newton et al., 2004) • The financial metric should be constrained by a critical resource (Cooper et al., 1998) • The financial metric should ignore sunk cost (Cooper et al., 1998) • The financial metric should incorporate the probability of success or failure (Cooper et al., 1998,
Pennings and Lint, 1997) • The financial metric should be user friendly and not too complex or appear as a black box (Faulkner,
1996, Cooper et al., 1999, Kester et al., 2009) • The financial metric should use the Minimum Acceptable Rate of Return (MARR) and Time Value of
Money (TVM) (Remer and Nieto, 1995)
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 14
There are alternatives to quantitative financial metrics. Another method for
maximizing the value of the NPD portfolio are scoring models (Cooper et al. 1998,
1999). Scoring models, can take the weight of different criteria into account when
calculating an overall score. Determining the weighting and scores is a subjective
process that depends as much on personal opinion as hard numbers (Coldrick et al.,
2003). Projects are ranked according to the overall score. Projects can be added or
deleted without affecting the scores of the remaining projects. Scoring models are less
appropriate when there is a high degree of interdependence between projects; when
activities and the results of one project depend on the activities and results of a
different project (Henriksen et al., 1999). The advantage of scoring models is that
they do not place too heavy an emphasis on (quantitative) financial criteria, whose
reliability is often doubtful in the early stages of a project. Another advantage of
scoring models is that they can capture multiple goals. The goal of an NPD portfolio
management method is to assist management in making sound portfolio selection
decisions. This is only possible if the NPD portfolio management methods enable
objective comparison of projects (Archer and Gasemzadeh, 1999), which means that
scoring models should produce score data (interval or ratio scale) (Graziano and
Raulin, 2005).
There are two methods for maximizing the value of the NPD portfolio, so which one
is best? Cooper et al. (1999) found that scoring models result in a portfolio with high
value, profitable, high return projects with solid economic prospects. The use of only
financial metrics for selecting NPD projects results in a portfolio with poor value
projects and a portfolio with too many projects for the resources available (Cooper et
al., 1999). In order to maximize the value of the NPD portfolio Cooper et al. (1999)
recommend that both methods are used. Scoring models because of their good
performance and financial metrics because they are dominant and widely used in
business. What applies for both NPD portfolio methods is that the criteria’s for
valuating each project should be determined in advance (Archer and Gasemzadeh,
1999). In addition, competing projects should be assessed on common criteria and
metrics (Archer and Gasemzadeh, 1999).
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 15
2.3.2 Goal Two: The portfolio must be aligned with the strategy The second goal formulated by Cooper et al. (1998) is that the portfolio should be
aligned with the organization’s strategy. Cooper and Kleinschmidt (1995) found that
the new product strategy is the second most important success factor for an NPD
program. The essence of strategy is selecting one position that an organization can
claim as its own (Markides, 1999). According to Markides (1999) a strategic positions
is the sum of an organization’s answers to the following questions:
6. Who should the organization target as customers?
7. What products or services should the organization offer the targeted
customers?
8. How, with what core competences and processes?
9. Why, what is the value proposition to the customers and its competitive
advantage?
10. What are the major threats in the environment?
The most common source of strategic failure is the inability to make clear and explicit
choices on these five dimensions (who, what, how, why, threats) (Markides, 1999). In
order to assess the alignment of the projects in a portfolio one must translate the
business strategy into a technical strategy (Loch and Tapper, 2002). This means that
the technology strategy must specify what technologies to master, and how these
technologies relate to the business: what products and segments these technologies
address, what they contribute to competitive advantage and to hedging against
environmental threats (Loch and Tapper, 2002). According to Boyer and McDermott
(1999) strategic alignment is the level of agreement within an organization regarding
the relative importance of cost, quality, delivery and flexibility to the organization’s
operational goals as well as the relationships between competitive priorities and
policies. Strategic alignment can be classified under two categories, internal fit and
external fit. Internal fit is the consistency between the task and policies and practices
(Skinner, 1974). External fit is the alignment of the operational strategy with the
business and corporate strategies and to examine the alignment-performance
relationship (Joshi et al., 2003).
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 16
Kaplan and Norton (1996) developed the Balanced Scorecard (BSC) method. The
first step of the BSC process is to translate the vision. In this step the vision of the
organization is translated into agreed upon integrated set of objectives measures that
describe the long-term drivers for success. The vision and strategy is translated into
four perspectives; a financial perspective, a customer perspective, an internal business
process perspective and a learning and growth perspective. “To succeed financially,
how should we appear to our share holders?” is the main question for the financial
perspective. For the customer perspective a organization must ask itself “To achieve
our vision, how should we appear to out customer?” The financial and customer
perspective form the basis of the internal business process perspective. Here the
question “To satisfy our shareholders and customers, what business process must we
excel at?” should be asked. For the learning and growth perspective the question is
“To achieve our vision, how will we sustain our ability to sustain and improve?” By
asking these questions objectives, measures, targets and initiatives will be formulated
for each perspective.
NPD portfolio management should be a top-down and bottom-up approach (Cooper et
al., 1998, Archer and Ghasemzadeh, 1999). The top down part is described above, top
management compose a vision and strategy and sets aside resources for achieving it.
Where money is spend mirrors the business’s strategy. The bottom-up approach
begins with a review of specific projects and focuses on selecting the best (Cooper et
al., 1998). Strategic criteria are built into the project selection tools: thus, strategic fit
is achieved by incorporating numerous strategic criteria into the Go/Kill and
prioritization methods (Cooper et al., 1998). Scoring models are excellent tools for
incorporating and assessing these criteria.
In order to reach the second goal, align the portfolio with the strategy, it is very
important that there is a strategy. Cooper et al. (1995) identified that a clear well
communicated new product strategy for the organization is a driver for new product
performance. The strategy of an organization is the basis of NPD portfolio
management and should therefore be defined before the NPD portfolio is selected
(Archer and Gasemzadeh, 1999). You can only align a portfolio with the strategy if
the strategy is clearly defined in advanced. This strategy must be translated in to clear
goals and objectives and should be able to answer the question proposed by Kaplan
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 17
(1996) and Markides (1999). In the end, the mission, vision and strategy of the
business must be operationalized in term of where the business spends money
(Cooper et al., 1998). It must become clear where an organization wants to spend its
resources, which types of projects, across which markets and products lines, etc.
Resource decisions should be made corresponding to the strategy (Archer and
Gasemzadeh, 1999). This can be done with strategic buckets. A strategic bucket is a
collection of NPD projects or NPD programs that are aligned with a particular
innovation strategy (Chao and Kavadias, 2007). The NPD programs in a strategic
bucket may involve process improvements and cost reductions, minor product
modifications, radical next-generation technological research, or groundbreaking
R&D initiatives, etc. (Chao and Kavadias, 2004, 2007). While the definition of each
strategic bucket is top-down, the assignment of the projects to buckets is a bottom-up
approach. Projects are assigned to only one bucket. The criteria for placing a project
in a bucket can differ per bucket. Projects are assigned until the resources of a bucket
are depleted. Barczak et al. (2009) found that top performing organizations use the
strategic buckets methods more often than the rest is. Strategic buckets prevent non-
related projects from competing for the same resources. Table 2.4 shows an overview
of criteria for achieving the second goal for NPD portfolio management methods
Table 2.4 An overview of criteria for achieving strategic alignment
• There must be a clear mission and vision present and corresponding goals and objectives (Kaplan and Norton 1996, Cooper et al., 1998, Markides, 1999, Loch and Tapper, 2002)
• A clear strategy consists of the who, what, how, why and threats (Markides, 1999, Loch and Tapper, 2002) or consist out of four perspectives (financial, customer perspective, internal business process and learning and growth perspective) (Kaplan and Norton, 1996)
• The NPD portfolio management tool should be able review projects based bullet one and two • The NPD portfolio management tool should be top-‐down and bottom-‐up (Cooper et al., 1998). • The strategy must make clear where and how many resources should be spent (Cooper et al., 1998). • The NPD portfolio management tool must assess projects on strategic fit (Cooper and Kleinschmindt,
1995, Cooper et al., 1998) • Projects should be assigned to buckets Barczak et al. (2009)
2.3.3 Goal Three: The portfolio must be balanced The last goal formulated by Cooper et al. (1998, 1999) is that the portfolio must be
balanced. As mention before success requires a fundamental trade off: short-term
benefits accrued through incremental improvements versus long term benefits
achieved through radical or new to the world products and services (Tushman and
O’Reilly, 1996). Consequently you need to balance short-term projects with long-
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 18
term projects or radical projects with incremental projects. What balance is, is
determined by senior management and should be based on the strategy of an
organization. It is hard to capture balance into a single metric, the most effective way
to asses the balance of a portfolio is trough visual tools (Cooper et al., 1998, Cooper
and Edgett 2001). The current state of a portfolio can be visually depicted with bubble
diagrams where projects are plotted on an X-Y plot or map. The most popular bubble
diagram depicts risk-reward (Cooper et al., 1998, Cooper and Edgett 2001). With this
bubble diagram one axis is some measure of the reward and the other axis represents
the probability of success. On the diagram, projects are depicted based on their
resource requirements (the bigger the resource requirement, the larger the diameter of
the circle). Consequently management is forced to consider the resource issues. Often
there are a finite number of resources available, therefore the sum of the areas of the
circles must be a constant. Of course there are many more parameter that can be used
for the X or Y-axis, table 2.5 give some example of them.
Table 2.5 Example of parameters used for X or Y-‐axis
• Fit with the business or corporate strategy (low, medium, high) • R&D cost to completion (e.g. dollars) • Time to completion (Years, months) • Competitive impact of technologies (base, key, pacing and embryonic technologies) • Durability of the competitive advantage (short-‐term, medium, long-‐term) • Etc.
Source Cooper et al., 1998 page 56
There are some risks involved in using bubble diagrams. A bubble diagram gives an
overview of the entire portfolio, this means that all the projects are shown on the
diagram. However, these projects are often not in the same development phase, this
means that the information used for depiction projects differs in reliability. Projects in
early stages lack reliable (financial) data while projects in later stages do have reliable
data. Consequently there is the probability that a bubble diagram incorrectly depicts
the portfolio due to unreliable data. There is also the risk of information overload
(Cooper et al., 1998), due to the almost limitless number of variants of visual
depictions where all kinds of variable are used. An organization should only use
bubble diagrams that depict the most important information. In addition a bubble
diagram does not rank projects! Therefore they can’t be used to directly make NPD
portfolio decisions, bubble diagrams are a starting point for discussion only (Cooper
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 19
et al., 1998, Cooper and Edgett, 2001). Management still has to translate these data
into actionable decisions (Cooper et al., 1998). The most important question of all is
what is the right balance? Management has to clearly define what balance is and what
imbalance is. If imbalance occurs what actions should be taken, should management
immediately start cancelling projects and approving others?
A balanced portfolio results in diverse a composition of projects. Not all NPD
portfolio tools are practical for all project types e.g., hard financial tools are less
applicable for radical projects while they are easy to use for incremental projects.
Archer and Ghasemzadeh (1999) propose that the NPD portfolio method must be
flexible enough so that stake holders can choose in advance a particular techniques or
methodologies with which they are comfortable in analyzing relevant data and
making choices of the type of projects at hand. Cooper and Edgett (1999) found that
the average organization uses 2,4 methods for their portfolio selection. It must be
noted that for each type of project (or strategic bucket) the same methodology should
be chosen because the only way to objectively compare projects is when they are
assessed on common measures. This will allow an equitable comparison of projects
during the selection process (Archer and Ghasemzadeh, 1999). Which (financial)
criterion is used for valuating each project should also be determined before the
portfolio selection process starts.
The third goal is a difficult goal to attain because balance cannot be captured in a
single metric. The most effective method for achieving balance is by using multiple
NPD portfolio methods that compensate each individual method’s biases towards a
certain type of project and/or development stage. The most effective method to
evaluate balance is with a visual depiction of the NPD portfolio. The major difficulty
is how to define a visual depiction as balanced or imbalanced. Table 2.6 shows an
overview of what a NPD portfolio management method must do in order to reach the
third goal set by Cooper et al. (1998, 1999).
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 20
Table 2.6 Criteria for NPD portfolio management for reaching balance in the portfolio • The NPD portfolio tool must be able to visually depict the portfolio (Cooper et al., 1998) • Management must define what balance and imbalance is (Cooper et al., 1998) • The NPD portfolio should only show the for management most important diagrams (Cooper et al.,
1998) • The Bubble diagram X and Y-‐axis should be based on a organization’s strategic goals and objectives
(Cooper et al., 1998) • The NPD portfolio management tool should use multiple assessment methods for valuating the
projects (Cooper and Edgett, 1999)
2.3.4 Risk There is one important aspect of NPD portfolio management that is barely covered
but has a great influence on the value of a project and thus portfolio selection: risk. As
mentioned before, risk should be taken into account in the financial metric (Cooper et
al., 1998, Pennings and Lint, 1997), risk is also an important input for bubble
diagrams. All NPD projects deal with risk! Risk is most commonly conceived as
reflecting variation in the distribution of possible outcomes, their likelihoods, and
their subjective values (March and Shapira, 1987). Another definition of risk is the
exposure to losses in a project (Webb, 1994) or the probability of losses in a project
(Remenyi, 1996). The impact of a risk factor is usually described as depending on its
likelihood of occurring and the consequence if it does occur (Turner, 1993).
Disappointment and failures in the realization of product innovation are often due to a
naïve but optimistic attitude towards the available and required knowledge and skills
(Halman and Keizer, 1994). These gaps are directly linked to risk, if there is a large
knowledge gap within a project there is also a high risk involved. There are numerous
kinds of risks, Abetti and Stuart (1988) found three categories of risk, market,
function and technology risk. Halman and Keizer (1994) also describe three kinds of
risk, technical, organizational and commercial. Abetti and Stuart (1988) conclude that
product risk increases with the degree of product newness (as viewed by the supplier)
and decreases with innovation uniqueness (as viewed by users and market place).
During the NPD process risk should be assessed repeatedly but what is the right
moment to do so? When it is done too early the evaluation may kill promising ideas
or otherwise hamper the creative process since risk in the early stage will probably
evaluated as high or very high (Abetti and Stuart, 1988). Abetti and Stuart (1988)
argue that when an NPD project’s risk assessment is done too late organizations may
realize that the risk of a project is too high and that the organization has wasted time,
money and human resources. According to Abetti and Stuart (1988) the optimum
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 21
timing for performing risk assessments is after the concept testing and concept
development phase at the end of the initiation phase before product development,
when significant technical and marketing efforts will be made. Across all categories,
uncertainty tends to be high at the initial planning stage and to be reduced over the
course of the project (Sommer et al., 2007, chapter 17).
According to Halman and Keizer (1994) risk has three dimensions: occurrence,
impact and control. A particular NPD activity is seen as risky if (1) the likelihood of a
bad result is considerable, (2) the impact on the success of the NPD project is great
and (3) the ability of the team to influence it within the time and resource limits of the
project is small (Keizer et al., 2005). Each risk should be assessed on these three
dimensions, but which risks should be assessed? Keizer et al. (2005) developed a risk
reference tool for NPD based on a meta-study on risk that initially identified 325 risks
that could be reduced to 142 (see appendix 3). These risks where divided into 12 main
risks categories. Not all risks identified came from the meta-study, Keizer et al.
(2005) also conducted interviews where they discovered additional risks. Hence, 142
risks seems to be too many for effective project assessment but Keizer et al. (2005)
argue that too many projects have failed because less frequently occurring risks were
not taken seriously. This argument applies for radical new projects, although it can be
assumed that in the case of incremental innovation an organization already has a fair
overview of the risks involved. Consequently in the case of incremental innovative
project a less throughout risk assessment should be sufficient. Hence it is important
that a project is assessed on a sufficient number of risks depending on the project
type. What “sufficient” is, is a consideration that each organization has to make on its
own.
Table 2.7 Overview of risk criteria
• The NPD portfolio management tool should assess projects on risk (Halman and Keizer, 1994. Keizer et al., 2005
• Risk should be assessed on three dimensions; likelihood, impact, and the ability to do something about it (Halman and Keizer, 1994)
• Risk should be assessed on a sufficient number of criteria (Halman and Keizer, 1994)
2.3.5 Usage of NPD Portfolio Management Methods In the previous paragraphs we have mainly looked at the technical side of NPD
portfolio management. However usability is as important as technical soundness. If a
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 22
model is technical sound but practically useless the value of this model is limited.
This paragraph describes what the difficulties are for using NPD portfolio
management models. The NPD portfolio process has a lot of similarities with the
Stage Gate® process. Consequently, it is assumed that the usability criteria are similar.
Defining governance roles and responsibilities is an important facet of Stage-Gate®
process (Cooper, 2008). The same applies for NPD portfolio management, it must be
clear who the project leader is, who assess the project on what criteria and how many
people are involved in project assessment. The project leader is responsible for the
project and leads the project through the various development stages. However,
(senior) management makes the NPD portfolio decisions. They oversee and finance
the project. Project leaders should not make portfolio decisions because they can get
emotionally involved and become very reluctant to terminate the project, even if there
are many clear signals that the project is not going to be successful (Balachandra and
Friar, 1997).
It must also be known when and how often an NPD portfolio is assessed. Cooper
(2008) recommends doing this once a month and setting one day aside due to the
often busy schedule of senior management. The goal of an NPD portfolio
management tool is to assess the entire portfolio and make decisions on which
projects to continue, pause or abandon. Consequently a decision must be made at the
end of the NPD portfolio management meeting. This is only possible if the NPD
portfolio management method is user friendly and not too complex or appears as a
black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009). Cooper (2008)
identified deliverables overkill as a major threat to effective use of the Stage-Gate® system. Too much bureaucracy could also hamper the effectiveness of an NPD
portfolio management method. This can be overcome by clearly defining what the
deliverables are for each portfolio assessment. One must also be aware of overly
detailed templates. According to Cooper (2008) you must keep the deliverables and
their templates limited to the essential information needed to make the gate decisions.
Users should not be overloaded with unneeded data, but should be able to access
relevant data when it’s needed (Archer and Ghazemzadeh, 1998). Table 2.8
summarizes the criteria mentioned in this paragraph.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 23
Table 2.8 Overview of NPD portfolio management model usability criteria
• Governance roles and responsibilities must be defined (Cooper, 2008) • The NPD portfolio should be periodically assessed (Cooper, 2008) • The NPD portfolio management method is user friendly and not too complex or appears as a black box
(Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009) • Deliverables must be clearly defined for each portfolio assessment (Cooper, 2008) • Deliverables should be focused on the essential information needed for NPD portfolio management
decisions (Cooper, 2008) • Templates should not be overly detailed and be limited to the essential information needed for NPD
portfolio management decisions (Archer and Ghazemzadeh, 1998, Cooper, 2008)
2.3.6 Conclusion: Many criteria This chapter described what portfolio management is and why it is so difficult. At the
same time various criteria for NPD portfolio management were presented. These
criteria range from general goals to specific properties of a financial metric or from
technical criteria to criteria concerning usability. Table 2.9 shows an overview of all
criteria, which are subdivided into 7 sub categories. These criteria will be used to
evaluate the original Kalypso model.
Table 2.9 Overview of criteria for NPD portfolio management General Criteria
1.1. The NPD portfolio management tool should be usable and understandable for its users (Kester et al., 2009, Archer and Chasemzadeh, 1999,2007, Loch and Kavadias 2002)
1.2. The NPD portfolio management tool should use multiple types of methods and criteria (Literature review)
1.3. The NPD portfolio management tool should be able to rank projects based on pre-‐determined criteria (Archer and Chasemzadeh, 1999,2007)
1.4. The selection methodology should be determined before the NPD portfolio selection starts (Archer and Chasemzadeh, 1999,2007)
1.5. The NPD portfolio management tool should use a common measure which enable an equitable comparison (Archer and Chasemzadeh, 1999,2007)
1.6. The NPD portfolio management tool should not solely depend on financial tools (Cooper and Kleinschmidt, 1999)
1.7. The NPD portfolio management tool should incorporate qualitative criteria (Cooper et al. 1998, 1999) Process Criteria
2.1. The NPD portfolio method should be divided in phases (Archer and Chasemzadeh, 1999,2007) 2.2. A project should be re-‐evaluated each time it transfers to another stage (Cooper, 1990, 2008, Archer
and Chasemzadeh, 1999,2007) 2.3. Individual project development and evaluation should be based on the Stage Gate® process (Cooper,
1990, 2008, Archer and Chasemzadeh, 1999,2007) 2.4. The NPD portfolio should be reviewed periodically (Cooper et al., 1998, 1999, Archer and
Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007) 2.5. Each project should be assessed individually on its (financial) value (Archer and Ghasemzadeh, 1999). 2.6. The NPD portfolio management tool should take into account the interdependencies between projects
(Archer and Ghasemzadeh, 1999) 2.7. The NPD portfolio management tool should take into account the time dependent nature of project
resource selection (Archer and Ghasemzadeh, 1999)
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 24
Table 2.9 Overview of criteria for NPD portfolio management (continued) Strategic Criteria
3.1. The NPD portfolio management tool must assess projects on strategic fit (Cooper and Kleinschmindt, 1995, Cooper et al., 1998)
3.2. The strategy should be determined before the NPD portfolio selection process starts (Archer and Chasemzadeh, 1999,2007)
3.3. A clear strategy consists of the who, what, how, why and threats (Markides, 1999, Loch and Tapper, 2002) or consist out of four perspectives (financial, customer perspective, internal business process and learning and growth perspective) (Kaplan and Norton, 1996)
3.4. The NPD portfolio management tool should be able to align the portfolio to the strategy (Cooper et al. 1998, 1999)
3.5. The NPD portfolio management tool should be top-‐down and bottom-‐up (Cooper et al., 1998) 3.6. The strategy must make clear where and how many resources should be spent (Cooper et al., 1998) 3.7. Projects should be classified in advance of the portfolio selection (e.g. strategic Buckets) (Cooper et al.,
1998, 1999, Archer and Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007) 3.8. Projects should be assigned to buckets (Barczak et al., 2009)
Financial Criteria 4.1. The NPD portfolio management tool should strive to maximize the value of the portfolio (Cooper et al.,
1998, 1999) 4.2. The financial metric should incorporate the cost of development and launch (Cooper et al., 1998) 4.3. The financial metric should have no bias towards long term radical projects (Chao and Kavadias, 2007) 4.4. The financial metric should capture the flexibility of the development process (Newton et al., 2004) 4.5. The financial metric should be constrained by a critical resource (Cooper et al., 1998) 4.6. The financial metric should ignore sunk cost (Cooper et al., 1998) 4.7. The financial metric should incorporate the probability of success or failure (Cooper et al. ,1998,
Pennings and Lint, 1997) 4.8. The financial metric should be user friendly and not too complex or appear as a black box (Faulkner,
1996, Cooper et al., 1999, Kester et al., 2009). 4.9. The financial metric should use the Minimum Acceptable Rate of Return (MARR) and Time Value of
Money (TVM) (Remer and Nieto, 1995) Balance Criteria
5.1. The NPD portfolio tool must be able to visually depict the portfolio (Cooper et al., 1998) 5.2. Management must define what balance and imbalance is (Cooper et al., 1998) 5.3. The NPD portfolio should only show the for management most important diagrams (Cooper et al.,
1998) 5.4. The Bubble diagram X and Y-‐axis should be based on a organization’s strategic goals and objectives
Cooper et al., 1998) Risk Criteria
6.1. The NPD portfolio management tool should assess projects on risk (Halman and Keizer, 1994. Keizer et al., 2005
6.2. Risk should be assessed on three dimensions; likelihood, impact, and the ability to do something about it (Halman and Keizer, 1994)
6.3. Risk should be assessed on a sufficient number of criteria (Keizer et al., 1994) Usability Criteria
7.1. Governance roles and responsibilities must be defined (Cooper, 2008) 7.2. The NPD portfolio should be periodically assessed (Cooper, 2008) 7.3. The NPD portfolio management method is user friendly and not too complex or appears as a black box
(Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009) 7.4. Deliverables must be clearly defined for each portfolio assessment (Cooper, 2008) 7.5. Deliverables should be focused on the essential information needed for NPD portfolio management
decisions (Cooper, 2008) 7.6. Templates should not be overly detailed and be limited to the essential information needed for NPD
portfolio management decisions (Archer and Ghazemzadeh, 1998, Cooper, 2008)
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 25
The NPD portfolio management criteria listed in table above for are not mutually
exclusive and some even conflict. Having a clearly formulated strategy is a
prerequisite for many other criteria (e.g., what bubble diagrams to use). Some criteria
conflict; the need for a usable and understandable NPD portfolio management method
conflicts with the criteria that the NPD portfolio management should incorporate
probabilities, interdependencies between projects and the time dependent nature of
projects. Consequently, for each NPD portfolio management method, a well-
considered trade-off has to be made between these criteria (usability versus
complexity).
2.4 Portfolio methods Now that we have discussed the criteria for NPD portfolio management we move on
to the actual NPD portfolio management methods available. As described in the
introduction, surveys have shown that the solutions provided by academic research
have found little use in practice (Shane and Ulrich, 2004). Scientific models often
require lots of data: e.g., information on the financial results, resources needs timing
plus probabilities of completion and success for all projects. Practitioners often don’t
have the required data and when they do get results, practitioners doubt them due to
lack of transparencies and robustness (Loch et al., 2002). Managers often perceive
solutions provided by scientists to be too difficult to understand and use (Cooper et
al., 1999). Practitioners often use multidimensional decision making tools or ranking
methods. A drawback of these tools is that they rely on an ad hoc list of dimensions
and their decision makers often manipulate these methods to generate desired
outcomes instead of using them as true decision support tools (Chao and Kavadias,
2007 and Loch and Kavadias, 2002). These tools do help management “think
through” a problem but they lack theoretical or empirical basis (Loch and Kavadias,
2002). Senior managers, R&D managers, and project managers are forced to make
resource allocation decisions based primarily on intuition or heuristic rules (Chao and
Kavadias, 2007). Several NPD portfolio management methods and tools have been
briefly mentioned during the description of the various criteria for NPD portfolio
management. These and additional methods are discussed in more detail in the next
part. A more extensive and comprehensive review of different NPD portfolio
management methods is in the literature review. What becomes clear is that there is
not a one-tool solution. In order to develop an NPD portfolio method that meets all
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 26
criteria set in the previous part the NPD portfolio management tool most be composed
out of multiple methods.
2.4.1 Financial methods Financial methods are the most frequent used NPD portfolio decision methods
(Cooper et al., 1999, 2001). Projects are selected based on financial indexes.
Different criteria can be used like Return on Investment (ROI), Internal Rate of
Return (IRR) or Net Present Value (NPV). The application of financial analysis to
projects or product lines is conceptual appealing because it allows management to use
the same criteria to evaluate each component of a business which stockholders use to
evaluate the organization as a whole (Cardozo et al., 1983). Senior management
formulates the financial criteria for NPD projects. Cooper and Kleinschmidt (1999)
found that organizations that only use financial tools perform less; financial methods
produce portfolios with poor-value projects, too many projects for the resources
available and gridlock in the pipeline. Another drawback of financial methods is that
the outcome can be unreliable due to the fact that NPD project data is often
speculative until market launch, there is no accurate data (Linton et al., 2002).
Financial methods also tend to disfavor advanced technology projects due to the long-
term payoffs and high likelihood of failure (Chao and Kavadias, 2008). For example
NPV often valuates inaccurately long-term and risky projects. This is because NPV
makes the assumption of managerial inflexibility, projects will proceed as planned
regardless of what happens in the future. The NPV doesn’t consider the option of
expansion, abandoning or modification of an NPD project although these options can
add value to the project (Newton et al., 2004). The NPV also assumes an immediate
initiation of the capital project (i.e. management has no leeway about the timing of the
investment).
2.4.1.1 Probabilistic financial methods Cooper et al. (1998) recommend the Excepted Commercial Value (ECV) method for
maximizing the value of the portfolio. The ECV seeks to maximize the value or
commercial worth of the portfolio, subject to certain budget constraints. The ECV
considers the future stream of earning from a project, the probabilities of both
commercialization and development cost (Cooper et al. 1998).
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 27
Exhibit 2.1 Determination of expected commercial value of project
!"# = !"# ∗ !!" − ! ∗ !!" − !
$ECV= Pts= Pcs= $D= $C= $NPV=
Expected Commercial Value of the project Probability of technical success Probability of commercial success (given technical success) Development cost remaining Commercialization (Launch cost) Net present value of the project’s future earnings (discounted to the present)
Source: Cooper et al. (1998) page 24
Advantages of the ECV over other financial metrics is that the ECV recognizes, that if
a project is halted partway through, certain expenses are not incurred and that the
development process is step-wise. The ECV method acknowledges that management
has GO/Kill options along the way, these options reduce the risk of a project, and that
the “correct value” of a project could be determined via a decision tree approach
(Cooper et al., 1998, Faulkner, 1996). The ECV penalizes projects that are years away
from launch by discounting all dollar amounts today (not just from launch date).
Both financial and probabilistic financial methods are most effective in later stages of
the development process. Stage 0 and 1 of the development process are characterized
by high uncertainty and consequently lack reliable information for the determination
of a quantitative financial metric. In later stages (from stage 2, development onward)
the information used to determine the quantitative financial metric become more
reliable. Hence, quantitative financial methods are best suited for the assessment of
projects from gate 3 onwards.
2.4.2 Strategic methods Strategic methods split resources into smaller strategic buckets. A strategic bucket is a
collection of NPD programs that are aligned with a particular innovation strategy
(Chao and Kavadias, 2007). The role of the strategic buckets method is to translate
the business’s strategy into clearly defined areas (buckets) and to decide resource
allocation to each (Cooper et al., 1998). Strategic buckets is a top down approach, the
number and type of buckets are determined by senior management and are based on
an organization’s strategy. Projects are assigned to only one bucket until the resources
of the bucket are depleted. The criteria for placing a project in a bucket can differ per
Development $D
Launch$C
TechnicalSucces
CommercialSucces $NVP
Commercial Failure
Technical Failure
Pts
Pcs
Yes
No
Yes
No
$ECV
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 28
bucket. Barczak et al. (2009) found that top performing organizations use the strategic
buckets methods more often than the rest is. An advantage of strategic buckets is that
it prevents totally different projects from competing for the same (scarce resources).
Project selection inside a bucket can be based on financial metrics. Note that the
strategic buckets method doesn’t have a tool for assigning projects to the buckets,
strategic buckets method decrease the scope of the NPD portfolio problem but doesn’t
resolve it.
Strategic methods are best used in the early stage of the development process (stage 0,
1 or 2). Strategic methods help organizations to implement their strategy by
dedicating resources to a specific strategic goal. After stage 2 the cost of a project
increase significant during the project development in stage 3. Therefore projects
have to be assigned to a specific bucket before stage 3. Consequently projects that
don’t fit any strategy should be filtered out at latest at gate 3. After gate 3 strategic
methods loose some of their importance.
2.4.3 Checklists & Scoring models Checklists and scoring models are examples of a bottom-up approach. Checklists are
used for examining whether or not a product met the requirements set. Projects are
ranked according to number of criteria’s met. Every criteria has an equal weight and
is consequently equally important. It occurs often that not all criteria are equally
important. Scoring models, opposite to checklist, can take the weight of different
criteria into account for calculating an overall score. Scoring models are used to
determine, various criteria, table 2.10 shows which criteria are used by organizations
rank by popularity.
Table 2.10 Criteria used to Rank Projects
1. Strategic fit/ leverages core-‐ competencies
2. Financial reward / pay-‐off 3. Risk and probability of success 4. Timing 5. Technological capability 6. Commercialization capability 7. Protectability 8. Synergy between projects
Source: Cooper et al. (2006) page 22
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 29
Determining the weighting and scores is a subjective process that depends as much on
personal opinion as on hard numbers (Coldrick et al., 2003). Projects are ranked
according to the overall score and can be added or deleted without affecting the scores
of the remaining projects. An advantage of scoring methods is that it is possible to
rate projects on multiple criteria from economic to non-economic. Scoring models are
effective analytical tools when data requirements and complexity of more sophisticate
approaches are considered not justified (Dean et al., 1965). According to Paolini and
Glaser (1977) scoring models are most applicable during basic research and
exploratory development. Another advantage of scoring models is that they are
quantitative enough to possess a certain degree of rigor but not so complex to mystify
and discourage potential users (Henriksen et al., 1999). Cooper (1981) states that
scoring models are often seen as oversimplifications, they attempt to reduce a
complicated decision situation to a simple equation yielding a composite score. A
drawback of scoring models is that criteria should be mutually exclusive, in real life,
criteria are often not independent (Cooper, 1981). Scoring models are less appropriate
when there is a high degree of interdependence between projects, when activities and
results of one project depend on the activities and results of a different project
(Henriksen et al., 1999). However, Cooper and Kleinschmidt (1999) found that
scoring models yield high performance results in terms of producing a portfolio of
high value products.
Checklists and scoring models are usable during the entire development process
depending on the criteria used. These criteria should increase in depth and scope
going from gate to gate during the development process. In early stages, the criteria
should be broad while later in stages the development process, when more detailed
information is available, projects should be assessed on specific and detailed criteria.
2.4.4 Bubble diagrams A bubble diagram is a diagram which represents information visually in the form of a
series of bubbles. Projects are plotted on an X-Y plot or map. The axes vary, the most
popular chart is the risk-reward bubble diagram, where the NPV is plotted versus
probability of technical success (an example of a bubble diagram is shown in figure
2.3).
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 30
Figure 2.3 Risk-‐Reward Bubble diagram
Pro
babi
lity
of T
echn
ical
Suc
cess
Reward (NPV)
1256 3 078
High
Low
Pearls
Oysters White Elephants
Bread and Butter
Circle size = resources annual Source: Cooper et al. (2001), page 396
Other types of bubble diagrams are; technical newness by market newness, technical
feasibility by market attractiveness and competitive position by project attractiveness
(Cooper et al, 2001). Bubble diagrams should not be used for making portfolio
decisions (Cooper and Edgett, 2001). They depict the current state of the portfolio and
where resources are going. A visual depiction portrays balance much better than
numbers or lists (Cooper and Edgett, 2001). Bubble diagrams provide a start point for
discussions on what should be and how your resources should be allocated (Cooper
and Edgett, 2001). Hence bubble diagrams are not recommended for making
decisions because they only depict the current situation, rather, they should be used
for information display.
Bubble diagram are only used for depicting the current state of the portfolio. Bubble
diagrams can be used during the entire development process. Users should be aware
that the information used in diagrams differs in reliability depending on the
development stage a project is in. E.g., a risk-reward bubble diagram of the entire
NPD portfolio will depict projects of all stages of the development process, the
information used for depicting a project in stage 0 will be far less reliable than the
information for depicting a project in stage 4.
2.4.5 Conclusion: Which Method? Every method has its advantages and disadvantages, some methods are better suited in
the beginning of the NPD process while other are more suited during the end.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 31
Organizations use different criteria at each gate, some methods are more useful than
other depending on the criteria used, hence organizations must use multiple methods.
In the second part of this chapter a list with criteria was formulated for an NPD
management model, this list is used to evaluate the five methods mentioned in the
previous paragraph. Each method received a score from 1 to 7 point scale for each
criteria. This is assessment is done by the author. If a method receives a score of 1 the
NPD portfolio management method doesn’t meet the criteria at all. When a project
receives a score of 7 the NPD portfolio management method complies totally with the
criteria. In this simple assessment the average score determines the overall score of
the method. The results and substantiation (when deemed necessary) of the
assessment are shown in appendix 4. A summary of this assessment is shown in table
2.11 accompanied with the main advantages and disadvantages of each method.
Table 2.11 overview of different NPD portfolio methods with their assessment score, advantages and disadvantages
Assessment score Advantage Disadvantage
Best used at gate
Financial methods 3,09
Conceptual appealing, popular metric in business
Disfavors high risk, very innovative projects. Less useful in the early stages of the development process
3, 4 or 5
Probabilistic financial methods 3,48
It doesn’t disfavor high risk projects, popular metric in business
The validity of the outcomes of these probabilistic methods depends on the estimates made. Less useful in the early stages of the development process
3, 4 or 5
Strategic Methods
3,98
Makes it possible to align portfolio to strategy, doesn’t disfavor high risk projects
How to determine what projects are put into the bucket? It does decrease the scope of the NPD portfolio problem but doesn’t solve it.
1, 2 or 3
Checklist and Scoring methods
3,73
Ability to rate projects on multiple factors from economic to non-‐economic. Effective analytical tool when data requirements and complexity of more sophisticate approaches are considered not justified
Can be seen as oversimplifications. Less appropriate when there is a high degree of interdependence between projects
All gates*
Mapping approaches or bubble diagrams
3,50
Depict the current state of the portfolio and where resources are going
Should not be used for making decision, only for information
All gates
Poor 1 2 3 4 5 6 7 Excellent *Criteria differ per gate
The first thing that strikes is that each individual method receives a relatively low
score on the assessment. Strategic methods receive the highest score, a 3,98 out of 7.
These scores are based on the how well each method meets the criteria set for an NPD
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 32
portfolio management method. Hence none of these NPD portfolio management
methods is sufficient enough to be used as the only NPD portfolio management tool
of an organization! This is not surprising, each method is best suited for assessing
projects in a specific stage. When a model only uses one method you expose a project
to the limitation and biases of such a method. There is no single criteria or value
driver that is able to correctly represent the value of a project throughout the entire
development process. Financial methods for example are less suited for assessing
projects in early development stages due to the lack of reliable data, when more
reliable data becomes available during later development stages financial methods
become more usable. Cooper and Kleinschmidt (1999) found that organizations that
only use financial tools perform less; financial methods produce portfolios with poor-
value projects, too many projects for the resources available and gridlock in the
pipeline. Organizations use more than one NPD portfolio method to compensate the
drawbacks the different methods (Cooper et al., 1999, 2001, 2008). The problem with
current portfolio methods is that most of them use different measurements to rank
projects, it impossible to compare projects ranked with different methods How to
compare a financial ranked project with a scored project or a project selected based on
consensus? The only way to objectively compare projects is that they are assessed on
common measures. This will allow an equitable comparison of projects during the
selection process (Archer and Ghasemzadeh, 1999). Even if one uses the same
method for different phases it is still very difficult to compare because more data is
available near the end of the process.
In the next chapter the NPD portfolio management tool developed by Kalypso is
described. Whether this tool performs better than the current NPD portfolio
management methods will become clear in chapter 5.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 33
Chapter 3: Description the Original Kalypso model This chapter describes the NPD portfolio model developed by Kalypso. First the goal
of the model is explained. Next we dive into the process, various inputs are discussed
and the output is described. Accompanied are various text boxes, in which two
(coded) projects and their journey through the model is described. One project is the
development of a desert and the other project concerns the development of a cereal
bar. This chapter ends with a conclusion.
3.1 Goal Kalypso model Kalypso developed an NPD portfolio management model for one of its clients. This
model enables the client to ranked and prioritize NPD ideas, concepts and projects.
Projects that are more attractive to the client should be ranked higher and should be
better resourced than less attractive projects. This model should also be able to give
insight into the composition of the portfolio e.g. how many project are in what
development stage, what is the distribution of risk in the portfolio with respect to the
earning, etc. The client operates in the food processing industry, they operate globally
although most of the products are developed locally for local markets. The client has
currently a wide spectrum of approaches to prioritize initiatives, ranging from no
prioritization to a complex prioritization matrix. The client has a Product Lifecycle
Management (PLM) maturity level 0, which is described by Batenburg et al. (2005) as
a level where project investments are done on an ‘ad-hoc’ basis only. Nobody is
responsible for PLM and there is no vision available for PLM. Therefore, there are no
consistent PLM processes and supporting systems. At this level, information about a
product is scattered throughout the organization, which hinders strategic decision-
making. PLM maturity refers to the cumulative nature of the deployment process, the
organization has to go through different stages of growth before PLM is implemented
at all levels and connects all managerial aspects (for a description of the different
maturity levels see appendix 2) (Batenburg et al., 2005). The NPD portfolio
management tool developed by Kalypso is the first step towards a higher PML
maturity. However NPD portfolio is tailored to the client’s current maturity level, this
has led too some “strange design” choices that will be discussed later.
One of the objectives of the Kalypso Model is to establish a standard prioritization
matrix for all of the client’s R&D initiatives around the world. The Kalypso model is
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 34
constrained by the metrics used at the client, e.g. Net Sales Value and the Gross Profit
minus Marketing expenses are the internal financial metrics and must be used. At the
client there are three types of R&D projects, innovation, range maintenance and cost
out. Innovative projects are projects that are new to the client, range maintenance are
changes in existing products while cost out projects concerns value engineering of
products. The main difference between range maintenance and cost out is that the first
one is done reactively to market and supply changes and the latter one is done
proactively. Range maintenance and cost out projects are often incremental of nature.
Consequently, there is already al lot of information available for cost-out and range
maintenance projects. The client has implemented a stage gate process, which is
depicted in exhibit 3.1.
Exhibit 3.1 implemented Stage Gate process
The last stage, stage 6 does not have a direct influence on the NPD portfolio selection
process and shall be ignored from now on. The portfolio is reviewed once a quarter by
senior management.
Develop ideas into consumer validated concepts
Con
cept
Qua
lifica
tion
Develop R&D prototype and marketing mix trough sequential recyclingPr
oduc
t Qua
lifica
tion
Develop, test and optimize product: finalize marketing and sales plans
Mar
ket Q
ualifi
catio
n
Full implementationof launch plans. Monitor inmarket performance
Laun
ch
Project closed
Post
-Lau
nch
Aud
it
Deep dive into the opportunity
Idea
tion
Stage 1 Stage 3 Stage 4Stage 2 Stage 5 Stage 6
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 35
3.2 Process The process diagram of the Kalypso model is depicted in exhibit 3.2, there are various
inputs which are used to calculate the project’s rank. In this paragraph the whole
Kalypso model process will be described in detail. Next we describe the various
inputs.
Exhibit 3.2 Kalypso model process diagram
*!" = !"# − !"#$ !" !""#$ !"#$ −!"#$%&'() !"#$%$%
Projecttype
Product category
Up-aging
Geography
Area of focus
Risk Category
Project Stage
ProjectStatus
Urgency
Capability
Portability and
scalability
Stage 3,4 & 5
Project priority
Financial value driver *
Strategic fit value driver
Risk value driver
Stage 1 & 2
Project priority
Financial value driver
Strategic fit value driver
Risk value driver
10 %
80%
10%
60%
10%
30%
High - 15Medium - 5Low -1
High - 15Medium -10Low - 1
High - 1Medium - 5Low - 15
High - 15Medium -10Low - 1
High - 1Medium - 5Low - 15
Value Category Value Driver
OutputProcessInput
Project priority
Project priority
Ranked Projects based on Priority
Project Information
Nametype
Project Owner
Work Country
Project Country
LaunchDate
GPConstant ×15
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 36
3.2.1 Input There are three kinds of input for the Kalypso model, project information, value
categories and value drivers. Project information contains regular project information
such as project name, project description and project number. Value categories are
input variables that can be used to identify or specify a project (they are used to
describe a project). An overview of all the value categories used in the Kalypso model
is shown in table 3.1.
Table 3.1 Overview of value categories Value Category Description 1. Project type The kind of project, innovation, range maintenance or cost out 2. Project category To which main product group does the project belong 3. Up-‐aging A project that aims to extended the age range of customers 4. Geography Where is the project launched, Europe, North America, Asia Pacific or the rest of the world 5. Project stage In which stage is a project 6. Area of focus Product, packaging, process conditions and scientific activities 7. Project status Project status indicates the overall state of a project and general point in the project
lifecycle: active, completed, on hold or canceled Only one value category has an influence on the ranking of projects; project stage.
The stage where a project is in determines what weight is allocated for each value
driver.
Value drivers are the third type of input. The overall project value is based on these
value drivers. Value drivers can and should be reduced to a number or a choice that
can be put in a table or plotted in a chart. There are 6 different value drivers identified
during the Kalypso model development project. Table 3.2 gives an overview of the
different value drivers.
Running Example Project Name Desert X Cereal Bar Y Project Owner Mr S. Mr. T. Project Country Italy India Work Country Italy India Launch Date Feb 2010 March 2010 Basic project information is entered.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 37
Table 3.2 Overview of value drivers Value driver Description 1. Financial Metrics that are used to capture the fiancial value of the project 2. Strategic Fit Represents the alignment of the project with the global strategy 3. Risk The likelihood and impact of an adverse event resulting form a 4. Urgency Projects that are urgent must be completed regardless of value drivers 5. Capability Does the development of the project fit in whith the capabilities of the
organization 6. Portability and scalability Can the product be launched elsewhere
Only three of the six value drivers are used for ranking a project; financial, strategic
fit and risk. The weight of each value driver differs per stage. In early stages the
financial and risk value drivers weigh less because the information on which these
values drivers are based are often vague and unreliable. Strategic fit on the other hand
is an important consideration in the early stages of the development process. Value
drivers are also used to visually depict the composition of the portfolio e.g., risk and
financial. Each value driver is described in the next part.
3.2.1.1 Financial value driver There are two kind of financial value drivers, a qualitative and a quantitative one. The
qualitative value driver is used for projects in the first two stages of the development
process. In these stages there is no reliable financial data available. The qualitative
financial value driver is based on the probability of financial success. Market size,
marked need and competition are important indicators for financial success and are
therefore used for assessing the probability of financial success (Montoya-Weiss and
Running Example Continued Desert X Cereal Bar Y Project Type Innovation Innovation Product category Infant Nutrition Area of focus Product Product Up-‐aging No Yes Geography Europe Asia Pacific Project stage Stage 3 Stage 2 Project status Active Active Both projects are new and thus belong to the innovation product type. Desert x belongs to the infant product category, which are products for babies (0-‐3 year). The two products are new to the organization, hence the focus lies on the product (not packaging or process). Cereal Bar Y is in the Nutrition product category which focuses on specific markets (special diets e.g., diabetics or gluten free). Cereal Bar Y could also be targeted at customers of various ages, Desert X not (only babies). Desert X is targeted at the European market while Cereal Bar Y is aimed at the Asia Pacific market. Desert X is development stage 3 whereas Cereal bar Y is in stage 2. Both projects are actively in development.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 38
Calantone, 1994, Cooper et al., 1995, Henard and Szymanski, 2001, Pattikawa et al.,
2006). In the Kalypso model there are three probability levels for financial success,
high, medium and low. Table 3.3 shows on what criteria the qualitative financial
value driver is based.
Table 3.3 Qualitative criteria for assessing the probability of financial success • High The market is large and growing, the idea or concept could be highly profitable, and the
need for the product is significant, competition is vulnerable. • Medium The market is moderate but growing, the idea or concept could be moderately profitable,
the need for the product has a slightly above average advantage over what is available, competition is moderate and sustainability of competitive advantage is 1-‐2 years.
• Low The market is slight and stable, the idea or concept has low-‐medium profitability, the need for the product is as yet uncertain, and competition is strong.
The quantitative financial value driver is used if a project is in stage 3, 4, or 5. In
these stages reliable financial is available because of market research. The metric used
is the Gross Profit Margin minus Total Marketing Expenses (GP). The GP is
determined for the 2nd full fiscal year after the launch year. For example, if a product
was launched in October 2010, the GP should be for 2012. The idea is to reflect
market penetration and defrayed marketing costs. The calculation of the GP is as
follows:
!" = !"# − !"#$ !" !""#$ !"#$ −!"#$%&'() !"#$!!"!
Where NSV stands for Net Sales Value. The NSV is revenue generated form the sales
of a specific project. The GP and NSV are standard metrics used at the client and are
therefore used in the Kalypso model.
Running Example Continued Desert X Cereal Bar Y Financial value driver $5.250.000 Low Because Desert X is in stage 3 a quantitative prediction of the Gross Profit Minus the Total Marketing Expenses can be made. Cereal Bar Y on the other hand is only in stage 2, therefore a qualitative assessment is made. Cereal Bar Y is targeted at a stable market where the margins are low. Competition is the Cereal Bar market is strong and the need for the product is not known. Hence the financial value driver for Cereal Bar Y is low.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 39
3.2.1.2 Strategic fit value driver The second value driver used to rank a project is strategic fit. This value driver
indicates how well a project is aligned with the strategy of the client. The assessment
of strategic fit is based upon qualitative criteria. There are three levels of strategic fit,
high, medium, and low. The criteria and the corresponding levels of fit are shown in
table 3.4.
Table 3.4 Criteria for determining level of strategic fit High The idea or concept keeps the client on track with areas of performance (AOP)/strategic plan
objectives Medium • The idea or concept keeps emerging markets growing at current rate.
• The idea of concept strengthens competitive position in developed markets • The idea or concept produces a competitive "game changing" advantage • Leverages global scale
Low The project does not fit with the client’s strategic
In the Kalypso model is strategic fit the most important value driver when a project is
in stage 1 or 2. In later stages of the project development process the weight of
strategic fit decreases because the other value drivers become more reliable.
3.2.1.3 Risk value driver The last value driver used for ranking projects is risk. Risk is the likelihood and
impact of an adverse event resulting from a project. Risk is divided into three
subcategories; technical risk, commercial risk and regulatory risk. The subcategory
with the highest risk determines the overall risk of a project. For the determination of
technical risk three factors are important; familiarity, time and resources. Familiarity
implies how much expertise the client has in developing a project, is it a totally new
venture or does the client has a lot of expertise in this field. The more familiar the
client is with a project the more likely it is that the product is successfully developed.
Time entails the amount of time available for the development of the project. Hence,
if there is plenty of time available for development the risk will be low and visa versa.
The third factor that determines technical risk is resources. The availability of key
Running Example Continued Desert X Cereal Bar Y Strategic fit value driver High Medium The strategic fit value driver of Desert X is high since it keeps Italy on track while the strategic fit of Cereal Bar Y is medium because it keeps emerging markets growing at current rate (The author does not know the precise considerations of this assessment).
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 40
resources is an important factor for the successful development of a project. The lack
of available resources poses a risk for the development.
The second subcategory is commercial risk. Commercial risk is about the probability
of unfavorable events due to poor market response and price point. Another part of
commercial risk is product introduction speed and method, are there distribution
channels available or need they be build? Proprietary position and patent defensibility
are also part of commercial risk. The last part of commercial risk is the probability of
a response from the competition with a similar or better product.
The last category is regulatory risk. Two factors are looked at when assessing
regulatory risk, regulatory and label claim. Regulatory is about the existence of
regulatory issues and how much effort is needed to comply. Label claims are specific
risks for the industry the client operates in.
Each subcategory is assessed on a five-point scale where 5 represents the highest and
1 the lowest risk. Qualitative criteria are used for determining the risk level of each
subcategory. The highest risk score determines the overall risk level of a project.
There are three levels of risk, high, medium and low. The weight of the risk value
driver is low when a project is in stage 1 or 2. Risk increases in importance when a
project is in stage 3, 4 or 5 because the information used to assess the risk of a project
becomes more reliable during later stages. The risk assessment tool is shown in
exhibit 3.3.
Exhibit 3.3 Criteria for determining project risk level Risk Level Technical Risk Commercial Risk Regulatory Risk
5 (Highest)
• Familiarity – This has not been not done before and is new to the world.
• Consumer – Little or no information on the consumers or price point is available.
• Regulatory – Major regulatory issues exist. It is likely the product will need radical effort to comply.
• Time – There is extremely little time to complete the project.
• Customer – Channels required are new to the world.
• Label Claims – It is very likely the product label claims cannot be communicated or will require extremely large investments in scientific research. These label claims are critical to competitive position.
• Resources – Key resources are highly likely to be unavailable.
• Competitor – Simultaneous or pre-‐emptive competitor response likely.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 41
Exhibit 3.3 Criteria for determining project risk level (continued)
4
• Familiarity – This is new to the client, but not new to the world.
• Consumer – The need is unarticulated by consumers, but is inferred from research. There is little data on price point.
• Regulatory – Regulatory issues exist. It is likely the product will require significant effort to comply.
• Time – There is significantly less time to complete the project than desired.
• Customer – New channels are required, but these channels have been used by others.
• Label Claims – It is likely the product label claims cannot be communicated or will require significant investment in scientific research. These label claims are important to competitive position.
• Resources – There is a strong chance that key resources are unavailable.
• Competitor – Immediate post-‐launch response likely to be substantial and vigorous.
3
• Familiarity – This is not new to the client, but it is not one of our present skills.
• Consumer – The need has been articulated by consumers though not validated. Price point data is available but not validated.
• Regulatory – Regulatory issues exist. The product may require some effort to comply.
• Time – The time period to complete the project is less than desired. However, this is not likely to severely impact the project outcome.
• Customer – New channels or customer relationships are required, although existing channels can be used as a foundation.
• Label Claims – There is a possibility that product label claims cannot be communicated as desired. These label claims have a moderate impact on competitive position.
• Resources – There is a moderate chance that key resources are unavailable.
• Competitor – Immediate post-‐launch response likely to be limited.
2
• Familiarity -‐ This is a good fit with the client core competencies, but a similar project has not been done before.
• Consumer -‐ The need has been articulated by consumers and validated. Price point has been validated.
• Regulatory -‐ Minor regulatory issues exist. It is unlikely that the product will require effort to comply.
• Time -‐ There is slightly less time to complete the project than desired. However, this is very unlikely to negatively impact the project.
• Customer -‐ Need to modify existing channels or customer relationships to accommodate new product.
• Label Claims -‐ There is a small possibility that the product label claims cannot be communicated as desired. These label claims are not crucial to the competitive position
• Resources -‐ There is a small chance that key resources are unavailable.
• Competitor -‐ Delayed competitive response likely.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 42
Exhibit 3.3 Criteria for determining project risk level (continued)
1 (Lowest)
• Familiarity -‐ The client has experts in this and have done this before.
• Consumer -‐ The need has been highly articulated by consumers, and validated. Price point has been extensively validated.
• Regulatory -‐ There are no known regulatory issues related to the product.
• Time -‐ There is sufficient time to complete the project.
• Customer -‐ Existing channels and customer relationships are in place and can be used.
• Label Claims -‐ There are no known issues related to product label claims.
• Resources -‐ There are no foreseen issues related to resource availability.
• Competitor -‐ No significant competitive response anticipated
3.2.1.4 Portability and Scalability, Urgency and Capability The value drivers portability and scalability, urgency and capability are not used for
ranking projects. The Kalypso model is used locally, the value driver portability and
scalability is better assessed globally, a local R&D unit isn’t able to assess the global
considerations that are needed for the portability and scalability assessment.
Consequently, the portability and scalability value driver is not used in the model. The
value driver capability is not included because this value driver is covered in the risk
assessment. Urgency is not included but it does impact the ranking of the project.
3.2.2 Processing & Output Once all value categories and value drivers are entered into the model the processing
of this data begins. The final ranking is based on three value drivers, financial,
strategic fit and risk. A project can receive minimal 1 point and maximal 15 points for
Running Example Continued Desert X Cereal Bar Y Risk value driver High Medium
Desert X: There is less time available for the development as desired although it is not likely that this will severely impact the outcome (risk level 3). The risk lies in the competition, there is a high probability that the competition will response vigorously (with competing products) to the launch of Desert X. (risk level 4) There are no known regulatory issues, it is very unlikely new regulations are created that will impact the product and there are no public relations risks (risk level 4). The overall risk ranking of Desert X is high due to the high probability of competitor response. Cereal Bar Y: There is sufficient time for the development of Cereal bar Y and the client is familiar with this kind of project (risk level 2). The customer needs have been articulated and validated (hence the customer group is very specific) (risk level 2). A low chance exists that new regulations will be introduced that impact the project (risk level 2). The overall risk level of Cereal Bar Y is medium (level 2).
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 43
each value driver. There is one exception if a project is in stage 3, 4, or 5, in this case
the amount of points is based on the Gross Profit minus the Marketing Expenses (GP).
The GP is divided by a constant and multiplied by 15. The distribution of points per
category is shown in table 3.5.
Table 3.5 Distribution of points per value driver Value driver Stage Level Score Financial Stage 1 or 2 High 15 points
Medium 5 points Low 1 point
Stage 3, 4 or 5 GP!"#$%&#%
∗ 15
Strategic fit Stage 1, 2, 3, 4 or 5 High 15 points Medium 10 points Low 1 point
Risk Stage 1, 2, 3, 4 or 5 High 15 points Medium 5 points Low 1 point
As mentioned before not all value drivers are weighted equal, the weight of each
value driver depends on the stage a project is in. In stages 1 and 2, is strategic fit the
most important value driver. The financial and risk value driver are less important
because they are difficult to determine due to the lack of reliable data. Strategic fit
becomes less important after stage 1 and 2 because projects that don’t fit the
organization should be filtered out. In addition the information used for the other two
variables becomes much more reliable in stage 3, 4 or 5. If a project is in stage 3, 4 or
5 the financial value driver weighs most. Table 3.6 shows an overview of the different
weights of each value driver per development stage.
Table 3.6 Weighting of each value driver for each phase Stage Financial Strategic Fit Risk Stage 1 10% 80% 10% Stage 2 10% 80% 10% Stage 3 60% 10% 30% Stage 4 60% 10% 30% Stage 5 60% 10% 30%
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 44
For the calculation of the NPD portfolio score each value driver score is dived by 15
and summed up.
!"# !"#$%"&'" !"#$% =!"!! ∗ !!15 +
!" ∗ !!15 +
!"! ∗ !!15
Where: ai = weight of Strategic Fit Score (SFS) 1 < ai <100 bi = weight of Risk Score (RS) 1 < bi <100 ci = weight of Financial Score (FS) 1 < ci <100 i = stage project is in ∈ (0,1,..,4)
The project with the highest NPD portfolio score suits the client best and should
therefore be resourced first and best. Ranked projects are divided into three
categories; gold, silver and bronze.
3.2.3 Charts and Diagrams The Kalypso model produces various charts and diagrams. These charts and diagrams
provide a visual depiction of the NPD portfolio. Table 3.7 lists various charts and
diagrams the Kalypso model produces. Most of the charts and diagrams consist out of
a value driver compare to a value category.
Table 3.7 List of charts and diagrams the Kalypso Europe model produces
• Number of and types of projects diagram • Number of urgent projects diagram • Number of gold, silver and bronze projects diagram • NSV by Launch Month chart • Average priority by launch chart • Number of projects by category (incl. gold, silver and bronze) (infant or nutrition) chart • Number of projects by type (incl. gold, silver and bronze) (e.g. innovation, cost out) chart • Number of projects by area of focus (e.g. product, packaging etc.) (incl. gold, silver and bronze) chart
Running Example Continued Desert X Weighting Cereal Bar Y Weighting Financial value driver 10534000 à !.!"#.!!!
!""""""∗ 15 =
39.4 points
60% Low à 1 point 10%
Strategic fit value driver
High à 15 points 10% Medium à 10 points 80%
Risk value driver High à 5 30 % Medium à 5 10% Overall Score 1,78 0,57 The weighting per stage (Desert X, stage 3 and Cereal Bar Y, stage 2) is based on table 3.6. Each weighted value driver is dived by 15, the sum of these three weighted value drivers is the final score. Desert X has a higher score than Cereal Bar Y, hence when management has to choose between the two projects, project Desert X is the better option.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 45
Table 3.7 List of charts and diagrams the Kalypso Europe model produces (continued)
• Number of projects by stage (incl. gold, silver and bronze) chart • NSV by risk chart • Gross profit by risk chart • Stage by NSV chart • Stage by gross profit chart • Stage by priority chart • NSV by project cost chart • Gross profit by project cost chart • Average project priority by category (infant, nutrition) chart • Average project priority by project type (e.g. innovation, cost out, etc.) chart • Average priority by area of focus (e.g. product, packaging) chart • NSV by strategic fit chart • Gross profit by strategic fit chart • Percentage of stage 1-‐2 projects by market attractiveness chart
3.3 Usability In the Kalypso model responsibilities are clearly defined. Exhibit 3.4 gives insight in
the task and responsibilities in different levels of management at the client.
Exhibit 3.4 Responsibilities in the Kalypso Model
The president of the organization is responsible for defining a global strategy and for
monitoring the high-level key performance indicators essential for the strategy. The
global strategy is the basis for the R&D strategy developed by the global VP of R&D
and global marketing. The regional R&D strategy is developed by the regional VP
and is based on the global R&D strategy. In the end the global strategy is cascaded
down until country level where the country president, R&D director, marketing
!"#$%&#'(
)*+,-*./!.012
)*+,-*.3-"4#5'6
)*+,-*.!!37"+8#$$.9:'#"
0#6%+'-*./!
;+<'("=.!"#$%$#'(
012.2%"#8(+"
3-"4#5'6.2%"#8(+"
><77*=.;?-%'.2%"#8(+"
@%'-'8#.2%"#8(+"
>#(.)*+,-*.>("-(#6=
>#(.0#6%+'-*.012.>("-(#6=
>#(.;+<'("=.012.
>("-(#6=
/-*%&-(#.;+<'("=.012.
>("-(#6=
2#A#*+7.012.
!+"B+*%+
/-*%&-(#.012.
!+"B+*%+
CD#8<(#.012.
!+"B+*%+
;+'&<8(!30.
E>(-6#.6-(#."#A%#:$FE3+'(?*=F
;+'$+*%&-(#!30.2-(-
;+'&<8(.!+"B+*%+.
0#A%#:.-'&.!"+A%&#.@##&,-84EG<-"(#*=
0#A%#:.-'&.>#'&.
@##&,-84
0#A%#:.-'&.>#'&.
@##&,-84
975H%I#.-'&.
CD#8<(#.+'.012.7+"B+*%+
>#(.)*+,-*.012
>("-(#6=
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 46
director, supply chain director and the finance director formulate together a country
specific R&D strategy. The regional VP validates this strategy. The R&D director,
marketing director, supply chain director and the finance director develop an NPD
portfolio which is validated by the country president. The R&D director, marketing
director, supply chain director and the finance director are responsible for the
execution of the R&D portfolio. Each month individual projects are assessed and
valuated by the country president, R&D director, marketing director, supply chain
director and the finance director. The global ppm process owner consolidates all the
individual assessments from different countries. These collected individual
assessments are reviewed (quarterly) in total by the president and global VP of R&D
and provide feedback. Subsequently this feedback is reviewed by the regional VP
who sends it thereafter to the country director. The country director also reviews the
feedback and sends his feedback towards the R&D director, marketing director,
supply chain director and the finance director who then optimize the R&D portfolio
according the feedback. Table 3.8 gives an overview of the responsibilities of the
different people in the process.
Table 3.8 Role and Responsibilities Role Responsibility President • Set global strategy for Infant & Nutrition.
• Monitor high-‐level KPIs to ensure strategy is adhered to. Global VP R&D • Translate global strategy into global R&D strategic goals.
• Monitor the global R&D project portfolio to ensure strategic R&D goals are accomplished.
Global Marketing • Help translate global strategy into global R&D strategic goals. • Monitor the global R&D project portfolio to ensure strategic goals are
accomplished. Global PPM Process Owner
• Act as an owner of the overall PPM process. • Facilitate the flow of data and prepare global portfolio views.
Regional VP • Translate global R&D strategy into a regional R&D strategy. • Provide guidance on local R&D strategy to countries. • Monitor regional performance.
Country President • Translate regional R&D strategy into actionable country R&D specific strategic goals.
• Monitor the country R&D project portfolio and ensure local R&D portfolio achieves the country’s goals.
Local R&D Director • Execute on projects in the portfolio. • Maintain the correct mix of projects to meet country goals. • Provide input on the feasibility of projects in the portfolio.
Marketing Director • Provide input on the feasibility of projects in the portfolio. Supply Chain Director • Provide input on the feasibility of projects in the portfolio. Finance Director • Provide input on the feasibility of projects in the portfolio.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 47
3.4 Conclusion The Kalypso model enables the client to assess each project individually on three
different value drivers (financial, strategic fit and risk). The weight of each value
driver is determined by the stage a project is in. The financial value driver is
determined in the first two stages of the development process with a qualitative
assessment. In later stages the financial value drivers is calculated based on the NSV.
Strategic fit and risk are determined via qualitative criteria. Based on these weighted
value drivers a general score is calculated that enables users to rank projects based on
their individual score. The Kalypso model produces various visual depictions of the
NPD portfolio after each project is ranked. Various people throughout the
organization are involved in the NPD portfolio process who each has his/her
individually responsibilities. In the next chapter the Kalypso model will be evaluated
with the criteria formulated in chapter 4 and will it become clear how well the
Kalypso model is able to fulfill its purpose.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 48
Chapter 4: Evaluation of the Original Kalypso Model In the previous chapter the Kalypso model is described. In this chapter the Kalypso
model is evaluated. The evaluation is based on the criteria discussed in chapter 2. The
Kalypso model will first be evaluated based on the general criteria. Thereafter the
Kalypso model will be evaluated based on process, strategic, financial, balance finally
risk and usability criteria. The Kalypso model receives a score of 1 (poor) to 7
(excellent). Having seven points tends to be a good balance between having enough
points of discrimination without having to maintain too many response options. This
chapter ends with a conclusion. The evaluation is conducted by the author in
cooperation with Laurens Broekhof, a Kalypso employee who was involved in the
development of original model. The author and Laurens Broekhof separately
evaluated the original Kalypso model. Afterwards we compared the assessments,
when the scores differed we discussed our consideration and reached consensus on
the score.
4.1.1 General Criteria Table 4.1 Kalypso model general criteria evaluation
1.1. The NPD portfolio management tool is usable and understandable for its users (Kester et al., 2009, Archer and Chasemzadeh, 1999,2007, Loch and Kavadias 2002) 5
1.2. The NPD portfolio management tool uses multiple types of methods and criteria (see paragraph 2.4.5) 7
1.3. The NPD portfolio management tool is able to rank projects based on pre-‐determined criteria (Archer and Chasemzadeh, 1999,2007) 5
1.4. The selection methodology is determined before the NPD portfolio selection starts (Archer and Chasemzadeh, 1999,2007) 7
1.5. The NPD portfolio management tool uses a common measure which enable an equitable comparison (Archer and Chasemzadeh, 1999,2007) 7
1.6. The NPD portfolio management tool does not solely depend on financial tools (Cooper and Kleinschmidt, 1999) 3
1.7. The NPD portfolio management tool incorporates qualitative criteria (Cooper et al. 1998, 1999) 6
Average 5.7 Poor 1 2 3 4 5 6 7 Excellent
The first criterion mentioned is that the NPD portfolio tool is usable and
understandable for its users (Kester et al., 2009, Archer and Chasemzadeh,
1999,2007, Loch and Kavadias 2002). The Kalypso model is understandable (1.1), the
input consist out of three variables, the financial value driver, the strategic fit value
driver and the risk value driver. These three variables can be rooted back to the goals
formulated by Cooper et al. (1998, 1999). There are no complex calculations involved
in calculation of the final score, hence the Kalypso model is not a black box. However
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 49
the transition from the Gross Profit minus Marketing Expenses to the financial value
driver (see paragraph 3.2.1.1) seems very pragmatic, how is the constant determined?
The same goes for the weighting of the different value drivers at the various
development stages, where are these based on?
There is not a single method or metric for effective NPD portfolio management. An
NPD portfolio management tool should use more than one method to compensate the
strength and weak points of each individual method or metric (1.2). The Kalypso
model uses multiple metrics, it uses a financial, risk and strategic fit metrics. The
metrics used support each other e.g. high risk can be compensated by a high financial
reward. A radical innovative project can still be valuable due to high financial
estimates despite high risks. Although multiple methods are used the emphasis,
especially in later stages, lays on the financial value driver. The weight of each value
driver differs per stage, the distribution of each value driver seems to be
pragmatically/ad hoc determined and not based on scientific research. The weighting
should at least be determined via consensus between various users of the model.
Archer and Chasemzadeh (1999, 2007) state that the NPD portfolio tool should be
able to rank projects based on pre-determined criteria. This is not the entirely case in
the Kalypso model. It is known on which criteria a project is assessed (financial,
strategic fit and risk), however, there are no minimum values set, e.g. a project must
have a minimal NPV of …, there are no thresholds formulated (1.3). The
methodology is known in advance (1.4).
Each project is assessed on the same three value drivers independent of the
development stage the project is in. These three value drivers are used to calculate a
common measure and thus enable equitable comparison (1.5). The Kalypso model
doesn’t rely solely on financial tools, it also uses risk and strategic fit. However there
is a very large emphasis on the financial tool in later stages. This could lead to an over
exposure to the biases of financial tools (see paragraph 2.3.1) (1.6).
In the early stages there is too little information available to make reliable quantitative
financial predictions. For this reason a qualitative assessment of the probability of
financial success is made. The qualitative criteria used to assess the financial value
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 50
driver (see table 3.3) correspond to marketing proficiency. Marketing proficiency is a
driver for new product success (Montoya-Weiss and Calantone, 1994, Henard and
Szymanski, 2001, Pattikawa et al., 2006). Cooper (1979), describes marketing
proficiency including knowledge about market size, understanding of the competition
and understanding user needs. Hence, the qualitative criteria used are appropriate for
assessing the financial value driver in the early stages in the development process.
The financial value driver in the early stages is a very rough approximation of
possible future performance, therefore the weight of the financial value driver in the
Kalypso model is low in the early stages. The qualitative approach enables the
Kalypso model to assess a project on a financial value driver despite the early stage a
project is in. The other value drivers (risk and strategic fit) are assessed entirely based
on qualitative criteria with the use of scoring models (1.7)
4.1.2 Process criteria Table 4.2 Kalypso model process criteria evaluation
2.1. The NPD portfolio method is divided in phases (Archer and Chasemzadeh, 1999,2007) 6 2.2. A project is re-‐evaluated each time it transfers to another stage (Cooper, 1990, 2008, Archer
and Chasemzadeh, 1999,2007) 7
2.3. Individual project development and evaluation is based on the Stage Gate® process (Cooper, 1990, 2008, Archer and Chasemzadeh, 1999,2007) 4
2.4. The NPD portfolio is reviewed periodically (Cooper et al., 1998, 1999, Archer and Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007) 7
2.5. Each project is assessed individually on its (financial) value (Archer and Ghasemzadeh, 1999). 7
2.6. The NPD portfolio management tool takes into account the interdependencies between projects (Archer and Ghasemzadeh, 1999) 1
2.7. The NPD portfolio management tool takes into account the time dependent nature of project resource selection (Archer and Ghasemzadeh, 1999) 1
Average 4.7 Poor 1 2 3 4 5 6 7 Excellent
The Kalypso model is divided in stages and has some resemblance to the NPD
portfolio process presented by Archer and Chasemzadeh, (1999, 2007) (2.1). The first
step of the Kalypso model is entering project data and classify the project (range
maintenance, cost out, innovation) this matches to some degree the prescreening stage
of process presented by Archer and Chasemzadeh, (1999, 2007). In this stage projects
are also classified, in the process of Archer and Chasemzadeh (1999, 2007) this
classification is partly based on the strategy of an organization. The next stage of the
NPD portfolio process is the individual project analysis. In this stage projects are
individually assessed bases on a common set of criteria (2.5). The output of this stage
is a common set of parameter estimates for each project. Projects in the Kalypso
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 51
model are individually assessed on the financial value driver, strategic value driver
and the risk value driver. A project receives a score for each variable. Based on these
scores an overall fit value is calculated. Because the development stage of the project
has an influence on the weight of the different value drivers each time the project
enters a new stage the individual assessment is done over (2.2). The Kalypso model
differs with the NPD portfolio process in these later stages. The NPD portfolio
process has an additional screening stage. In the screening stage, projects that
obviously doesn’t belong in the portfolio are eliminated, hence it is a threshold. The
final portfolio selection in the NPD process is based on the set of common parameters
found at the individual project analysis. The Kalypso model produces after the
individual project assessment and calculation of a project score a variety of visual
depictions of the portfolio. This matches the last phase of the NPD portfolio process,
portfolio adjustment. What can be concluded is that the Kalypso model and the NPD
portfolio process of Archer and Chasemzadeh, (1999, 2007), share a lot of resembles.
The Kalypso model thus enables decision makers to move logically towards an
integrated consideration of projects most likely to be selected. The Kalypso model
also uses the Stage Gate® process (2.3), the stage a project is in determines the weight
of the value categories for calculating the final score. However the execution of the
Stage Gate® process is flawed, often project skip the first two stages and when they
are screened the screening is weak. The portfolio is reviewed periodically, once a
quarter (2.4). The Kalypso model doesn’t take into account interdependencies
between projects and/or the time dependent nature of projects (2.6 and 2.7).
4.1.3 Strategic criteria Table 4.3 Kalypso model strategic criteria evaluation
3.1. The NPD portfolio management tool assesses a projects on strategic fit (Cooper and Kleinschmindt, 1995, Cooper et al., 1998) 7
3.2. The strategy is determined before the NPD portfolio selection process starts (Archer and Chasemzadeh, 1999,2007) 2
3.3. Does the clear strategy consists of the who, what, how, why and threats (Markides, 1999, Loch and Tapper, 2002) or consist out of four perspectives (financial, customer perspective, internal business process and learning and growth perspective) (Kaplan and Norton, 1996)
1
3.4. The NPD portfolio management tool aligns the portfolio to the strategy (Cooper et al. 1998, 1999) 3
3.5. The NPD portfolio management tool is top-‐down and bottom-‐up (Cooper et al., 1998) 3 3.6. The strategy makes clear where and how many resources should be spent (Cooper et al.,
1998) 1
3.7. Projects are classified in advance of the portfolio selection (e.g. strategic Buckets) (Cooper et al., 1998, 1999, Archer and Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007) 4
3.8. Projects are assigned to buckets (Barczak et al., 2009) 1
Average 2.8 Poor 1 2 3 4 5 6 7 Excellent
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 52
The Kalypso model pursuits the second goal of Cooper et al. (1998, 1999), the
portfolio must be aligned with the strategy. This is achieved by assessing each project
on strategic fit (3.1). A big drawback of the Kalypso model is the way in which
strategic fit is assessed. A clear strategy should be determined before the NPD
portfolio selection process starts (3.2). This is not the case in the Kalypso model, the
little strategic criteria available are too vague. Strategic fit is measured on six criteria
(see table 4.4), when a project matches a criterion the corresponding level of strategic
fit is determined (high, medium or low).
Table 4.4 Criteria for determining level of strategic fit High The idea or concept keeps the client on track with areas of performance (AOP)/strategic plan
objectives Medium • The idea or concept keeps emerging markets growing at current rate.
• The idea of concept strengthens competitive position in developed markets • The idea or concept produces a competitive "game changing" advantage • Leverages global scale
Low The project does not fit with the client’s strategic
The distribution of fit in the Kalypso model is questionable, what if a project matches
all criteria for a medium strategic fit? In this case it seems more sensible to rank a
project with a high degree of strategic fit. The transition from medium to low appears
big, a project that has a medium strategic fit receives 10 point, a project with a low
strategic fit receives 1 point. This difference is too large and leaves no room for
nuance. What if there is a project where 10 or 1 doesn’t represent the strategic fit but
should receive a value in between?. Hence the strategic fit assessment is hampered
due to the lack of a clearly formulated strategy and corresponding goals and
objectives (3.3 and 3.4). Due to the absence of a clear strategy top management isn’t
able implement (or cascade down) the strategy by setting strategic objectives and
criteria. Because of this the effectiveness of the input from the bottom is also limited
(3.5). E.g., when you assess a project on vague criteria the value of this assessment is
limited. Where money is spend mirrors the business’s strategy. It must be clear from
the strategy where and how many resources are allocated, this is currently not the case
(3.6). Projects are classified in advance however this classification has no influence
on specific spending (3.7). All projects in the Kalypso model compete for the same
resources, there is no differentiation between projects. Hence, the Kalypso model
does not use buckets (3.8). This can lead to a skewed composition of the NPD
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 53
portfolio. Cost out projects or range maintenance have fewer risk and have a more
predicable future performance than innovative projects. Consequently cost out and
range maintenance projects are (on short term) more interesting to develop. When all
projects are competing for the same resources there is a good chance that innovative
projects are chosen last.
4.1.4 Financial Criteria Table 4.5 Kalypso model financial criteria evaluation
4.1. The NPD portfolio management tool strives to maximize the value of the portfolio (Cooper et al., 1998, 1999) 5
4.2. The financial metric incorporates the cost of development and launch (Cooper et al., 1998) 1 4.3. The financial metric has no bias towards long term radical projects (Chao and Kavadias,
2007) 4
4.4. The financial metric captures the flexibility of the development process (Newton et al., 2004) 1
4.5. The financial metric is constrained by a critical resource (Cooper et al., 1998) 1
4.6. The financial metric ignores sunk cost (Cooper et al., 1998) 7 4.7. The financial metric incorporates the probability of success or failure (Cooper et al., 1998,
Pennings and Lint, 1997) 1
4.8. The financial metric uses the Minimum Acceptable Rate of Return (MARR) and Time Value of Money (TVM) (Remer and Nieto, 1995) 1
4.9. The financial metric should be user friendly and not too complex or appear as a black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009). 7
Average 3.1 Poor 1 2 3 4 5 6 7 Excellent
Cooper et al. (1998, 1999) propose financial tools and scoring models to maximize
the value of the NPD portfolio. The Kalypso model uses both a financial metric
(Gross Profit Margin minus Total Marketing expenses) and scoring models (for
strategic fit and risk) and thus strives to maximize the value of the NPD portfolio
(4.1). However this attempt is hampered by the financial metric used: Gross Profit
Margin minus Total Marketing Expenses (GP) generated by a product in the 2nd fiscal
year. This financial metrics has been the dominant decision metric at the client and is
for this reason used in the Kalypso model. A major drawback of the GP is that it
doesn’t take into account the investment necessary for development (4.2). Hence, it is
impossible to determine which project developers most “bang for buck”. All GP
predictions have a time horizon of 2 years. A highly innovative project needs more
time to perform well than an incremental innovative project. Is it fair to demand the
same performance development of all projects? The GP doesn’t has a total bias
towards radical projects. Radical project are often more costly to develop, the current
financial metric as mentioned before doesn’t consider the development cost which
limits the bias towards radical projects. The GP is not flexible (4.4), it does ignore
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 54
sunk cost (4.6), nevertheless the GP also ignores probabilities of success or failure
(4.7) plus MARR and TVM (4.8). What can be concluded is that the GP is not suited
to make NPD portfolio decisions upon. The current financial metric doesn’t meet
most of the criteria set and thus isn’t able to maximize the value of the NPD portfolio.
On the other hand is the GP not a complex metric and is easy to comprehend (4.9).
4.1.5 Balance criteria Table 4.6 Kalypso model balance criteria evaluation
5.1. The NPD portfolio tool is able to visually depict the portfolio (Cooper et al., 1998) 7
5.2. Management has defined what balance and imbalance is (Cooper et al., 1998) 1 5.3. The NPD portfolio shows only the, for management, most important diagrams (Cooper et
al., 1998) 3
5.4. The Bubble diagram’s X and Y-‐axis are based on a organization’s strategic goals and objectives Cooper et al., 1998) 4
Average 3.8 Poor 1 2 3 4 5 6 7 Excellent
The Kalypso model is able to visually depict the portfolio and produce various charts
(5.1). These chart provide insight into the portfolio e.g., types of projects in the
portfolio, number of project by strategic fit etc. However it is not clear what balance
or imbalance is (5.2)? Due to the lack of a definition of balance it is difficult to
determine what the most important diagrams and charts are. The Kalypso model
produces a vast amount (23) of charts and diagrams, this increases the chance of
information overload (5.3). Because of the lack of a clear strategy and corresponding
goals, the Kalypso model can’t depict the portfolio based on these goals and
objectives (however if there were a clear strategy available the Kalypso model should
be able to do so).
4.1.6 Risk Criteria Table 4.7 Kalypso model risk criteria assessment
6.1. The NPD portfolio management tool assesses projects on risk (Halman and Keizer, 1994. Keizer et al., 2005 5
6.2. Risk is assessed on three dimensions; likelihood, impact, and the ability to do something about it (Halman and Keizer, 1994) 1
6.3. Risk is assessed on a sufficient number of criteria (Keizer et al., 2005) 3
Average 3.0 Poor 1 2 3 4 5 6 7 Excellent
Risk is assessed in the Kalypso model (6.1). Risk is assessed on three subcategories;
technical, commercial and regulatory risk. The category with the highest risk
determines the overall risk of the project. However the risk assessment of the Kalypso
model is limited compared to the risk reference framework proposed by Keizer et al.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 55
(2005) where risk is assessed on 12 subcategories. The weight of the risk value driver
differs per stage the project is in. If a project is in stage 1 or 2 the risk score is
weighted at 10% and for later stages 30%, but according to Sommer et al, 2007 risk
decreases during the development process, hence the distribution of weight doesn’t
make much sense. Risk should be assessed on three dimensions; (1) likelihood, (2)
impact and (3) the ability to do something about it. Currently risk is only assessed on
occurrence and impact trough qualitative criteria linked to a 5-point risk scale. The
Kalypso model doesn’t use the third criteria proposed by Halman and Keizer (1994),
control (6.2). High risk isn’t necessary bad; it depends on the reward. In the Kalypso
model this link isn’t very explicit. Furthermore the number of criteria used is very
limited compared to the reference framework developed by Keizer et al. (2005) (6.3).
4.1.7 Usability Criteria Table 4.8 Kalypso model usability criteria evaluation
7.1. Governance roles and responsibilities are defined (Cooper, 2008) 6
7.2. The NPD portfolio is periodically assessed (Cooper, 2008) 7 7.3. The NPD portfolio management method is user friendly and not too complex or appears as a
black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009) 4
7.4. Deliverables are clearly defined for each portfolio assessment (Cooper, 2008) 6 7.5. Deliverables are focused on the essential information needed for NPD portfolio
management decisions (Cooper, 2008) 3
7.6. Templates are not overly detailed and be limited to the essential information needed for NPD portfolio management decisions (Archer and Ghazemzadeh, 1998, Cooper, 2008) 5
Average 5.2 Poor 1 2 3 4 5 6 7 Excellent
In the Kalypso model it is clear who is responsible for what (7.1). There are a many
people at different levels of the organization involved in the NPD portfolio
management process. This makes the execution of the Kalypso model difficult
especially for high-level employees who often have a full schedule. This can be
overcome by conducting the NPD portfolio management process periodically. In the
Kalypso model, Stage Gate® (or product management review) reviews are done
monthly and NPD portfolio reviews are on a quarterly basis (7.2). The Kalypso model
is a user-friendly model, it is not complex and does not require huge amounts of data,
and consequently does not appear to be a black box. However, the weighting of the
various value driver and the constant in the calculation of the financial value driver
(from stage 3 onwards, see paragraph 3.2.2) seems to be very pragmatic and ad hoc
(7.3). Because the project assessment methods and criteria are known in advance it is
also clear what (data) has to be delivered for each NPD portfolio assessment (7.4).
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 56
However it is not clear what the essential information is due to the lack of a clear
strategy (7.5). This could lead to templates which are overly detailed although this
seems not the case with Kalypso model. As mentioned before, the data requirements
of the Kalypso model are limited but lack purpose due to the absence of a clear
strategy (7.6).
4.2 Conclusion The Kalypso model has been assessed on various criteria in seven categories. Table
4.9 shows the average score the Kalypso model received in each category.
Table 4.9 Average scores Kalypso model per category
1. General Criteria 5.7
2. Usability Criteria 5.2
3. Process Criteria 4.7
4. Balance Criteria 3.8
6. Financial Criteria 3.1
5. Risk Criteria 3.0
7. Strategic Criteria 2.8
Overall score 4.0 Poor 1 2 3 4 5 6 7 Excellent
The Kalypso model received an overall score of 4.0, this score is the average score of
the category scores. The Kalypso model receives a higher score than the alternative
models described in paragraph 2.4. What becomes clear is that there is an opportunity
for improvement. Table 4.9 provides insight in the strong and weak points of the
Kalypso model (appendix 5 gives and overview of Kalypso model performance per
criteria ranked by score). The Kalypso model scores well on general, usability and
process criteria. E.g. the Kalypso model uses multiple types of methods for valuating
projects (1.2). However, the Kalypso model fails in the details. The assessment of the
various value drivers is flawed. Due to the lack of clear strategy accompanied by
well-defined goals and objectives it is very difficult to determine the strategic fit of
individual projects (3.3). Because of the absence of strategic buckets it is very
problematic to top-down implement a strategy while totally different projects are
competing for the same resources (3.4, 3.6). The quantitative financial metric is not
suited for NPD portfolio management decisions. In addition, many projects have
failed because less frequently occurring risks were not taken seriously (Keizer et al.,
2005). The risk assessment is not sufficient, projects are assessed on too few risks
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 57
(6.3). The risks that are reviewed are not assessed on the three dimensions of risk
(6.2). Due to the absence of a clearly formulated strategy the graphs and charts lack
purpose, 23 charts are just too many (5.3). In addition the lack of a clear definition of
what balance is also hampers the value of the charts (5.2).
The basis of the Kalypso model is good, the model uses relevant variables for
determining the value of a project. The model also uses multiple methods for
assessing the value of projects. However the model lacks detail and rigor in the
assessment these individual value drivers. Because of this, the Kalypso model
produces a sub-optimal portfolio.
In this chapter the Kalypso model was evaluated, many strong and weak points were
identified. There is a great opportunity for improvement and improvement will be the
main focus of the next chapter. In chapter 5 a modified Kalypso model will be
presented, this model continues to build on the strong points and tries to overcome the
weak points of the original Kalypso model.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 58
Chapter 5: Improving the Original Kalypso model In chapter 4 various strong and weak points of the original Kalypso model were
identified. These points were found by assessing the Kalypso model on the criteria
listed in chapter 2 (see table 2.9). In this chapter, an improved Kalypso model will be
presented; the Kalypso model v2. Table 4.9 provides insight in what the weakest
aspects of the original Kalypso model are. The modifications of the Kalypso model
are prioritized on the results shown in table 4.9. Scientific literature and experts were
consulted for the development of the Kalypso model v2. During expert discussions
with, Mick Broekhof, Laurens Broekhof from Kalypso and Linda Kester of the TU
Delft, valuable input was attainted. We will first focus on the strategic criteria,
thereafter risk, financial, balance, process, usability and last general criteria. Various
references to the Stage Gate® process are made in this chapter. Figure 5.1 shows the
Stage Gate® process used for the Kalypso model v2.
Figure 5.1 Stage Gate® process by Cooper
Source: Cooper (2008), page 215
5.1.1 Strategic criteria The Kalypso model received the lowest score on the strategic criteria assessment. The
Kalypso model failed to capture the strategy and consequently failed to assess
projects on their fit with the strategy (3.3). This is partly due to the fact that the client
doesn’t have a clearly defined strategy with corresponding goals and objectives.
Another big drawback of the strategic assessment was the absence of a statement of
where and how many resources should be spent (3.6), no strategic buckets were
defined (3.8).
We will start with inability of the original Kalypso model to effectively assess the fit
of a project with an organization’s strategy. You can only assess a project on strategic
fit if there is a clear strategy present. The strategy must originate in the top of the
organization (Kaplan and Norton, 1996, Markides, 1999, Loch and Tapper, 2002).
Stage 0 Discovery
Stage 1Scoping
Stage 2Build business
caseStage 3
DevelopmentStage 4
Testing & validation
Stage 5Launch
Gate1
Idea Screen
Gate2
Second Screen
Gate3
Go To Development
Gate4
Go To Testing
Gate5
Go To Launch
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 59
The CEO or president of an organization is responsible for the formulation of the
organization’s strategy. This responsibility is already clearly defined in the original
Kalypso model (see paragraph 3.3). However, where a strategy consists of must be
defined more clearly in the modified model. The Kalypso model v2 must make clear
what the deliverables for a strategy are. It is important that the strategy is clear and
explicit (Markides, 1999). Objectives and goals must be derived from the strategy in
order to assess projects on strategic fit as shown in figure 5.2.
Figure 5.2 Strategy development
An organization has to define when a goal is attained by setting various criteria. Every
organization has its own unique strategy, consequently each organization has to
define its own goals, objectives and corresponding conditions and criteria.
Subsequently the Kalypso model v2 cannot present a list of reference criteria and
objectives. As described in chapter 2 a good strategy must be able to answer the
following questions:
1. Who should the organization target as customers?
2. What products or services should the organization offer the targeted
customers?
3. How, with what core competences and processes?
4. Why, what is the value proposition to the customers and its competitive
advantage?
5. What are the major threats in the environment?
Based on these questions organizations should be able to formulate numerous
strategic criteria. Andrews (1995) states that one critical requirement for the
successful implementation of a strategy is to ensure that decisions made by managers
Strategy
Goal 1
Objective 1.1 Objective 1.2
Criteria 1.1.1 Criteria 1.1.2 Criteria 1.2.1 Criteria 1.2.2
Goal 2
Objective 2.1 Objective 2.2
Criteria 2.1.1 Criteria 2.1.2 Criteria 2.2.1 Criteria 2.2.2
Goal 3
Objective 3.1 Objective 3.2
Criteria 3.1.1 Criteria 3.1.2 Criteria 3.2.1 Criteria 3.2.2
Broader
Narrower
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 60
and senior managers are consistent with the organization’s goals and objectives. The
strategy must identify tradeoffs and priorities among conflicting goals (Loch and
Tapper, 2002). An effective way to prioritize goals is by assigning resources to each
goal. This brings us to the next improvement of the original Kalypso model.
The second (strategic) limitation of the original Kalypso model is the inability to
focus resources. The original Kalypso model doesn’t use strategic buckets. As
described in chapter 2, a strategic bucket is a collection of NPD projects that are
aligned with a particular innovation strategy or strategic objective (Chao and
Kavadias, 2007). These innovation strategies and objectives are the result of the
strategy development described in the previous part. The goal of the strategic buckets
method is to translate the organization’s strategy into clearly defined arenas (buckets)
and to dedicate (a finite amount of) resources to each (Cooper et al, 1998).
Management determines the amount of resources assigned to each bucket. Cooper et
al. (1998) identified various considerations for the development of strategic buckets,
which are shown in table 5.1.
Table 5.1 Various consideration for the development of strategic buckets
Strategic goals Management spilt resources across the specific strategic goals Product lines Resources are spilt across product lines. Market segments The business may operate in several different market segments, with varying
degrees of attractiveness and potential for the future. Hence management splits resources across market segments
Technology types Resources could be split across technologies, technology types or platforms Project types What percent of resources should go to new product development? To platform
development? To process improvement? Etc. Geography Resources could be split based on geography. E.g. what proportion goes to Europe,
North America, Asia, etc.? Source: Cooper et al. (1998), page 87-‐89
Based on, for an organization relevant considerations, various strategic buckets
should be defined. As with the strategy, the buckets are cascaded down. If necessary
lower level management could further specify or split buckets (this depends on the
autonomy of lower level management). Figure 5.3 illustrates the notion of cascading
down the strategic buckets.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 61
Figure 5.3 Example of cascading down strategic Buckets
A (global) strategy is formulated by top management. In this example three goals are
formulated. For each goal various criteria and preconditions are formulated, which
give insight in how and when the goal is obtained. The total available resources are
divided into three buckets (one for every goal). Each goal mirrors a specific part of
the (global) strategy. Regions A and B are involved in attaining goal III. Hence the
bucket for goal III must be divided into buckets 3.1 and 3.2 (one for each region).
This division is done at the highest organizational levels. Bucket 3.2 is cascaded down
to the management of region B. There are three technologies necessary for achieving
goal III. The selection of these technologies is based on the initial criteria and
preconditions set by top management. Consequently, the management of region B
must divide the resources of bucket 3.2 into three smaller buckets; 3.2.1, 3.2.2, 3.2.3.
They must define (additional) conditions for each bucket. Each technology bucket can
be filled with individual projects based on the assessment on the conditions
formulated by regional management and by local technology management. The
criteria for assigning, markets, technologies, projects, etc. become more specific with
each lower level. Exhibit 5.1 shows the process of strategic buckets
Goal I --- Bucket 1Goal II --- Bucket 2Goal III --- Bucket 3Region A --- Bucket 3.1
Region B --- Bucket 3.2
Technology X---
Bucket 3.2.1
Technology Y---
Bucket 3.2.2
Technology Z---
Bucket 3.2.3
Proj
ect 1
Proj
ect 2
Proj
ect 3
Proj
ect 4
Proj
ect 5
Proj
ect 6
Proj
ect 7
Proj
ect 8
Proj
ect 8
Dev
elop
ed G
loba
lly
Dev
elop
ed R
egio
nally
Dev
elop
ed L
ocal
ly
Dev
elop
ed L
ocal
ly
Dev
elop
ed L
ocal
ly
Company Strategy
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 62
Exhibit 5.1 Process of strategic buckets
x ∈ (Top management level,.., Project management level)
*Project or bucket depends on the management level
Project interdependencies should be assessed during the strategic fit assessment.
Verma and Sihna (2002) classified three kinds of interdependencies; resource
interdependencies, technology interdependencies, and market interdependencies. The
resource allocation for each project is inversely related to resources for other
concurrent projects; an increase in the resource level for one project would lead to a
decrease in the resource level of another project. Technology interdependencies result
from leveraging common technology across multiple projects (Verma and Sihna,
2002). Market interdependencies, according to Verma and Sihna (2002), stem from
(1) a new product’s diffusion into an already existing product market and (2) utilizing
a current product’s market knowledge (e.g. how to manage a dealer network) for
development of a new product for an entirely different market. Therefore, both
technology and market interdependencies are what Thompson (1967) refers to as
“sequential interdependencies”: outcome of a project affects the resource allocation of
other projects that follow. Especially technology market interdependencies are of
strategic importance. Which technologies to uses and which markets to target are
strategic considerations. Consequently, technology market interdependencies should
be assessed during strategic fit assessments. E.g. “does project x built on the
technology of project y?” Technology interdependencies could be basis for assigning
project to a specific bucket (see table 5.1).
A scoring method is the most effective way to assess strategic fit due to the qualitative
nature of variable (Cooper and Kleinschmidt, 1995, Cooper et al., 1998). In order to
determine the strategic fit of a project it must be assessed on various criteria. These
Define BucketsManagement level x
Define Criteria
Project/Buckets*Management level x-1
Fill Bucket!"#$%&'(
)*++$%&,-
.$+$/0$%&,-
Strategy
!"#$%&'(!"#$%&'(
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 63
criteria should be directly linked to the strategy and must address the various facets of
a strategy. Often a strategy consists of several goals, every goal is pursued with
different projects. Hence each goal has its own criteria for the strategic fit assessment
of projects. Each criterion is ranked on a seven-point scale (1 (poor) to 7 (excellent)).
The overall strategic fit score is calculated by:
!"# =S!!!"
!!
! ∗ 7 ∗ 100
Where: SCF = Strategic Fit Score ci = Strategic criteria i ∈ (1,2,..,N) N = Number of criteria used Sci = Received score (1 to 7) on strategic criteria i
Figure 5.4 depicts the strategy process in the Kalypso model v2. The strategy is
developed by answering the who, what, how, why and threats proposed by Markides
(1999). Based on the strategy clear goals are formulated accompanied by well-defined
criteria. Resources are dedicated to each goal (buckets are defined), the amount of
resources is based on the strategy. Projects are appointed to one bucket based on the
criteria formulated for each goal. Projects in each bucket are ranked (and prioritized)
based on strategic fit, risk and financial. The strategic fit assessment is based on the
criteria defined for each goal. A detailed description of the project ranking process
can be found in paragraph 5.4.
Figure 5.4 Strategy process
*Not part of the strategy process
Develop strategy
Define Goals
Define criteria for each goal
Dedicate resources to each goal (define buckets)
Appoint projects to each bucket
Based on
Projects
Rank projects*
Based on
Risk*
Financial*Based on
Based on
Based on
Who,What, How, Why and Threats?Based On
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 64
5.1.2 Risk criteria The risk assessment is the second weakest aspect of the original Kalypso model.
Especially the number of risks that are assessed and the manner or depth of this
assessment is lacking. Risk should be assessed on three dimensions (occurrence,
impact and control) instead of two (occurrence and impact) (Halman and Keizer,
1994). In the Kalypso model v2 the assessment of risk is done more rigorously.
According to Keizer et al. (2002) a risk assessment approach would:
• Evaluate each potential risk on its likelihood, its controllability and its relative
importance to project performance (the three dimensions).
• Take a cross-functional perspective by identifying and evaluating technological,
market, financial and operational risks.
• Conduct the risk assessment at the end of the feasibility phase (stage 2) and
periodically reassess the project for unforeseen risks and deviations from the risk
management plan.
• Identify and evaluate the product innovation risks individually and generate,
evaluate and select alternative solutions in subgroups and plenary sessions.
The Kalypso model v2 risk assessment will be based on the Risk Diagnosing
Methodology (RDM) developed by Keizer et al. (2002) which aims to identify
various risks. RDM consists of three stages with various steps that are shown in figure
5.5.
Figure 5.5 Outline of risk diagnosing methodology
Source: Keizer et al. (2002), page 216
Risk IdentificationStep 1: Initial briefing between project manager and risk facilitatorStep 2: Kick-Off meeting: project manager & team and risk facilitatorStep 3: Individual interviewing of participants by risk facilitator
Risk AssessmentStep 4: Development of risk questionnaire by risk facilitatorStep 5: Answering of risk questionnaire by participantsStep 6: Constricting of risk profile by risk facilitator
Risk Response Development and ControlStep 7: Preparing of risk management session by project manager & risk facilitator Step 8: Risk management session: project manager & team and risk facilitator Step 9: Drawing up & execution of risk management plan
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 65
The most relevant part of RDM for the Kalypso model v2 is the risk assessment
because it covers the actual assessment of the various risks. The risk identification
and risk response development and control stages should be done before and after the
NPD portfolio selection process (see paragraph 5.1.5). However these stages are
important for the success of RDM.
The numbers of risks used in the original Kalypso model are not sufficient, a too
limited number of risks are assessed (see paragraph 4.1.6). Appendix 3 lists the
various risks identified by Keizer et al. (2005). These risks are based on risks found in
literature (e.g. the meta-study by Montoya-Weiss and Calantone (1994) of success
factors for new product success) and case interviews. These risks are generic. A
thorough risk assessment must also include context specific risks. Most organizations
have their own shortlist of critical mishaps that management hopes future projects
will escape (Keizer et al., 2002). Therefore, the risks used for the assessment of risk in
the Kalypso model v2 should be based on both the risk reference framework and
context specific risks found at the organization. This approach should help team
members to think of less obvious issues (Keizer et al., 2002). In addition,
interdependencies should be assessed during the risk assessment. E.g., “abandoning
this project has no influence on other projects in the portfolio” can be asked.
The risks identified are assessed on three dimensions: (1) likelihood, (2) impact and
(3) the ability to influence the outcome. Each risk must be positively framed because
Kahneman and Tversky (1979) found that negative framing of risks induces more
positive perceptions opposed to positive framing. People who are confronted with
negative framing tend to think: “Its not that bad”. While confronted with positive
framing people tend to respond: “They may suppose so, but I am not so sure”.
Positive framing leads to the most critical assessment, therefore all risks in the
Kalypso model v2 should be framed positively. The typical number of risks in a risk
assessment is, according to Keizer et al. (2002), 50-60 and should take 45-60 minutes
to complete. Exhibit 5.2 shows an example of a risk assessment (for two risks).
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 66
Exhibit 5.2 Example of risk questionnaire (for two risks) What is the level of
certainty that the statement will be true? (likelihood)
Ability of the team to influence course of actions within time &resource limits (ability to influence the outcome)
Relative importance of statement for obtaining project success (impact)
Very Low
Very High
Very Low
Very High
Very High
Very Low
Risk statements: 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1. The new product will be safe to
use for people with sensitive skin.
. ✔ . . . . ✔ . . . . ✔ . . .
2. With the trade customer clear after sales arrangements have been agreed.
. . . ✔ . . ✔ . . . . . . . ✔
Source: Keizer et al. (2002), page 217
All project team members should preferably conduct the risk assessment. Each risk
receives for each dimension a score in range of 1 to 5. After each team member has
conducted his or her assessment a risk profile is made. This risk profile presents both
the degrees of risk perceived by the majority of the respondents and the distribution of
their perceptions (Keizer et al., 2002). Exhibit 5.3 shows a risk profile for two risks.
Exhibit 5.3 Example of a risk profile (for two risks) What is the level of certainty
that the statement will be true?
Ability of the team to influence course of actions within time &resource limits
Relative importance of statement for obtaining project success
Score for each dimension of risk
Risk class
Very Low
Very High Very Low
Very High
Very High
Very Low
Risk statements: 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 C A I 1. The new product will be safe to use for people with sensitive skin.
N. resp. 0 0 1 3 5 0 0 1 3 5 0 2 5 2 0
0 0 m L Cum % 0 0 11 44 100 0 0 11 44 100 0 22 78 100 100
50% � � �
2. With the trade customer clear after sales arrangements have been agreed.
N. resp. 3 2 4 0 0 0 2 5 2 0 3 5 1 0 0
* m * H Cum % 33 56 100 100 100 0 22 78 100 100 33 89 100 100 100
50% � � �
Decision rules: N resp.: Number of team members who scored in certain
column “*”: At least 50 % of scores are 1 or 2 on the five point
scale and there are no scores of 5 Cum %: Cumulative percentage of team respondents “0”: At least 50 % of scores are 4 or 5 on the five point
scale and there are no scores of 1 50%: Column in which at least 50 response is first met “m”: At least 50 % of scores are 3 on the five point
scale and there are no scores of 1 or 5 “?”: Lack of consensus: there is a wide distribution of
opinions Risk Classes used: S = Safe; L = Low risk; M = Medium Risk; H = High Risk; F = Fatal Risk
Source: Keizer et al. (2002), page 218
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 67
Every risk is classified along three dimensions into four groups (“*”, “0”, “m” and
“?”) by the decision rules shown in exhibit 5.3. Each risk is classified next, RDM uses
five risks classes; S = Safe; L = Low risk; M = Medium Risk; H = High Risk and F =
Fatal Risk. This classification is based in the combination of scores (e.g. “*,*,*”) for
each risk. There are 64 combinations, each combination has its own risk class (shown
in appendix 6). It is possible that, due to a lack of consensus, multiple risk classes
apply for a single risk. A combination of classes means that the risk team should work
out whether the disagreement can be resolved and a single classification can be
achieved. If consensus can’t be achieved the worst possible case should be assumed.
After each risk is assessed and classified an overall risk score must be attained. This is
different from the RDM where no overall score is calculated. This is because a project
can fail because of one risk, consequently, risks shouldn’t be put on a heap. However
in order to compare different projects on risk, one score must be determined in the
Kalypso model v2. Not all risks are equal, one fatal risk should have a larger impact
on the risk score than a medium risk. What is acceptable differs per organization and
should be determined by the each individual organization. The strategy should be
used as a guide for what acceptable risk levels are. E.g., for some parts of the strategy
(specific buckets) a higher risk is acceptable than in other parts of the strategy. The
risk process described in this paragraph is shown in figure 5.6.
Figure 5.6 Risk process
*Not part of the risk process
Rank projects*
Strategy*
Financial*Based on
Based on
Individual risk assessment
Generic risks(Risk reference list) Context specific risks
Relevant riskBased on
Risk score Based on
Project team members
Identified by
Conducted byRisk facilitator (NPD process
manager)
Determined by
Determined by
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 68
5.1.3 Financial criteria Paragraph 4.1.4 gave insight in the strong and weak points of the original Kalypso
model regarding its financial assessment. Paragraph 4.1.4 made clear that the current
financial metric is not suited to base NPD portfolio decisions upon. In this paragraph
a new financial metric will be presented.
In the Kalypso model v2 the Expected Commercial Value (ECV) will be used. The
ECV overcomes most of the limitations of the current quantitative financial metric.
As described in chapter 2.3.1.1, the ECV seeks to maximize the value or commercial
worth of the portfolio.
Exhibit 5.4 Determination of expected commercial value of project
!"# = !"# ∗ !!" − ! ∗ !!" − !
$ECV= Pts= Pcs= $D= $C= $NPV=
Expected Commercial Value of the project Probability of technical success (range 0.0 -‐ 1.0) Probability of commercial success (given technical success) (range 0.0 -‐ 1.0) Development cost remaining Commercialization (Launch cost) Net present value of the project’s future earnings (discounted to the present)
Source: Cooper et al. (1998) page 24
The calculation of the ECV is based on the development costs, launch costs, Net
Present Value (NPV) and the probability of technical and commercial success. The
NPV represents the future earnings of a project where all cash flows are moved to the
present. The NPV is calculated with the following formula:
!"# =!"#!(1+ !)!
!
!!!
Where: NPV = Net present value NCF = Net cash flow of year t ∈ (1,2,..,n) i = time value of money or minimum attractive rate of return (MARR)
Development $D
Launch$C
TechnicalSucces
CommercialSucces $NVP
Commercial Failure
Technical Failure
Pts
Pcs
Yes
No
Yes
No
$ECV
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 69
The Minimum Attractive Rate of Return (MARR) is the minimum rate of return on a
project a manager or organization is willing to accept before starting a project, given
its risk and the opportunity cost of forgoing other projects (Park, 2007). Hence this
rate is different for each organization and should be based on an organization’s
strategy. Criterion 4.5 (2.3.6) notes that the financial metric should be constrained by
a critical resource (e.g. development cost). Constraining the ECV by a critical
resource requirement enables users to get insight in which projects deliver most “bang
for buck”. Hence the calculations:
!"#$#%"$& !"#$% =!"#
!"#$%&'(#% !"#$%!&" =!"# ∗ !!" − ! ∗ !!" − !!"#$%&'i!" !"#o!"#$
Where: ECV = Expected Commercial Value of the project NPV = Net present value Pts = Probability of technical success (range 0.0 -‐ 1.0) Pcs = Probability of commercial success (given technical success) (range 0.0 -‐ 1.0) D = Development cost remaining C = Commercialization (Launch cost)
The variables Pts (probability of technical success) and Pcs (probability of commercial
success) should be determined with qualitative and established criteria (Cooper et al.,
1998). Keizer et al. (2005) developed a risk reference framework (shown in appendix
3). Some of the risk identified by Keizer et al. (2005) could also be used for the
assessment of these two variables. Hence, if there is a high risk there will be a low
probability of success and visa versa. Not all risks are relevant for each stage of the
development process. E.g. it doesn’t make much sense to assess a project in the first
stage of the development process on product format meets functional requirement
because there is no reliable information about this specific risk. Consequently each
risk is best suited from a specific development stage onward. Each organization has to
develop its own list of relevant technical and commercial risks for each stage of the
development process. The risks identified (especially, product technology risks and
manufacturing technology risks categories for Pts, and consumer acceptance and
marketing risks and commercial viability risks categories for Pcs) in the risk reference
framework (appendix 3) are a good starting point for developing a list of relevant
risks for each variable. Risk should be assessed only once for each development stage.
This means that the risks used to determine Pts and Pcs are assessed during the risk
assessment (described in 5.1.2). The transformation of project risk scores (one for
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 70
technical and one for commercial success) into probabilities should be based on a
predetermined distribution (e.g., table 5.2). Each organization should determine its
own distribution.
Table 5.2 Example of distribution of risk Risk class Probability of success
S 90% L 70% M 50% H 30% F 10%
One strong aspect of the original Kalypso model is that it endorsed the notion that
quantitative financial metrics are not reliable in the early stages of the development
process. Consequently, the original Kalypso model assessed the financial metric in the
early stages with a qualitative assessment. The Kalypso model v2 will continue with
this approach. In stages 0 and 1 is the financial score solely based on qualitative
criteria. These criteria are similar to the criteria used to determine the probabilities of
technical and commercial success. Each organization has to compose its own list of
risks that are relevant for financial success. Again these risks should be assessed in
the same manner as described in paragraph 5.1.2. However at gates 1 or 2 the
classification is not transformed into a probability but into a financial score.
There are two ways for obtaining a financial score, at gates 1 and 2 a qualitative one
and at the later gates a (partly) quantitative one. Each project receives a score of 1 up
to a 100. If a project is at gates 1 or 2 the score depends on the risk profile obtained
via the assessment of relevant risk for financial success. Again, each organization
should determine its own distribution of scores depending on this assessment. This
distribution could differ per bucket. If a project is at gates 3, 4, and 5, the financial
score is determined via a quantitative metric. In the Kalypso model v2 financial score
computation differs from the formula suggest by Cooper et al. (1998) shown on the
previous page.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 71
!"#$#%"$& !"#$% = 1 −!"#$%&'(#% !"#$%!&"
!"#∗ 100
= 1 −
Constrained resource!"# ∗ !!" − ! ∗ !!" − !
∗ 100
The formula is rewritten because this enables a better comparison between the
financial assessments of project at gates 1or 2 with later gates. Figure 5.7 depicts the
financial process as described in this paragraph.
Figure 5.7 Financial process
*Not part of the financial process
Rank projects*
Strategy*
Risk*Based on
Individual financial assessment
Technical risks
Financial score Based on
Gates 1 or 2 Gates 3,4 or 5
Commercial risks
Probability of technical success
Probability of commercial success
NPV
ECV Development cost
Launch cost
Technical risks
Commercial risks
Additional risks
Based on
Based on
Distribution of score per risks
Contrained resource
Individual risk assessment*
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 72
5.1.4 Balance criteria The original Kalypso model is able to depict numerous graphical representations of
the NPD portfolio based on various criteria (criteria 5.1). This leaves little room for
improvement. What does need improvement is the definition of what balance is and
what the most important metrics for an organization are. In the Kalypso model v2 the
number of visual depictions will be limited, consequently, an organization has to
choose. Cooper et al. (1998) found that most organizations where able to reduce the
number of (relevant) diagrams to three. However, this was refuted during expert
discussions. It was stated that limiting the number of depictions was not realistic. The
information needs are different for each level of the organization. A CEO has
different information needs for instance than a regional manager. Consequently, the
number of depictions that the Kalypso model v2 produces will depend on the
organizational level of its end users. Every organizational level has to determine the
number of relevant depictions. The risk that too many depictions are chosen is, from
experience, low. Every organization has to define what balance is. The strategy of an
organization is an important aid for determining what balance is and which diagrams
should be depicted.
Figure 5.7 Balance process
*Not part of the balance process
Definition of Balance
Rank projects*
Risk*
Financial*
Based on Strategy*
Based on
Based on
X amount of Graphical depictions
Based on
Organizational level of end users
amount depends on
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 73
5.1.5 Process criteria The Kalypso scored well on many of the process criteria (see paragraph 4.1.2).
Consequently, the Kalypso model v2 will have a lot of similarities with current
model. The process of the Kalypso model v2 will be based on the NPD portfolio
process developed by Archer and Chasemzadeh (1999, 2007). The process of the
Kalypso model v2 is shown in figure 5.8 on the next page. In the new model various
aspects have to be determined before the process starts. The most important pre-
process activity is the determination of a clear strategy and corresponding goals. In
addition, resources must be dedicated to each goal by creating buckets. Another pre-
process activity is defining what balance is using the strategy as starting point.
In the next stage, prescreening, new NPD projects enter the process. These projects
are in the first stage of the development process. In this stage, each project is assigned
to a specific bucket. This assignment is based on criteria formulated for each goal
(bucket) (see paragraph 5.1.1). For each project a list of potential risk is prepared, this
list is based on the risks identified in the risk reference framework and context
specific risks identified by project members as described in paragraph 5.1.2.
In the next stage projects are assessed individually. Each project is assessed on risk,
strategic fit and financial criteria (for detailed descriptions see PARAGRAH 5.1.1,
5.1.2 and 5.1.3). Using the Risk Diagnosing Methodology (RDM) every project is
assessed on relevant risks defined in the previous stage (prescreening). During the
strategic assessment projects are assessed on the strategic criteria formulated during
the strategy development. This assessment is done with a scoring model where each
project is ranked on a 7-point scale (see paragraph 5.1.1). The financial assessment
depends on the development stage a project is in. If a project is in stage 0 or 1 the
individual assessment is based on qualitative criteria. These criteria are based on
relevant risks. A project is assessed on technical, commercial and (if necessary)
additional risks. The RDM is used for the assessment of these risks. If a project is in
stages 2,3 or 4 a more quantitative assessment is done. The NPV, development and
launch costs are estimated for each project.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 74
Figure 5.8 Kalypso model v2 process diagram
Pres
cree
ning
Port
folio
A
djus
tmen
tPo
rtfo
lio S
elec
tion
Scre
enin
gIn
divi
dual
Pro
ject
Ana
lysi
sPr
e Pr
oces
s A
ctiv
ity
Deve
lop
stra
tegy
Defin
e G
oals
Defin
e cr
iteria
for e
ach
goal
Dedi
cate
reso
urce
s to
eac
h go
al (d
efine
buc
kets
)
Who
,Wha
t, H
ow, W
hy a
nd
Thre
ats?
Defin
ition
of B
alan
ce
Appo
int p
roje
cts
to e
ach
buck
et
Proj
ects
Gen
eric
risks
(Risk
refe
renc
e lis
t)
Cont
ext s
pecifi
c ris
ks
Rele
vant
risk
Indi
vidua
l risk
ass
essm
ent
Indi
vidua
l stra
tegi
c as
sess
men
t
Indi
vidua
l fina
ncia
l as
sess
men
t
Risk
sco
re
(1-1
00)
Fina
ncia
l sco
re
(1-1
00)
Stra
tegi
c sc
ore
(1-1
00)
Stag
e 0
or 1
Tech
nica
l risk
s
Com
mer
cial
risks
Addi
tiona
l ris
ks
Dist
ribut
ion
of
scor
e pe
r risk
s
ECV
Cont
rain
ted
reso
urce
NPV
Deve
lopm
ent
cost
Laun
ch c
ost
Tech
nica
l risk
sCo
mm
ercia
l ris
ks
Prob
abilit
y of
te
chni
cal s
ucce
ssPr
obab
ility
of
com
mer
cial s
ucce
ss
or
Dist
ribut
ion
of
scor
e pe
r risk
s
NPD
Portf
olio
sco
re
GAT
E
Stag
e
Rank
ed P
roje
cts
Gra
phica
lde
pict
ions
Proj
ect
Dev
elop
men
tPr
ojec
tD
atab
ase
Risk
Dia
gnos
ing
Met
hodo
logy
or
Stag
e 0
or 1
Stag
e 2,
3 o
r 4
Base
d on
Base
d on
Base
d on Base
d on
Base
d on
Base
d on
Base
d on
Base
d on
GAT
EC
riter
ia
depe
nd o
n
Port
folio
C
ompl
etio
n
!"#$%"&'()*"+,-.
"'-/'0#,"
In
!""!
!"#$#%"$&!!"#$%
!
!!!"#$%&'(#%!!"#$%!&"
!"#
!!""
!
!!
!"#$%&'(#%!!"#$%!&"
!"#
!! !"!!!! !"!!
!!""
The
form
ula
is r
ewrit
ten
beca
use
this
ena
bles
a b
ette
r co
mpa
rison
bet
wee
n th
e
finan
cial
ass
essm
ents
of
proj
ect
at g
ates
1or
2 o
r la
ter
gate
s. F
igur
e X
dep
icts
the
finan
cial
pro
cess
as
desc
ribed
in th
is p
arag
raph
.
!"#$
%&'('!")*)
+"*,'-%.+&//'
!
!! !"
#$%&'($%#
)%$*+%),-'
-.,'/%&(#.+00%
Ran
k pr
ojec
ts*
Stra
tegy
*
Ris
k*Ba
sed
on
Indi
vidu
al fi
nanc
ial
asse
ssm
ent
Tech
nica
l ris
ks
Fina
ncia
l sco
reBa
sed
on
Gat
es 1
or 2
Gat
es 3
,4 o
r 5 Com
mer
cial
ris
ks
Prob
abilit
y of
te
chni
cal s
ucce
ssPr
obab
ility
of
com
mer
cial
suc
cess
NPV
ECV
Dev
elop
men
t co
st
Laun
ch c
ost
Tech
nica
l ris
ks
Com
mer
cial
ris
ks
Addi
tiona
l ris
ks
Base
d on
Base
d on
Dis
tribu
tion
of
scor
e pe
r ris
ks
Con
train
t re
sour
ce
Indi
vidu
al ri
sk a
sses
smen
t*
Org
aniza
tiona
l leve
l of
end
use
rs
Dep
end
on
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 75
The probabilities of technical and commercial success have an influence on the
(financial) value of a project. These probabilities are determined via the assessment of
technical and commercial risk (see paragraph 5.1.3).
After the individual project analysis it is time for the project screening. In this phase a
project receives a score on risk, strategic fit and financial based on the previous stage.
The risk score depends on individual risk assessment and the predetermined
distribution of score per risk (what risks are acceptable differ per organization or even
projects, depending on the strategy). A strategic fit score is calculated based on the
individual strategic assessment (see paragraph 5.1.1). The determination of the
financial score depends on the stage the project is in. If a project is in one the first two
stages the financial score is based on a predefined distribution of score per risks
(found in the previous stage). If a project is in stage 2, 3 or 4, the financial score is
calculated with the formula on page 71. Every project receives a score of 1 to 100 for
each category.
After the screening phase the projects reach a gate (part of the Stage Gate process), at
this gate projects are continued or abandoned. This selection process should be
conducted on predetermined criteria (see paragraph 2.2 and appendix 1), the criteria
differ depending on the stage a project is in. Next, each project receives and overall
NPD portfolio score. This score is based on the three individual scores (risk, strategy
and a financial score), the weight of each score depends on the development stage a
project is in. Every organization determines the specific weighting, however
paragraph 5.1.7 describes some rules for determining the various weights for each
development stage. The next step is to rank each project based on the NPD portfolio
score. Remember, since the prescreening phase projects belong to a specific bucket.
Therefore, projects only compete for resources with projects in the same bucket.
Consequently projects are ranked “within” a bucket. E.g. if there are five buckets you
will have five stacks of ranked projects.
The next phase is the portfolio adjustment, a number of graphical depictions (for each
bucket) show the “state” of the NPD portfolio. Based on the definition of balance
(defined in the preprocess phase), the NPD portfolio can be adjusted if necessary. The
next stage is development, in this stage the selected projects are developed until they
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 76
reach the next development stage (for a stage gate assessment) or for the next
periodical NPD portfolio assessment, which starts at the individual assessment.
5.1.6 Usability criteria The original Kalypso model scored well on the usability criteria. As described in the
previous paragraph, the NPD portfolio should be reviewed periodically. This period is
organization specific; it depends on the average development time, the amount of new
projects started, etc. Cooper (2008) recommends doing this once a month. The
original Kalypso model clearly defined who is responsible for what (see paragraph
3.3). In the Kalypso model v2 there are additional people, tasks and responsibilities
involved (see figure 5.9 on the next page). The president and senior management set
the global strategy and define goals. In addition, resources have to be committed to
each goal. Hence, they have to split the portfolio into smaller buckets, each created to
serve a specific goal of the strategy. Specific criteria are formulated for each bucket.
The management level below senior management (intermediate level) adjust the
strategy to their level of management, hence, the strategy is cascaded down. Based on
their strategy and the criteria set by higher management, intermediate level
management formulates buckets with corresponding criteria for low-level
management. The size and goals of these buckets are constrained by the size and
goals of higher-level buckets. E.g., a lower level bucket can’t pursuit totally different
goals or be larger in size than the bucket it consists out of (as described in paragraph
5.1.1). Intermediate level management has the responsibility to review the project
assessments made low-level management, and fill buckets with these projects based
on the criteria set by senior level management. This cycle is repeated by lower level
management. However, the criteria set for individual projects are also based on input
from project manager and members. E.g., project manager and project members are
able to identify project specific risks that management couldn’t identify (due to lack
of project specific knowledge). Project managers are responsible for delivering the
necessary information for low-level management to make their NPD portfolio
decisions. Project assessments are done by project manager and team member but the
final decision is made by low level management. All relevant departments are
involved in the criteria formulation: e.g., R&D, marketing, supply chain and financial
under the supervision of NPD process manager. The NPD process manager is
responsible for collecting of relevant risks, this is done in cooperation with the project
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 77
manager. The NPD process manager is not part of the project team and ensures that
no criteria are overlooked. The NPD process manager is the “neutral” party. Figure 5.9 Responsibilities in the Kalypso v2 model
Note: this is an universal diagram, reposibilites could differ per organizion
The individual assessment of each project is the responsibility of the project leader
and is done by the project leader and the team members under the supervision of the
NPD process manager. Projects that are not sufficiently assessed will not be admitted
to the NPD portfolio reviews, and consequently lose their possibility for additional
funding! Low-level management in correspondence with the NPD process manager
and project leader make gate decisions. Hence, buckets are defined from the top-
down and are filled bottom-up. NPD process manager monitor the definition of
criteria and check of each criteria is assessed correctly. Table 5.2 shows the
responsibilities for each level.
Bucket Y.y
Bucket Y
Projects
Develop Criteria for Bucket Y
Develop Criteria for Bucket Y.y
Develop Criteria for Projects
Develop Strategy Portfolio
Senior Management level
Intermediate Management level
Low Management Level
Based on
Based on
Project Level
Review and Fill
Review and Fill
Feedback
Feedback
Feedback
NPD Process Manager
NPD Process Manager
NPD Process Manager
Project Manager
Project Member
Monitor
Monitor
Monitor
Collect
Collect
Collect
Review and Fill Based on
Develop & Assess Provide Input
Develop Intermediate Level Strategy
Develop Low Level Strategy
CascadedDown
CascadedDown
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 78
Table 5.2 Role and Responsibilities Kalypso model v2 Role Responsibility President & Senior Management
• Set global strategy for the organization. • Define goals. • Dedicate resources to each goal by defining buckets. • Monitor high-‐level KPIs to ensure strategy is adhered to.
Intermediate level management
• Translate global strategy into intermediate level management specific actionable strategy
• Dedicate resources to each R&D goal by defining (sub) buckets. • Fill buckets with program/buckets • Monitor the R&D project portfolio to ensure strategic R&D goals are
accomplished. Low level management
• Translate intermediate level strategy into low level management specific actionable strategy
• Dedicate resources to each R&D goal by defining (sub sub) buckets. • Fill buckets with projects • Monitor the R&D project portfolio to ensure strategic R&D goals are
accomplished. NPD Process Manager
• Collect projects/programs/buckets • Monitor criteria development (risk, strategic fit and financial) • Supervise project assessment.
Project Leader • Identify risks. • Collect project data (risk assessment, strategy assessment, financial assessment). • Assess project
Project Team Members
• Identify risks. • Assess project
5.1.7 General criteria The original Kalypso model scored best on the general criteria. Consequently the
Kalypso model v2 has the most similarities with the original Kalypso model regarding
general criteria. Similar to the original Kalypso model, the Kalypso model v2
produces an overall score based on multiple variables (strategic fit, risk and financial
score). Paragraph 2.4 explained that not all variables are equally important and
reliable in all stages. As in the original Kalypso model, variables will be weighed in
the Kalypso model v2. The exact weight assigned to each variable differs per
development stage and is determined by each organization. Exhibit 5.5 presents some
rules regarding the weighting of each variable per development stage.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 79
Exhibit 5.5 Weighting of the various variables depending on development phase
Source: Cooper (2008), page 215
SFS1>RS1>FS1 SFS2>RS2>FS2 RS3>FS3>SFS3 RS4>FS4>SFS4
(RS4<RS3 and FS4 >FS3)
FS5>RS5>SFS5
100 = !"! + !" + !" SFS = Strategic Fit Score RS = Risk Score FS = Financial Score
At gate 1 (idea screen) strategic fit is the most important variable. The primary goal of
this gate is to assign projects in buckets (see appendix 1). The assignment of projects
to buckets is primary based on strategic criteria (Cooper et al. 1998, Chao and
Kavadias, 2004). There is little reliable financial information available at gate 0
(Linton et al., 2002), hence the financial variable should have the least impact. At the
second gate a similar (although more rigorous) assessment is conducted.
Consequently the weighting of the variables is the same. After gate 2 all projects have
to be assigned to a bucket. Gate 3 is the gate before a project enters full development.
According to Abetti and Stuart (1988) and Keizer et al. (2002) the risk assessment is
most important at this gate. After gate 3 the project’s costs increase significantly.
Consequently, you want to know how risky each project is before you invest in it.
Therefore risk is the most important variable at gate 3. Due to the detailed
investigation of markets conducted during stage 2 a more reliable financial
assessment is possible. The financial variable should have the second largest impact at
gate 3. Projects that have no fit with the strategy should be filtered out since the
previous two gates. Accordingly, strategic fit has the least impact at gate 3. The
distribution of weights at the gate 4 is similar to the previous stage. However the
financial metric increases in impact while the risk variable decreases in impact. After
testing most risks are identified and dealt with accordingly. Across all categories,
uncertainty tends to be reduced over the course of the project (Sommer et al., 2007,
chapter 17). Consequently there is reliable financial data available. In the last stage
the financial variable is has the greatest impact followed by risk. For the same reasons
mentioned earlier the strategic fit variable has the least impact.
Stage 0 Discovery
Stage 1Scoping
Stage 2Build business
caseStage 3
DevelopmentStage 4
Testing & validation
Stage 5Launch
Gate1
Idea Screen
Gate2
Second Screen
Gate3
Go To Development
Gate4
Go To Testing
Gate5
Go To Launch
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 80
The final NPD portfolio project score is calculated by multiplying each variable score
with its weight and add them as is shown in the following formula:
!"# !"#$%"&'" !"#$% = !"!! ∗ !! + !" ∗ !! + !"! ∗ !! Where: ai = weight of Strategic Fit Score (SFS) 1 < ai <100 bi = weight of Risk Score (RS) 1 < bi <100 ci = weight of Financial Score (FS) 1 < ci <100 i = stage project is in ∈ (0,1,..,4)
The NPD portfolio score enables users to compare projects independent of the
development stage. The project with the highest NPD portfolio score should receive
top priority. An organization should strive to maximize the sum of the NPD portfolio
scores constrained by the available resources. You can only compare projects that are
in the same bucket! This is because projects, in different buckets, are often assessed
with other criteria and the variables could be weighted differently.
It is possible that projects, critical for an organization, receive a low NPD portfolio
score. E.g., the development of a (underperforming) project for a specific client that,
if not developed, could lead to the loss of the client. If an organization values this
client the project has to be undertaken, hence the NPD portfolio score doesn’t have
influence the decisions. This is similar with urgent project in the original Kalypso
model. The NPD portfolio score is an aid, in the end, managers have to decide which
projects to pursuit and which one to abandon.
5.2 Evaluation of the Kalypso Model v2 The Kalypso model v2 is evaluated by the same criteria as the original model (chapter
4) and alternative NPD portfolio management models (paragraph 2.4). The author,
Mick Broekhof and Linda Kester conducted the assessments. Mick Broekhof is the
founding and managing partner of Kalypso Europe and has over 30 years of
professional experience in business management and consulting. He has extensive
expertise within the process industries. Linda Kester is a PhD student in product
portfolio management (since November 2006) at the department of Industrial Design
Engineering, Delft University of Technology (the Netherlands). The assessments were
conducted independent of each other. The final score of each criterion is average
score of each assessment. In the next part the assessment is discussed in more detail.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 81
5.2.1 General Criteria Table 5.3 KE model general criteria evaluation
M. B
roekho
f
M. D
e Mulde
r
L. Kester
Average
1.1. The NPD portfolio management tool should be usable and understandable for its users (Kester et al., 2009, Archer and Ghsemzadeh, 1999,2007, Loch and Kavadias 2002)
6 5 2 4,3
1.2. The NPD portfolio management tool should use multiple types of methods and criteria (see paragraph 2.4.5)
7 7 7 7,0
1.3. The NPD portfolio management tool should be able to rank project based on pre-‐determined criteria (Archer and Ghsemzadeh, 1999,2007)
7 7 6 6,7
1.4. The selection methodology should be determined before the NPD portfolio selection starts (Archer and Ghsemzadeh, 1999,2007)
7 7 6 6,7
1.5. The NPD portfolio management tool should use a common measure which enable an equitable comparison (Archer and Ghsemzadeh, 1999,2007)
7 7 6 6,7
1.6. The NPD portfolio management tool should not solely depend on financial tools (Cooper and Kleinschmidt, 1999)
7 7 6 6,7
1.7. The NPD portfolio management tool should incorporate qualitative criteria (Cooper et al. 1998, 1999)
7 7 6 6,7
Average 6,9 6,7 5,6 6,4 Poor 1 2 3 4 5 6 7 Excellent
Linda Kester is of the opinion that the Kalypso model is very complex, this hampers
the understandability and limits the usability of the model. She based this opinion on
the process diagram presented in paragraph 5.1.5. Mick Broekhof and the author
disagree, they perceive the Kalypso model v2 to be understandable, each project is
assessed on three value drivers; financial, strategic fit and risk. Not all value drivers
are equally relevant in each development stage. Consequently, weights are assessed to
each value driver depending on the development stage a project is in. The size of each
weight is determined by predefined rules. The project with the highest score should
have the highest priority for the organization. Every variable (risk, financial and
strategic fit) is thoroughly assessed, which is a time consuming process. This could
limit the usability of the Kalypso model v2. Users might be hesitant to use the
Kalypso model v2 because of these resource requirements (time and people). The
Kalypso model v2 uses multiple criteria and is able to rank projects on predetermined
criteria (1,2, 1.3 and 1.4). The final NPD portfolio score enables users to compare
projects independent of their development phase and allows an equitable comparison
(1.5). The Kalypso model v2 does not solely depend on financial tools and
incorporates qualitative criteria (1.6 and 1,7).
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 82
5.2.2 Process Criteria Table 5.4 KE model process criteria evaluation
M. B
roekho
f
M. D
e Mulde
r
L. Kester
Average
2.1. The NPD portfolio method should be divided in phases (Archer and Ghsemzadeh, 1999,2007)
7 7 4 6,0
2.2. A project should be re-‐evaluated each time it transfers to another stage (Cooper, 1990, 2008, Archer and Ghsemzadeh, 1999,2007)
7 7 5 6,3
2.3. Individual project development and evaluation should be based on the Stage Gate® process (Cooper, 1990, 2008, Archer and Ghsemzadeh, 1999,2007)
7 7 4 6,0
2.4. The NPD portfolio should be reviewed periodically (Cooper et al., 1998, 1999, Archer and Ghsemzadeh, 1999, 2007, Chao and Kavadias, 2007)
7 7 5 6,3
2.5. Each project should be assessed individually on its (financial) value (Archer and Ghasemzadeh, 1999).
7 7 7 7,0
2.6. The NPD portfolio management tool should take into account the interdependencies between projects (Archer and Ghasemzadeh, 1999)
7 5 2 4,7
2.7. The NPD portfolio management tool should take into account the time dependent nature of project resource selection (Archer and Ghasemzadeh, 1999)
1 1 2 1,3
Average 6,1 5,9 4,1 5,4
Poor 1 2 3 4 5 6 7 Excellent
The Kalypso model v2 process is divided in phases (see paragraph 5.1.5) where each
project is re-evaluated each time a project transfers to another stage (2.1 and 2.2).
This process is based on the Stage Gate® process (2.3). The portfolio is reviewed
periodically based on individual project assessments (2.4 and 2.5). Project
interdependencies are treated during the risk and strategic fit assessments. However,
these assessments only look at interdependencies in a very limited manner. Linda
Kester is of the opinion that this assessment of interdependencies is too limited. This
is a design choice, complexity would increase drastically if a more rigorous
assessment of project interdependencies were implemented (2.6). This would be at the
cost of the criteria 1.1 (usability and understandable) and would limit the practical
applicability of the model (page 2). The same goes for implementing an assessment of
the time depended nature of project resource selection (2.7).
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 83
5.2.3 Strategic Criteria Table 5.5 Kalypso model v2 strategic criteria evaluation
M. B
roekho
f
M. D
e Mulde
r
L. Kester
Average
3.1. The NPD portfolio management tool must assess projects on strategic fit (Cooper and Kleinschmindt, 1995, Cooper et al., 1998)
7 7 4 6,0
3.2. The strategy should be determined before the NPD portfolio selection process starts (Archer and Ghsemzadeh, 1999,2007)
7 7 4 6,0
3.3. A clear strategy consists of the who, what, how, why and threats (Markides, 1999, Loch and Tapper, 2002) or consist out of four perspectives (financial, customer perspective, internal business process and learning and growth perspective) (Kaplan and Norton, 1996)
7 7 3 5,7
3.4. The NPD portfolio management tool should be able to align the portfolio to the strategy (Cooper et al. 1998, 1999)
7 7 3 5,7
3.5. The NPD portfolio management tool should be top-‐down and bottom-‐up (Cooper et al., 1998)
7 7 3 5,7
3.6. The strategy must make clear where and how many resources should be spent (Cooper et al., 1998)
7 7 3 5,7
3.7. Projects should be classified in advance of the portfolio selection (e.g. strategic Buckets) (Cooper et al., 1998, 1999, Archer and Ghsemzadeh, 1999, 2007, Chao and Kavadias, 2007)
7 7 4 6,0
3.8. Projects should be assigned to buckets Barczak et al. (2009) 7 7 6 6,7
Average 7,0 7,0 3,8 5,9
Poor 1 2 3 4 5 6 7 Excellent
The Kalypso model v2 assesses projects on strategic fit (3.1). This assessment only
works if the strategy is determined beforehand (3.2). The strategy in the Kalypso
model v2 must be able to answer the who, what, how, why and threats (3.3). By using
strategic buckets it is clear where and how many resources are spend (3.6, 3.7 and
3.8). Top management defines a strategy, including goals and objectives. In addition,
top management formulates buckets. Each bucket is accompanied by a set of criteria.
Assessed on the criteria formulated by higher management, lower level management
fills the buckets with projects. (The projects are individually assessed by a
consecutive lower level.) Consequently, the model is a top down and bottom up
approach (3.5). Projects with the highest (overall) score, will probably score high on
strategic fit. Hence, by selecting projects with the highest score, users are able to align
the portfolio to an organization’s strategy. This all results in a model that is able to
align the portfolio to an organization’s strategy (3.4). Linda Kester argues that the
Kalypso model v2 aids users too little in the development of the strategy. She fears
that this will be the weak point of the Kalypso model v2, consequently she scored the
Kalypso model v2 low on other criteria.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 84
5.2.4 Financial Criteria Table 5.6 Kalypso model v2 Financial criteria evaluation
M. B
roekho
f
M. D
e Mulde
r
L. Kester
Average
4.1. The NPD portfolio management tool should strive to maximize the value of the portfolio (Cooper et al., 1998, 1999)
7 7 5 6,3
4.2. The financial metric should incorporate the cost of development and launch (Cooper et al., 1998)
7 7 6 6,7
4.3. The financial metric should have no bias towards long term radical projects (Chao and Kavadias, 2007)
4 4 3 3,7
4.4. The financial metric should capture the flexibility of the development process (Newton et al., 2004)
7 6 6 6,3
4.5. The financial metric should be constrained by a critical resource (Cooper et al., 1998)
7 7 6 6,7
4.6. The financial metric should ignore sunk cost (Cooper et al., 1998) 7 7 7 7,0
4.7. The financial metric should incorporate the probability of success or failure (Cooper et al. ,1998, Pennings and Lint, 1997)
7 7 6 6,7
4.8. The financial metric should use the Minimum Acceptable Rate of Return (MARR) and Time Value of Money (TVM) (Remer and Nieto, 1995)
7 7 7 7,0
4.9. The financial metric should be user friendly and not too complex or appear as a black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009).
4 5 4 4,3
Average 6,3 6,3 5,6 6,1
Poor 1 2 3 4 5 6 7 Excellent
The Kalypso model v2 strives to maximize the value of an NPD portfolio by using
both financial (qualitative (stages 0 and 1) and quantitative (stages 2,3 and 4)) and
strategic criteria to determine the value of projects (4.1). The quantitative metric used
(ECV), incorporates the costs of development and launch and is constrained by a
critical resource (4.2 and 4.5) (see paragraph 5.1.3). The ECV has a limited bias
towards long-term radical projects because it captures the flexibility of the
development process by incorporating the probabilities of success or failure into the
calculations (4.3, 4.4 and 4.7). These probabilities are determined by assessment of
relevant risks (see paragraph 5.1.3). ECV uses the NPV for its calculations, the NPV
uses the MARR and TVM (4.8). The use of the ECV and probabilities increase the
complexity and decreases user-friendliness (4.9).
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 85
5.2.5 Balance Criteria Table 5.7 Kalypso model v2 Balance criteria evaluation
M. B
roekho
f
M. D
e Mulde
r
L. Kester
Average
5.1. The NPD portfolio tool must be able to visually depict the portfolio (Cooper et al., 1998)
7 7 6 6,7
5.2. Management must define what balance and imbalance is (Cooper et al., 1998) 5 5 5 5,0
5.3. The NPD portfolio should only show the for management most important diagrams (Cooper et al., 1998)
4 5 1 3,3
5.4. The Bubble diagram X and Y-‐axis should be based on a companies strategic goals and objectives Cooper et al., 1998)
7 7 1 5,0
Average 5,8 6,0 3,3 5,0
Poor 1 2 3 4 5 6 7 Excellent
The Kalypso model v2 can produce various visual depictions (5.1). E.g., the model
uses risk and financial data, consequently, it can depict a risk reward diagram. It uses
Stage Gate® data, hence it can depict which projects are in what development stage.
Etc. The Kalypso model v2 addresses the notion that balance has to be defined,
however it doesn’t aide users in determining what balance is (5.2). Balance is defined
by management and has to be based on the strategy. E.g., an organization wants to
spend 70% of their R&D resources on radical new projects. When a “type of project –
development cost diagram” shows that only 40% of the budget is spend on radical
projects while 60% is spend on incremental innovation one can conclude that the
portfolio is imbalanced. Another possibility is to look at the risk profiles of each
project. The number of acceptable high, medium and low risk projects could be
predetermined. However, the Kalypso model v2 can't define what balance is, its
unique to each organization. The same applies to determining what the most
important diagrams are (5.3). Linda Kester, argues that this is a management decision
and, consequently, isn’t a function of the Kalypso model v2. The author and Mick
Broekhof disagree, if an organization decides what the most important diagrams are,
the Kalypso model is able to depict them. The Kalypso model v2 helps organizations
to implement their strategy, this enables the model to produce diagrams that are based
on an organization’s goals and objectives (5.4).
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 86
5.2.6 Risk Criteria Table 5.8 Kalypso model v2 Risk criteria evaluation
M. B
roekho
f
M. D
e Mulde
r
L. Kester
Average
6.1. The NPD portfolio management tool should assess projects on risk (Halman and Keizer, 1994. Keizer et al., 2005
7 7 7 7,0
6.2. Risk should be assessed on three dimensions; likelihood, impact, and the ability to do something about it (Halman and Keizer, 1994)
7 7 7 7,0
6.3. Risk is assessed on a sufficient number of criteria (Keizer et al., 2005) 6 6 7 6,3
Average 6,7 6,7 7,0 6,8
Poor 1 2 3 4 5 6 7 Excellent
The Kalypso model v2 assesses projects on risk (6.1). Each project is assessed on
three dimensions; occurrence, impact and ability to do something about it (6.2). The
Kalypso model advocates using a sufficient number of risks (60) for the assessment of
each project (6.3) (see paragraph 5.1.2).
5.2.7 Usability Criteria Table 5.9 Kalypso model v2 Usability criteria evaluation
M. B
roekho
f
M. D
e Mulde
r
L. Kester
Average
7.1. Governance roles and responsibilities must be defined (Cooper, 2008) 6 6 6 6,0
7.2. The NPD portfolio should be periodically assessed (Cooper, 2008) 7 7 5 6,3
7.3. The NPD portfolio management method is user friendly and not too complex or appears as a black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009)
7 5 2 4,7
7.4. Deliverables must be clearly defined for each portfolio assessment (Cooper, 2008)
7 6 5 6,0
7.5. Deliverables should be focused on the essential information needed for NPD portfolio management decisions (Cooper, 2008)
7 7 3 5,7
7.6. Templates should not be overly detailed and be limited to the essential information needed for NPD portfolio management decisions (Archer and Ghazemzadeh, 1998, Cooper, 2008)
7 5 2 4,7
Average 6,8 6,0 3,8 5,6
Poor 1 2 3 4 5 6 7 Excellent
There are more people involved in the Kalypso model v2 compared to the original
Kalypso model. Consequently extra roles and responsibilities are defined in the
Kalypso model v2. It is clear who is responsible for what and why (see paragraph
5.1.6) (7.1). The NPD portfolio is periodically assessed (7.2), however this
assessment period differs per organization. Mick Broekhof and the author find the
overall Kalypso model v2 not complex and easy to understand because all variables
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 87
and their weight are clearly defined. Consequently the deliverables are also clearly
defined (it is clear on what criteria a project is assessed), however each organization
has to define its own deliverables (7.3). As mentioned before, using the Kalypso
model v2 requires resources (especially time). It takes time to collect all information
and conducted the necessary assessments. The usability of the Kalypso model v2
could be limited when users are not prepared to commit these resources (7.3). By
defining on what variables (risk, strategic fit and financial) a project is assessed, it
becomes easy to focus the deliverables (7.5). The Kalypso model v2 risks having too
detailed templates (7.6). This is especially the case for the risk assessment when too
many risks are assessed.
5.3 Conclusion The results of the assessment of the Kalypso model v2 are shown in table 5.10. Table
5.10 also shows the results of the assessment of the original Kalypso model.
Table 5.10 Average scores per category Original Kalypso
model Kalypso model v2
General Criteria 5,7 6,4
Process Criteria 4,7 5,4
Strategic Criteria 2,8 5,9 Financial Criteria 3,1 6,1 Balance Criteria 3,8 5,0
Risk Criteria 3,0 6,8
Usability Criteria 5,2 5,6
Overall score 4,0 5,9 Poor 1 2 3 4 5 6 7 Excellent
What become clear is that the Kalypso model v2 is a significant improvement over
the original Kalypso model. The Kalypso model v2 receives a higher score on all
criteria. Big improvements are made on strategic, risk and financial criteria. These
criteria were the weakest aspects of the original Kalypso model. Based on the results
of the assessment it be concluded is that the Kalypso model v2 can be used as the
single and only NPD portfolio management model in an organization. Hence, the
Kalypso model v2 enables organizations to make the best NPD portfolio decisions
possible.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 88
Chapter 6: Conclusion The goal of this master thesis was to develop an NPD portfolio management reference
model that enables users to make the best possible NPD portfolio decisions. The
result of this master thesis project is the Kalypso model v2. By assessing each project
on three relevant variables (strategic fit, financial and risk) and using these
assessments to conduct an overall score, the Kalypso model v2 enables users to make
the best NPD portfolio decisions possible. The overall NPD portfolio score allows
users to compare projects independent of their development phase. The Kalypso
model v2 is the result of combining various processes and models developed by
scientists, while building on experience of practitioners and experts. The developed
NPD portfolio reference framework enables Kalypso to implement NPD portfolio
management solutions at any client. In the next paragraphs, the scientific and practical
implications are discussed in more detail, accompanied by the model’s limitations.
This chapter ends with recommendations for further research.
6.1 Scientific Implications The Kalypso model contributes to the body of knowledge by combining existing
models with practical knowledge. The Kalypso model v2 is the first model that
combines the NPD portfolio management process developed by Archer and
Ghasemzadeh (1999), research conducted by Cooper et al. (1998, 2001) and the Risk
Diagnosing Methodology by Keizer et al. (2002).
During this research, a tool was developed for evaluating NPD portfolio management
methods. Based on the scientific literature, a list of relevant criteria was composed
(table on which an NPD portfolio management tool can be assessed. The assessment
score provides an indication of usability of an NPD portfolio method. This NPD
portfolio management method assessment tool will also indicate what the strong and
weak points of the method are.
6.2 Scientific Limitations A limitation of this research is that the model presented in this master thesis report is
not empirically tested. Empirical testing provides insight in how the model works in
practice. It will highlight what is applicable and what is not. Another limitation
concerns the validation of the model. Experts were involved in the validation of the
new model. Their views and comments were used to adjust and improve the new
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 89
model. However, the number of experts involved is limited. The background of these
experts is consulting and academic research. Hence, no people who actually conduct
portfolio management were involved. This hampers the validity of the Kalypso model
v2.
Table 6.1 Scientific Limitations
• The Kalypso model v2 is not tested • Limited number of experts involved • No practitioners who actual conduct NPD portfolio management were involved
6.3 Practical Implications The first goal of this master thesis is to develop an NPD portfolio model that can be
used by practitioners. The Kalypso model v2 receive a 5,9 out of 7,0 in the
assessment and therefore can be used as the only and primary tool in an organization
(see paragraph 5.3). The Kalypso model v2 scored significantly higher on the
assessment then the original model and the various other methods. The new model
can help practitioners overcome many of the difficulties encountered in NPD portfolio
management today. A recent (2010) benchmark study, commissioned by Planview®,
with 922 respondent, found that companies have too many projects for their
resources. This can partly be explained by the reluctance of organizations to abandon
under-performing projects (Planview®, 2010). In the Kalypso model v2 projects are
assessed thoroughly and the results are translated into a universal score. This enables
organizations to objectively compare different projects independent of the
development phase. Consequently, organizations have a better insight in the
performance of various projects. Due to this “ranking”, deciding to abandon or
continue projects becomes more rational; there is less room for subjectivity. Hence,
using the Kalypso model v2 will decrease the probability of an overloaded NPD
portfolio. In addition, the implementation of buckets will lead to more focused
resource allocations, this will also results in a decreased risk of a portfolio overload.
Every bucket has its own committed resources. The number of projects in one bucket
should have no effect on the resource availability in another bucket.
The benchmark study, commissioned by Planview®, found that the most important
criteria for a company is strategic alignment followed by revenue and margin (hence,
the last two are financial criteria). Risk wasn’t indicated as a top priority, but still
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 90
important. The Kalypso model v2 uses the same variables (strategic, financial and
risk) to conduct an overall score for each project. The Kalypso model v2 uses criteria
which are relevant for practitioners.
Not all organizations are able to directly implement the Kalypso model v2. As
described earlier, every organization can be ranked on a maturity scale. Dooley et al.
(2001) define maturity as “the extent to which a process is explicitly defined,
managed, measured and continuously improved”. There are various levels of
organizational maturity and not all levels are compatible with the Kalypso model v2.
Figure 6.1 shows the various maturity levels defined by the Organization Project
Management Maturity Model (OPMP3tm) accompanied by some of their properties.
Figure 6.1 OPMP3tm maturity levels
Source: based on P3M3 Public Consultation Draft v2.0 http://www.ogc.gov.uk/documents/p3m3.pdf
Only organizations with a maturity level 3 or higher are able to implement the
Kalypso model v2. There are many people involved in the Kalypso model v2. E.g. all
project team members, project managers, process managers, high level management
etc. Consequently, an organization should be committed to provide these people to the
NPD portfolio management process. This can only be achieved if top management is
Level 1 - Initial Process- Ad-hoc- No proces defined- No standards- Limited formal checking or review- Information outdated- No formalization
Level 2 - Repeatable Process- Partly, localized, defined process- Localized standards- Estimation is more "guestimation" and does not use standard techniques-Inconsistent business objectives-lack of fully integrated risk management
Level 3 Defined Process- Centrally defined process- Organization wide standards-Strategies transformed into portfolio initiatives- Redundant initiatives are eliminated-Roles are defined- High level management involvement
Level 4 - Managed Process-Robust framework in place-High level organizational involvement- Information is current and extensively referenced for better decision making-Quantitative and qualitative measure are used
Level 5 - Optimized Process-Portfolio management processes proven and optimized-Key Performance Indicators aligned to the organization's strategic objectives
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 91
involved. NPD portfolio management requires various departments to cooperate and
provide information. Therefore organization wide standards and processes have to be
defined, otherwise collaboration and communication will be a cumbersome effort.
For organizations that have a maturity level of 1 or 2, the Kalypso model v2 can
provide a starting point or reference guide from which they can improve their NPD
portfolio management process. The Kalypso reference model can provide
practitioners various insights concerning NPD portfolio management like:
• The Kalypso model provides insight in the application of strategic buckets
(paragraph 5.1.1).
• The Kalypso model v2 provides insight in which variables should be assessed
and how an overall score can be conducted (paragraph 5.1.7)
• Risk should be assessed on three dimensions occurrence, impact and control
(paragraph 5.1.2)
• Although financial criteria are the second most important criteria for
practitioners (Planview®, 2010), quantitative financial metrics are less
appropriate in the early stages of the development process. However, via a
qualitative assessment it is possible to determine a financial score (paragraph
5.1.3).
• The Kalypso model v2 provides insight in which variable is important in
which development phase (paragraph 5.1.7)
• Responsibilities throughout an organization with regards to NPD portfolio
management (paragraph 5.1.6)
6.4 Practical Limitations There are some possible risks involved in using the Kalypso model v2 . During one of
the expert discussions, the concern was raised that the implementation of the Kalypso
model could result in a loss of organizational flexibility. This concern was based on
the formal process proposed in the Kalypso model v2. Formalization comes at the
cost of flexibility (Kester et al., 2009).
Another possible limitation is the level of information and commitment required in
order to effectively use the Kalypso model v2. Many people throughout the
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 92
organization are involved in the new model. Consequently, it takes up precious
resources. In order to make thoroughly considered NPD portfolio decisions a
significant number of people are required in the Kalypso model v2. They collect
information, conduct assessments and monitor the process (e.g. the project risk
assessment is conducted by project team members and is monitored by a NPD process
manager). By involving various people throughout the organization, the risk of
continuing underperforming projects is reduced. In the Kalypso model v2, employees
who are directly involved in the development of a project do not make the final
portfolio decisions. By providing objective scores, decision makers are enabled to
make a sound decision without being influenced by various biases. However, this is at
the cost of including numerous employees in the process.
The Kalypso model v2 is a reference model. Consequently, it needs to be tailored to
every organization. This takes time and effort, e.g., the determination of project
specific risks is a cumbersome process. The same applies to the development of the
strategy, corresponding objectives and translating them to NPD portfolio management
criteria. Table 6.2 lists the practical limitations of the Kalypso model v2.
Table 6.2 Practical Limitations
• The Kalypso model v2 could lead to loss of flexibility • High commitment is required of the organization • Many people are involved • The Kalypso model v2 can’t directly be applied
6.5 Recommendation for further research There are still many areas concerning NPD portfolio management that are not covered
in this research. In this paragraph various recommendations for further research are
proposed.
• Project interdependencies are complex to measure and manage.
Interdependencies play a small part in the Kalypso model v2. Future research
could be aimed at investigating the role of interdependencies with regard to
NPD portfolio management.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 93
• The implementation of the Kalypso model is dependent on the maturity of an
organization. Future research could focus on which steps organizations must
take in order to implement the Kalypso model.
• Future research could be aimed at testing the Kalypso model in practice. The
current model is theoretical and the next step could be testing the Kalypso
model in practice.
• The focus of this research was NPD portfolio management at the level of
individual projects. At program or business unit level, the scope of variables
change from specific to more general (Chao and Kavadias, 2007). This brings
a different dynamic into the NPD portfolio process. Further research could be
conducted regarding different levels in NPD portfolio management.
• Not all industries are alike, e.g. pharmaceutical manufactures differ
substantially from electronics manufactures with regard to the NPD process.
These differences could affect the applicability of the Kalypso model v2.
Further research is recommend to address these possible influences.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 94
References Abetti, P.A., Stuart, R.W., 1988, Evaluating new product risk, Research Technology Management, vol. 31, pp. 40-
44
Adler, P.S., Mandelbaum A., Nguyen V., Schwerer, E., 1995, From project to process management: an
empirically-developed framework for analyzing product development time, Management Science, vol. 41, pp. 458-
484.
Andrews K. R. 1995. The concept of corporate strategy. The strategy process. H. Mintzberg, J. B. Quinn, S.
Ghoshal. Prentice Hall London, from K. R. Andrews. 1980. The concept of corporate strategy. Copyright Richard
D. Irvin, Inc.
Archer, N., Ghasemzadeh, F., 2007, Project portfolio selection and management, in Morris, P.W.G., Pinto, J.K.,
The Wiley Guide to Project Program & Portfolio Management, pp.94-112, New Jersey: John Wiley & sons
Barczak, G., Griffin, A., Kahn, K.B., 2009. Perspective: trends and drivers of success in NPD practices: results of
the 2003 PDMA best practices study, Journal of Product Innovation Management, vol. 26, pp. 3–23
Balachandra, R., Friar, J.H., 1997, Factors for success in R&D projects and new product innovation: a contextual
framework. IEEE Transactions on Engineering Management, vol. 44, pp. 276–287.
Batenburg, R., Helms, R., Versendaal, J., 2005, The maturity of product lifecycle management in Dutch
organizations—A strategic alignment perspective, Institute of Information and Computing Sciences, Utrecht
University, Technical report UU- CS-2005-009
Boyer, K.K., McDermott, C., 1999, strategic consensus in operations strategy, Journal of Operations
Management, vol. 17, pp. 289-305
Chao, R., Kavadias, S., 2004, A theoretical framework for managing the new product development portfolio: when
and how to use strategic buckets, Management Science, vol. 54, pp. 907-921
Chao, R., Kavadias, S., 2007, Resource allocation and new product portfolio management, Handbook of New
Product Development Research, Oxford: Elsevier/Butterworth
Coldrick, S., Longhurst, P. Ivey, P., Hannis, J., 2003, An R&D options selection model for investment decisions,
Technovation, vol. 25, pp. 185-193
Cooper, R.G., 1990, Stage-gate systems: a new tool for managing new products, Business Horizons, vol. 33, pp.
44-54
Cooper, R.G., 1993, Winning at new products, 2nd edition, Addison-Wesley, Reading, Ma
Cooper, R.G., 2008, Perspective: the Stage Gate® idea-to-launch process- update, what’s new, and nexgen
systems, Product Innovation Management, vol. 24, pp. 213-132
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 95
Cooper, R.G., Edgett, S.J., 2001, Portfolio management for new products: picking the winners, Product
Development institute, http://www.stage-gate.net/downloads/working_papers/wp_11.pdf
Cooper, R.G., Edgett, S.J., 2006, 10 ways to make better portfolio and project selection decisions, Stage Gate®
International, http://www.stage-gate.net/downloads/working_papers/wp_24.pdf
Cooper, R.G., Edgett, S.J., Kleinschmidt, E.J., 1998, Portfolio management for new products, Perseus Books,
Cambridge, Ma
Cooper, R.G., Edgett, S.J., Kleinschmidt, E.J., 1999, New product portfolio management: practices and
performance, Journal of Product innovation Management, vol. 16, pp. 333-391
Cooper, R.G., Edgett, S.J., Kleinschmidt, E.J., 2001. Portfolio management for new product development: results
of an industry practices study, R&D Management, vol. 31, pp. 361- 380
Cooper, R.G., Edgett, S.J., Kleinschmidt, E.J., 2002, Optimizing the stage-gate process: What best –practice
companies do- II, Research Technology Management, vol. 2, pp 43-49
Cooper, R.G., Kleinschmidt, E.J., 1995, Benchmarking the firm’s critical success factors in new product
development, Journal of Product innovation Management, vol. 12, pp. 374-351
Docherty, M., 2006, Primer on "Open Innovation ": Principles and Practice, Vision, vol.2, pp 13-17
Faulkner, T.W., 1996, Applying “option thinking” to R&D valuation, Research Technology Management, vol. 39,
pp. 50-56
Graziano, A.M., Raulin, M.L., 2005, Research methods, A process of Inquiry – Fifth edition, Pearson Education
Group, Boston, MA
Griffin, A., 1997, PDMA research on new product development practices: updating trends and benchmarking best
practices, Journal of Product Innovation Management, vol. 14, pp. 429-458
Henard, D.H., Szymanski, D.M., 2001, Why some new products are more successful than others, Journal of
Marketing Research, vol.38, pp. 362-275
Joshi, M.P., Kathuria, R., Porth, S.J., 2003, Alignment of strategic priorities and performance: an integration of
operation and strategic management perspectives, Journal of Operations Management, vol. 21, pp. 353-369
Kahneman, D., Tversky, A., 1979, Prospect theory: an analysis of decision under risk, Econometrica, vol. 47, pp.
263-292
Kaplan, R.S., Norton, D.P., 2001, Transforming the balanced scorecard form performance measurement to
strategic management: Part I, Accounting Horizons, vol. 15, pp. 87-104
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 96
Keizer, J.A., Halman, J.I.M., Song, M., 2002, From experience: applying the risk diagnosing methodology,
Journal of Product Innovation Management, vol. 19, pp. 213-232
Keizer, J.A., Vos, J.P., Halman, J.I.M., 2005, Risks in new product development: devising a reference tool, R&D
Management, vol. 35, pp. 297-307
Kester, L., Griffin, A., Hultink,E.J., Lauche, K., 2009, Towards a general model of portfolio decision making
effectiveness, Accepted for Acadeny of Management Annual Meeting
Kester, L., Hultink, E.J., Lauche, K., 2009, Portfolio decision-making genres: a case study, Journal of Engineering
and Technology Management, vol. 26, pp. 327-341.
Loch, C. H., Tapper, S., 2002, Implementing a strategy-driven performance measurement system for an applied
research group, Journal of Product Innovation Management, vol. 19, pp 185-198.
March, J.G., Shapira, Z., 1987, Managerial perspectives on risk and risk taking, Management Science, vol. 33, pp.
1404-1418
Markides, C.C., 1999, A dynamic view of strategy, Sloan Management Review, vol. 40, pp. 55-64
Martino, J.P. 1995, R&D project selection, Wiley, New York, NY
Montoya-Weiss, M.M., Calantone, R., 1994, Determinants of new product performance: a review and meta
analysis, Journal of Product Innovation Management, vol. 11, pp. 397-417.
Newton, D.P., Paxson, D.D., Widdicks, M., 2004, Real R&D options, International Journal of Management
Reviews, vol. 5/6, pp. 112-130
Park, C.S., 2007, Contemporary Engineering Economics (4th Edition). Prentice Hall.
Pattikawa, L.H., Verwaal, E., Commandeur, H.R., 2006, Understanding new product project performance,
European Journal of Marketingi, vol. 40, pp. 1178-1193
Pennigs, E., Lint, O., 1997, The option value of advanced R&D, European Journal of Operations Research, vol.
103, pp. 83-94
Pennings, E., Lint, O., 1997, The option value of advanced R&D, European Journal of Operational Research, vol.
103, pp. 83-94
Pich, M.T., Loch, C.H., De Meyer, A., 2002, On Uncertainty, Ambiguity, and Complexity in Project Management,
Management Science, vol. 48, pp. 1008-1023
Planview®, 2010, 2nd Annual benchmark study: Product portfolio management, www.planview.com
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 97
Remenyi, D. and Heafield, A. (1996), “Business process re-engineering: some aspects of how to evaluate and
manage the risk exposure”, International Journal of Project Management, Vol. 14 No. 6, pp. 349-57
Remer, D.S., Nieto, A.P., 1995, A compendium and comparison of 25 project evaluation techniques. Part 2 Ratio,
payback and accounting methods, International Journal of Production Economics, vol. 42, pp. 101-129
Skinner, W., 1974, The focused factory, Harvard Business Review, vol. 3, pp. 113-119
Sommer, S.C., Loch, C.H., Pich, M.T., 2008, Project risk management in new product development, Loch, C.H.,
Kavadias, L., Handbook of New Product Development Management, Elsevier Linacre House, Oxford
Stark, J., 2004, Product lifecycle management – 21st century paradigm for product realization, Springer, London
Thompson, J.D., 1967, Organizations in Action, McGraw-Hill, New York.
Turner, J R, 1993, The Handbook of Project Based Management, Improving the Processes for Achieving Strategic
Objectives, McGraw-Hill, UK
Tushman, M.L., O’Reilly, C.A., 1996, Ambidextrous organizations: Managing evolutionary and revolutionary
change, California Management Review, vol. 38, pp. 8–30
Verma, D., Sinha, K.K., 2002, Toward a theory of project interdependencies in high tech R&D environments,
Journal of Operations Management, vol. 20, pp. 451-468
Webb, A., 1994, Managing Innovative Projects, Chapman & Hall, London
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 98
Appendices
Appendix 1: Description of the different stages of the Stage Gate® process
Description of the different stages of the Stage Gate® process Stage 0: Discovery The process starts with the first stage discovery phase, in this phase ideas are generated freely.
Gate 1: Initial Screen The ideas are submitted to the first gate idea screen. At this gate the first decisions to commit resources to the project. Project are selected with a set of must meet and should meet criteria. These criteria are cover strategic alignment, project feasibility, magnitude of the opportunity differential advantage, synergy with the firm’s core business and resources and market attractiveness. Financial criteria are not part in this gate.
Stage 1: Preliminary Assessment When a project passes the first gate it enters a stage called scoping. This is an inexpensive stage which has the objective to determine a project’s technical and marketplace merits. This stage is about gathering market and technical information at low cost and in a short time.
Gate 2: Second Screen At gate 2 the project undergoes a second screening. The project is reevaluated based on the information gathered during the previous stage. The project is screened with the same criteria as gate 1. There are also extra criteria about dealing with sales force and customer reaction to the proposed product. A quick and simple financial return is calculated but a checklist and scoring model facilitate the gate decision.
Stage 2: Building business case During the next stage a business case is build. The attractiveness of a project is verified, market research studies are undertaken to determine the customer’s needs, wants and preferences. A detailed technical assessment focused on the “do-‐ability” of the project must be made. Customer needs and wants must be translated into technically and economically feasible solutions. A detailed financial analysis is conducted involving a discounted cash flow approach, complete with sensitivity analysis.
Gate 3: Decision on Business Case The next gate is the final gate prior to the development stage. After this gate the spending on the project increases significant. The project is evaluated on qualitative and quantitative criteria. The qualitative evaluation involves a review of each of the activities in stage 2. The quantitative evaluation involves a review of the financial analysis made in the previous stage. At gate 3 agreements must be reached on a number of key items before the project proceeds into development. These items include target market definition, definition of the product concept, specification of a product positioning strategy, definition of the product benefits to be delivered and agreement on essential and desired product features attributes and specification. If a project passes gate 3; development is started.
Stage 3: Development During this stage the product, detailed tests, marketing and operational plans are developed. The financial analysis is updated with new and more accurate data.
Gate 4: Post Development Review Gate 4 is a check on the progress and the continued attractiveness of the product and project. Development work is reviewed and checked, ensuring that the work has been completed in a quality fashion. This gate revisits the economic question via a revised financial analysis based on new and more accurate data.
Stage 4: Validation This stage tests the entire viability of the project: the product itself, the production process, customer acceptance and the economics of the project.
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 99
Description of the different stages of the Stage Gate® process (continued)
Gate 5: Pre-‐Commercialization Decision Gate 5 is the final gate between testing and validation and launch. Financial projections play a key role in the decision to move ahead. Operations and marketing plans are reviewed an approved for implementation. Gate 5 is the final point at which the project can still be killed. Stage 5: Launch The final stage of the Stage Gate® process is the launch stage. This stage involves the implementation of both marketing and launch plan and the operations plan. Source: Cooper (1990), page 52-‐53
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 100
Appendix 2: Description of different levels of PML maturity Description of different levels of PLM maturity No PLM investment, or on ‘ad-‐hoc’ basis only (level 0). Nobody is responsible for PLM and there is no vision available for PLM. Therefore, there are no consistent PLM processes and supporting systems. At this level, information about a product is scattered throughout the organization, which hinders strategic decision-‐making. On departmental level (‘silo’ orientation) (level 1). PLM is seen as a data management problem that should be dealt with on departmental level, but there is no overall vision to coordinate local initiatives. Often the development or engineering department is the first department that starts to implement PLM systems. At this level, at least all information regarding the early stages of a product is stored in a central system. On the organizational level (cross-‐departments) (level 2). PLM is interpreted as a business problem that requires a corporate vision and an integral approach. Besides engineering and development also other departments are heavily involved. PLM processes are defined that cross-‐departmental borders and organization wide PLM systems are implemented to support these processes. Moreover, PLM systems are integrated with other major enterprise systems, such as ERP. At this level, all product information within the organization is stored in a central system and there is control information available regarding PLM processes. On the inter-‐organizational level (cross supply chain partners) (level 3). PLM is seen as a business problem that spans the complete product lifecycle. Therefore, the supply chain should be involved in defining a PLM vision. PLM processes are defined that cross-‐organizational borders and PLM systems are integrated with those of the suppliers to enable collaboration. At this level, all product information across the product lifecycle is stored in a central system making the product lifecycle become transparent enabling proper decision-‐ making concerning a product. Source: Batenburg et al. (2005) page 5
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 101
Appendix 3: Risk Reference tool including risk issues taken from literature and issues resulting from case studies Risk Reference Framework including risk issues taken from literature and issues resulting from case studies
Product Family and Brand positioning Risks
1 Literature Contribution to category’s business strategy
2 Literature Contribution to project portfolio
3 Cases Contribution to brand name position
4 Cases Global roll out potential and schedule
5 Cases Fit within existing brand
6 Cases Cannibalization of existing portfolio
7 Cases Fit with brand image
8 Literature Family development potential
9 Cases Platform deployment
10 Cases Organization reputation
11 Cases Brand recovery potential
12 Cases Brand development potential
13 Cases Consumer acceptance of platform
Product technology risks
1 Literature Intended functions known and specified
2 Literature New product fulfills intended functions
3 Cases In-‐use conditions known and specified
4 Cases Interactions of product in-‐use with sustaining materials, tools etc.
5 Cases Components’ properties, function and behavior
6 Cases Correct balance between product components
7 Cases Product format meets functional requirements
8 Literature Safety and technical requirements for assembled product
9 Literature Alternatives to realize intended product functions
10 Cases Parity in performance compared with other products
11 Cases Stability of product while in storage (factory, shop/warehouse, transportation, at home)
Manufacturing technology risks
1 Cases Raw materials meeting technical requirements
2 Cases Process steps to realize the new product
3 Literature Conditions (temperature, energy, safety requirements) to guarantee the processing of a good product quality
4 Literature Production means (equipment and tools) necessary to guarantee good product quality
5 Literature Scale up potential according to production yield standards
6 Literature Production system requirements (quality and safety standards, training of human resources, facilities etc.)
7 Cases Product packaging implications
8 Cases Alternative approaches to process the required product
9 Cases Efficiency of production
10 Literature Adequate production capacity available
11 Literature Adequate Production Start Up
12 Cases Reusability of rejects in production
Intellectual property position
1 Literature Protection of original know-‐how
2 Literature Dependency on third party development
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 102
Risk Reference Framework including risk issues taken from literature and issues resulting from case studies (continued) 3 Literature Availability of required external licenses or know-‐how
4 Literature Relation to legal and patent rights of competitors
5 Literature Knowledge of relevant patent issues
6 Cases Patent crossing potential
7 Literature Trade mark registration potential
Supply chain and sourcing risks
1 Cases Constant and predictable product quality
2 Cases Capacity to meet peak demands
3 Literature Appropriate after sales services
4 Cases Contingency options for each of the selected suppliers
5 Cases Reliability of each supplier in delivering according to requirements
6 Cases Financial position of each supplier ensuring a long-‐term supply performance
7 Cases Past experiences with each of the suppliers
8 Cases Suppliers’ readiness to accept modifications if required
9 Cases Possibility to cancel supply contracts
10 Cases Ability to produce required quantities against acceptable prices
11 Cases Appropriate contract arrangements with suppliers
Consumer acceptance and Marketing risks
1 Literature Product specifications meeting consumer standards and demands
2 Cases Fit of new product with consumer habits and/or user conditions
3 Literature Consumer benefits of new product: unique features or attributes
4 Literature Consumers’ conviction that they get value for money, compared with competitive products
5 Literature Product’s appeal to generally accepted values (e.g. health, safety, nature, environment)
6 Literature Product’s easy-‐in-‐use advantages, compared with competitive products
7 Literature Product offering additional enjoyment, compared with competitive products
8 Literature Product reducing consumer’s costs, compared with competitive products
9 Cases Non-‐intended product use by consumers
10 Cases Stability of target consumer’s attitudes during the development period
11 Literature Communicating the product with target consumers
12 Literature Knowledge of primary consumer requirements
13 Literature Target consumers accepting key product ingredients
14 Literature Niche marketing capabilities
15 Cases Realistic product claim
16 Literature Efficacy of advertising
17 Literature Product claims stimulating target consumers to buy
18 Literature Repeat sales potential of new product
Trade customer acceptance risks
1 Cases Product specifications meeting trade customer standards and demands
2 Cases Trade customers welcoming the new product from the perspective of potential sales
3 Cases Trade customers welcoming the new product from the perspective of profit margin
4 Cases Trade customers welcoming the new product given required surface and volume on shelf and storage facilities
5 Cases Trade customer’s attitude remaining unchanged during the development period
6 Cases Communicating the product with trade customers
7 Cases Use of right distribution channels
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 103
Risk Reference Framework including risk issues taken from literature and issues resulting from case studies (continued) 8 Cases Appropriate product care by trade
9 Cases Trade supporting persons endorsing the new product
10 Cases Meeting stock demands
Competitor risks
1 Literature Implications of being technology leader or follower for this project
2 Literature Product providing clear competitive advantages
3 Literature Impact of the introduction of the new product on market share positions
4 Literature Impact of the introduction of the new product on market prices
5 Literature Competitive product being launched before launch of new product
6 Literature Response actions towards public and media expected from competitors
7 Literature New product enabling the creation of potential barriers for competitors
8 Literature Acting effectively to competitor’s actions
9 Literature Foreseeing future competitor’s challenges
Commercial viability risks
1 Literature The market target clearly defined and agreed
2 Literature Market target based on convincing research data
3 Literature Capital cost projection for the product’s life cycle feasible and based on convincing data
4 Literature Delays in product launch affecting the commercial viability of the product
5 Literature Sales perspectives being realistic
6 Literature Profit margin based on convincing research data
7 Literature Profit margin meeting the organization’s standards
8 Literature The return on investment (R.O.I.) projection meeting the organization’s standards
9 Cases Clear and reliable volume estimates
10 Cases Product viability because of repeat sales
11 Cases Attractive purchasing agreements with suppliers
12 Cases Knowledge of pricing sensitivity
13 Cases Adequate investments to secure safety in production
14 Cases Long term market potential
15 Cases Financing of capital investment being secured
16 Cases Fall back potential to use prior product concept
17 Cases Product viability in spite of market restrictions
Organizational and Project Management Risks
1 Literature Internal political climate being in favor of this project
2 Literature Top management’s priority and support for the project
3 Literature Sponsor’s interest for the project
4 Literature Project mission and project goals being clearly specified and feasible
5 Literature Project team being sufficiently authorized and qualified for the project
6 Literature Project team utilizing the knowledge and experience of (internal) experts effectively
7 Literature Roles, tasks and responsibilities of all team members defined and appropriate
8 Literature Organization and management of the project
9 Literature Decision making process
10 Literature Communication between members in the project team
11 Literature Reliable and feasible estimation of required money, time and (human) resources
12 Literature Required money, time and (human) resources being available when needed
13 Literature Project team being timely informed about project progress
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 104
Risk Reference Framework including risk issues taken from literature and issues resulting from case studies (continued) 14 Literature External parties’ ability and willingness to deliver in time, within budget and conform
technical specifications
15 Literature Contingency options for the parts in the project that have been contracted out to external parties
16 Literature Collaboration within the project team
17 Literature Collaboration with external parties
18 Literature Collaboration between project team and the parent organization
19 Literature Motivated and committed project team
20 Literature Project team paying attention to the right issues
21 Literature Project planning and contingency planning
22 Literature Learning from past experiences
External risks
1 Cases Clearness about who is responsible for PR of this project
2 Cases Clearness abut who are key opinion formers
3 Cases Anticipating effectively to possible negative external reactions
4 Cases Anticipating effectively to legal and political restrictions for markets where the product will be launched
5 Cases Environmental issues
6 Cases Safety issues
8 Cases In case of new technology: checking prior (external) experience
9 Cases Support of key opinion formers
Screening and appraisal
1 Cases Testing and measuring of new product performance targets
2 Cases Testing and measuring trade customer appreciation
3 Cases Testing and measuring consumer appreciation
4 Cases Testing and measuring adverse properties as a consequence of the technological change
5 Cases Credibility of the (internal) measures to external agencies
6 Cases Tests providing reliable evidence
Source: Keizer et al., 2005
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 105
Appendix 4: Assessment of the five different NPD portfolio management methods
Assessment of the five different NPD portfolio management methods
General criteria evaluation Fina
ncial
metho
ds
Prob
abilistic
finan
cial
Strategic
Metho
ds
Checklist and
Scoring Mod
els
Map
ping
approa
ches
1.1. The NPD portfolio management tool is usable and understandable for its users (Kester et al., 2009, Archer and Chasemzadeh, 1999,2007, Loch and Kavadias 2002)
6 3 7 6 7
1.2. The NPD portfolio management tool uses multiple types of methods and criteria (see paragraph 2.4.5) 1 2 7 7 7
1.3. The NPD portfolio management tool is able to rank projects based on pre-‐determined criteria (Archer and Chasemzadeh, 1999,2007) 7 7 7 7 1
1.4. The selection methodology is determined before the NPD portfolio selection starts (Archer and Chasemzadeh, 1999,2007) 7 7 7 7 7
1.5. The NPD portfolio management uses a common measure which enable an equitable comparison (Archer and Chasemzadeh, 1999,2007) 7 7 7 7 7
1.6. The NPD portfolio management tool does not solely depend on financial tools (Cooper and Kleinschmidt, 1999) 1 1 7 7 7
1.7. The NPD portfolio management tool incorporates qualitative criteria (Cooper et al. 1998, 1999) 1 2 7 7 7
Most methods are easy to understand, only the determination of the probabilities of probabilistic financial methods could be a cumbersome enterprise. Strategic methods and scoring model assess projects on multiple criteria and mapping approaches and diagrams can depict multiple criteria. However financial and probabilistic financial methods only assess a project only on a single (financial) metric. When every method is applied on the same manner it is possible to equitable compare different projects.
Process criteria evaluation 2.1. The NPD portfolio method is divided in phases (Archer and Chasemzadeh,
1999,2007) 1 1 1 1 1
2.2. A project is re-‐evaluated each time it transfers to another stage (Cooper, 1990, 2008, Archer and Chasemzadeh, 1999,2007) 4 4 4 4 4
2.3. Individual project development and evaluation are based on the Stage Gate® process (Cooper, 1990, 2008, Archer and Chasemzadeh, 1999,2007)
4 4 4 4 4
2.4. The NPD portfolio is reviewed periodically (Cooper et al., 1998, 1999, Archer and Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007) 4 4 4 4 4
2.5. Each project is assessed individually on its (financial) value (Archer and Ghasemzadeh, 1999). 7 7 1 1 1
2.6. The NPD portfolio management takes into account the interdependencies between projects (Archer and Ghasemzadeh, 1999) 1 1 1 1 1
2.7. The NPD portfolio management tool takes into account the time dependent nature of project resource selection (Archer and Ghasemzadeh, 1999)
1 1 1 1 1
Financial and probabilistic methods are not able to align the portfolio to the strategy because these methods only assess a project on a single metric while a strategic assessment involves multiple criteria. Strategic methods and scoring models are able to align the strategy, and they are both bottom-‐up and top-‐down. Poor 1 2 3 4 5 6 7 Excellent
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 106
Assessment of the five different NPD portfolio management methods (continued)
Strategic criteria evaluation Fina
ncial
metho
ds
Prob
abilistic
finan
cial
Strategic
Metho
ds
Checklist and
Scoring mod
els
Map
ping
approa
ches
3.1. The NPD portfolio management tool assesses projects on strategic fit (Cooper and Kleinschmindt, 1995, Cooper et al., 1998) 1 1 7 5 4
3.2. The strategy is determined before the NPD portfolio selection process starts (Archer and Chasemzadeh, 1999,2007) 1 1 7 5 4
3.3. Does the strategy consists of the who, what, how, why and threats (Markides, 1999, Loch and Tapper, 2002) or consist out of four perspectives (financial, customer perspective, internal business process and learning and growth perspective) (Kaplan and Norton, 1996)
1 1 7 5 4
3.4. The NPD portfolio management tool is able to align the portfolio to the strategy (Cooper et al. 1998, 1999) 1 1 7 5 4
3.5. The NPD portfolio management tool is top-‐down and bottom-‐up (Cooper et al., 1998) 3 3 7 4 4
3.6. The strategy must make clear where and how many resources should be spent (Cooper et al., 1998) 1 1 1 1 1
3.7. Projects should are classified in advance of the portfolio selection (e.g. strategic Buckets) (Cooper et al., 1998, 1999, Archer and Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007)
1 1 1 1 1
3.8. Projects are assigned to buckets (Barczak et al., 2009) 1 1 1 1 1
Financial criteria evaluation
4.1. The NPD portfolio management tool strives to maximize the value of the portfolio (Cooper et al., 1998, 1999) 3 3 6 6 1
4.2. The financial metric incorporates the cost of development and launch (Cooper et al., 1998) 7 7 1 1 1
4.3. The financial metric has no bias towards long term radical projects (Chao and Kavadias, 2007) 1 5 7 7 4
4.4. The financial metric captures the flexibility of the development process (Newton et al., 2004) 1 7 4 5 1
4.5. The financial metric is constrained by a critical resource (Cooper et al., 1998) 7 7 1 1 1
4.6. The financial metric ignores sunk cost (Cooper et al., 1998) 7 7 7 7 7 4.7. The financial metric incorporates the probability of success or failure
(Cooper et al., 1998, Pennings and Lint, 1997) 1 7 1 1 1
4.8. The financial metric uses the Minimum Acceptable Rate of Return (MARR) and Time Value of Money (TVM) (Remer and Nieto, 1995) 4 4 1 1 1
4.9. The financial metric is user friendly and not too complex or appear as a black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009). 7 4 1 1 1
As mentioned in chapter X (goal 1) financial methods are often used of maximizing the value of the portfolio they seldom succeed in it. Scoring models on the other hand do succeed in composing an NPD portfolio with high value projects.
Balance criteria evaluation
5.1. The NPD portfolio tool is able to visually depict the portfolio (Cooper et al., 1998) 1 1 1 1 7
5.2. Management has defined what balance and imbalance is (Cooper et al., 1998) 1 1 1 1 4
5.3. The NPD portfolio only shows, the for management, most important diagrams (Cooper et al., 1998) 1 1 1 1 7
5.4. The Bubble diagram’s X and Y-‐axis are based on a organization’s strategic goals and objectives Cooper et al., 1998) 1 1 1 1 7
Poor 1 2 3 4 5 6 7 Excellent
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 107
Assessment of the five different NPD portfolio management methods (continued)
Risk criteria assessment Fina
ncial
metho
ds
Prob
abilistic
finan
cial
Strategic
Metho
ds
Checklist and
Scoring mod
els
Map
ping
approa
ches
6.1. The NPD portfolio management tool assesses projects on risk (Halman and Keizer, 1994. Keizer et al., 2005 1 4 7 7 1
6.2. Risk is assessed on three dimensions; likelihood, impact, and the ability to do something about it (Halman and Keizer, 1994) 1 4 5 5 1
6.3. Risk is assessed on a sufficient number of criteria (Keizer et al., 2005) 3 3 3 3 3
Probabilistic financial methods, strategic methods and scoring models are all able to assess a project on risk. Because they are able to assess risk, they also should be able to assess each risk on the three dimensions.
Usability criteria evaluation
7.1. Governance roles and responsibilities are defined (Cooper, 2008) 1 1 1 1 1 7.2. The NPD portfolio is periodically assessed (Cooper, 2008) 7 7 7 7 7 7.3. The NPD portfolio management method is user friendly and not too
complex or appears as a black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009)
5 4 4 4 4
7.4. Deliverables are clearly defined for each portfolio assessment (Cooper, 2008) 6 6 6 6 6
7.5. Deliverables are focused on the essential information needed for NPD portfolio management decisions (Cooper, 2008) 5 5 4 4 4
7.6. Templates are not overly detailed and be limited to the essential information needed for NPD portfolio management decisions (Archer and Ghazemzadeh, 1998, Cooper, 2008)
3 3 2 2 2
There is the risk with all these methods that they are too detailed, this risk is especially high with strategic methods, scoring models and diagrams (see paragraph 2.3.5)
Overall score 3,09 3,48 3,98 3,73 3,50
Poor 1 2 3 4 5 6 7 Excellent
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 108
Appendix 5: Overview of Kalypso model performance per criteria ranked by score
Overview of Kalypso model performance per criteria ranked by score 1.4. The selection methodology should be determined before the NPD portfolio selection starts (Archer
and Chasemzadeh, 1999,2007) 7
1.5. The NPD portfolio management tool should use a common measure which enable an equitable comparison (Archer and Chasemzadeh, 1999,2007)
7
2.2. A project is re-‐evaluated each time it transfers to another stage (Cooper, 1990, 2008, Archer and Chasemzadeh, 1999,2007)
7
2.4. The NPD portfolio is reviewed periodically (Cooper et al., 1998, 1999, Archer and Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007)
7
2.5. Each project is assessed individually on its (financial) value (Archer and Ghasemzadeh, 1999). 7 3.1. The NPD portfolio management tool must assess projects on strategic fit (Cooper and Kleinschmindt,
1995, Cooper et al., 1998) 7
4.6. The financial metric should ignore sunk cost (Cooper et al., 1998) 7 4.9. The financial metric is user friendly and not too complex or appear as a black box (Faulkner, 1996,
Cooper et al., 1999, Kester et al., 2009). 7
5.1. The NPD portfolio tool must be able to visually depict the portfolio (Cooper et al., 1998) 7 7.2. The NPD portfolio is periodically assessed (Cooper, 2008) 7 1.7. The NPD portfolio management tool should incorporate qualitative criteria (Cooper et al. 1998, 1999) 6 2.1. The NPD portfolio method is divided in phases (Archer and Chasemzadeh, 1999,2007) 6 7.1. Governance roles and responsibilities must be defined (Cooper, 2008) 6 7.4. Deliverables must be clearly defined for each portfolio assessment (Cooper, 2008) 6 1.1. The NPD portfolio management tool is usable and understandable for its users (Kester et al., 2009,
Archer and Chasemzadeh, 1999,2007, Loch and Kavadias 2002) 5
1.3. The NPD portfolio management tool is able to rank project based on pre-‐determined criteria (Archer and Chasemzadeh, 1999,2007)
5
4.1. The NPD portfolio management tool should strive to maximize the value of the portfolio (Cooper et al., 1998, 1999)
5
6.1. The NPD portfolio management tool should assess projects on risk (Halman and Keizer, 1994. Keizer et al., 2005
5
7.6. Templates should not be overly detailed and be limited to the essential information needed for NPD portfolio management decisions (Archer and Ghazemzadeh, 1998, Cooper, 2008)
5
2.3. Individual project development and evaluation are based on the Stage Gate® process (Cooper, 1990, 2008, Archer and Chasemzadeh, 1999,2007)
4
3.7. Projects are classified in advance of the portfolio selection (e.g. strategic Buckets) (Cooper et al., 1998, 1999, Archer and Chasemzadeh, 1999, 2007, Chao and Kavadias, 2007)
4
4.3. The financial metric should have no bias towards long term radical projects (Chao and Kavadias, 2007)
4
5.4. The diagram’s X and Y-‐axis are based on a organization’s strategic goals and objectives Cooper et al., 1998)
4
7.3. The NPD portfolio management method is user friendly and not too complex or appears as a black box (Faulkner, 1996, Cooper et al., 1999, Kester et al., 2009)
4
1.6. The NPD portfolio management tool should not solely depend on financial tools (Cooper and Kleinschmidt, 1999)
3
3.4. The NPD portfolio management tool is able to align the portfolio to the strategy (Cooper et al. 1998, 1999)
3
3.5. The NPD portfolio management tool is top-‐down and bottom-‐up (Cooper et al., 1998) 3 5.3. The NPD portfolio should only show the for management most important diagrams (Cooper et al.,
1998) 3
6.3. Risk is assessed on a sufficient number of criteria (Keizer et al., 2005) 3 7.5. Deliverables are focused on the essential information needed for NPD portfolio management
decisions (Cooper, 2008) 3
3.2. The strategy is determined before the NPD portfolio selection process starts (Archer and Chasemzadeh, 1999,2007)
2
2.6. The NPD portfolio management tool should take into account the interdependencies between projects (Archer and Ghasemzadeh, 1999)
1
2.7. The NPD portfolio management tool should take into account the time dependent nature of project resource selection (Archer and Ghasemzadeh, 1999)
1
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 109
Overview of Kalypso model performance per criteria ranked by score (continued) 3.3. A clear strategy consists of the who, what, how, why and threats (Markides, 1999, Loch and Tapper,
2002) or consist out of four perspectives (financial, customer perspective, internal business process and learning and growth perspective) (Kaplan and Norton, 1996)
1
3.6. The strategy must make clear where and how many resources are spent (Cooper et al., 1998) 1 3.8. Projects are assigned to buckets (Barczak et al., 2009) 1 4.2. The financial metric should incorporate the cost of development and launch (Cooper et al., 1998) 1 4.4. The financial metric should capture the flexibility of the development process (Newton et al., 2004) 1 4.5. The financial metric is constrained by a critical resource (Cooper et al., 1998) 1 4.7. The financial metric should incorporate the probability of success or failure (Cooper et al., 1998,
Pennings and Lint, 1997) 1
4.8. The financial metric should use the Minimum Acceptable Rate of Return (MARR) and Time Value of Money (TVM) (Remer and Nieto, 1995)
1
5.2. Management must define what balance and imbalance is (Cooper et al., 1998) 1 6.2. Risk is assessed on three dimensions; likelihood, impact, and the ability to do something about it
(Halman and Keizer, 1994) 1
Poor 1 2 3 4 5 6 7 Excellent
NPD Portfolio Management: The Development of a Reference Model For Kalypso
M.L.W. De Mulder 110
Appendix 6: Decision rules for classification into risk classes Decision rules for classification into risk classes
Certainty Ability of team to influence the course of action
Relative importance to project success
RISK CLASS
Certainty Ability of team to influence the course of action
Relative importance to project success
RISK CLASS
1. * * * F
28. * * ? M-‐F 2. * * 0 L
29. * ? * H-‐F
3. * 0 * M
30. ? * * M-‐F 4. 0 * * H
31. * ? ? L-‐F
5. 0 0 * L
32. ? * ? L-‐F 6. 0 * 0 L
33. ? ? * L-‐F
7. * 0 0 L
34. ? ? ? S-‐F 8. 0 0 0 S
35. ? 0 0 L
9. * * m H
36. 0 ? 0 L 10. * m * H
37. 0 0 ? L
11. m * * H
38. ? ? 0 S-‐M 12. * m m M
39. ? 0 ? S-‐H
13. m * m M
40. 0 ? ? S-‐M 14. m m * M
41. * ? 0 L-‐M
15. m m m M
42. * 0 ? L-‐H 16. 0 * m M
43. 0 * ? L-‐M
17. * 0 m M
44. 0 ? * L-‐M 18. 0 m * M
45. ? 0 * L-‐H
19. * m 0 M
46. ? * 0 L-‐M 20. m * 0 M
47. * ? m M-‐H
21. m 0 * M
48. * m ? M-‐H 22. 0 0 m L
49. m ? * M-‐H
23. 0 m 0 L
50. m * ? M-‐H 24. m 0 0 L
51. ? m * M-‐H
25. 0 m m M
52. ? * m M-‐H 26. m m 0 M
53. m ? 0 L-‐M
27. m 0 m M
54. m 0 ? L-‐M F = Fatal risk 55. 0 ? m L-‐M H = High risk 56. 0 m ? L-‐M M = Medium risk 57. ? 0 m L-‐M L = Low risk 58. ? m 0 L-‐M S = Safe, no risk 59. ? m m L-‐M A combination of classes means that the risk team should work out whether the disagreement can be resolved and hence a single classification can be achieved. If consensus can’t be achieved the worst possible case should be assumed.
60. m ? m M
61. m m ? M
62. ? ? m L-‐H
63. ? m ? L-‐H
64. m ? ? L-‐H Source: Keizer et al. (2002), page 230