ORIGINAL RESEARCH Open Access Contemporary methods for ... · ORIGINAL RESEARCH Open Access Contemporary methods for evaluating complex project proposals Hans J Thamhain Abstract

Thamhain Journal of Industrial Engineering International 2013, 9:34http://www.jiei-tsb.com/content/9/1/34

ORIGINAL RESEARCH Open Access

Contemporary methods for evaluating complexproject proposalsHans J Thamhain

Abstract

The ability to evaluate project proposals, assessing future success, and organizational value is critical to overallbusiness performance for most enterprises. Yet, predicting project success is difficult and often unreliable. A four-yearfield study shows that the effectiveness of available methods for evaluating and selecting large, complex projectdepends on the specific project type, organizational culture, and managerial skills. This paper examines the strengthand limitations of various evaluation methods. It also shows that, especially in complex project situations, thedecision-making process has to go beyond the application of just analytical methods, but has to incorporate bothquantitative and qualitative measures into a combined rational judgmental evaluation process. Equally important, theevaluation process must be effectively linked among functional support groups and with senior management in orderto strategically align the project proposal and to unify the evaluation team and stakeholder community behind themission objectives. All of this requires leadership and managerial skills in planning, organizing, and communicating.The paper suggests specific leadership actions, organizational conditions, and managerial processes for evaluatingcomplex project proposals toward future value and success.

Keywords: Project evaluation and selection; Project management; Team leadership; Technology; Decision making;Rational; Judgmental

Challenges of determining potential for futuresuccessPredicting project success has never been easy. The longlist of prominent project failures, ranging from productdevelopments to government social programs, from com-puters to pharmaceutical, and from public transit to super-sonic transport, reminds us of this reality (Cicmil et al.2006; Gulla 2012; Lemon et al. 2002; Standish Group2013). Many projects do not live up to their expectationsor outright fail even before their technical completion inspite of careful feasibility analysis during their proposal orselection stages (El Emam and Koru 2008; Shore 2008).Obviously, the ability to evaluate project proposals and as-sess future success and organizational value is critical tooverall business performance. In fact, few decisions havemore impact on enterprise performance than the resourceallocations for new projects (Shenhar et al. 2007). Virtually,every organization evaluates, selects, and implements pro-jects. Whether these projects are product developments,

Correspondence: [email protected] Department Bentley University, 175 Forest Street Waltham,Massachusetts, USA

© Thamhain; licensee Springer. This is anAttribution License (http://creativecommons.orin any medium, provided the original work is p

2013

organizational improvements, R&D undertakings or bidproposals, pursuing the ‘wrong’ project not only wastescompany resources, but also causes the enterprise to (1)miss critical alternatives, (2) perform less agile in themarket place, and (3) miss opportunities for leveragingcore competencies. Yet, in spite of pressures to avoid thesehigh-cost errors, predicting project success is difficult, andexisting models are often unreliable. Especially in today’scomplex and changing business environment (see ‘Today’sComplex Project Environments’ section), the process ofevaluating and selecting the ‘best’ projects, most suit-able and beneficial for the enterprise, has become both anart and a science, strongly influenced by human andorganizational factors.

Objective and rationale for this studyWhile much has been published on specific methods forproject evaluation, the literature lacks a summary andcomparison of available tools and techniques. Moreover,few studies have examined the human side as a ‘cross-functional tool’ for collectively evaluating project pro-posals toward potential success, an important link that is

open access article distributed under the terms of the Creative Commonsg/licenses/by/2.0), which permits unrestricted use, distribution, and reproductionroperly cited.

mailto:[email protected]

http://creativecommons.org/licenses/by/2.0

http://www.jiei-tsb.com/content/9/1/34

Thamhain Journal of Industrial Engineering International Page 2 of 142013, 9:34http://www.jiei-tsb.com/content/9/1/34

missing in the management literature. This field studycontributes to both areas. First, the paper summarizesthe benefits, challenges, and limitations of popular pro-ject evaluation methods, tools, and techniques. Second,the paper reports the lessons learned from a five-yearfield study of organizational conditions, leadership style,and decision making with regard to impact on projectselection processes.In today’s increasingly complex and dynamic business

environment (‘Today’s Complex Project Environments’section), predicting success is multifaceted. Typically, itincludes not only technical, but also financial, marketing,social, legal, ethical, and technological dimensions, manyof them fuzzy or unknown at the time of the proposedproject evaluation (Thamhain 2011a,b). Hence, the DNAof success is highly complex, and outcomes are difficultto predict. Evaluating and selecting projects is not onlyan art and a science, but has to go beyond a simple cost-benefit analysis for most cases. To be effective, projectopportunities must be analyzed relative to their potentialvalue, strength, and importance to the enterprise. Wehave to understand and examine the whole spectrum ofcosts, risks, and benefits as part of the evaluation process,far beyond conventional project economics. This requiresa comprehensive approach with sophisticated leadership,integrating organizational resources and facilitating ashared vision of risks across organizational borders, time,and space (Baker 2012; Bstieler 2005; Cicmil et al. 2006).Currently, we are quite efficient on the analytical side ofproject evaluation/selection. However, we are still weak indealing with the hidden, less obvious dimensions of pre-dicting success that involve a broader spectrum of projectperformance variables, connecting to the enterprise and itssocio-economic environment (Thamhain and Skelton2007). These are the issues explored in the study reportedin this paper.

Today’s complex project environmentsOur complex, fast-changing business environment re-quires sophisticated decision making for selecting theright project with the most desirable outcome assuringsuccess and enterprise performance. Companies areunder pressures for quicker, cheaper, and smarter solu-tions. They are globally networked and can leveragetheir resources and accelerate their schedules by formingalliances, consortia, and partnerships with other firms,universities, and government agencies, which range fromsimple cooperative agreements to open innovation, aconcept of scouting for new product and service ideas,anywhere in the world. Projects are complex in natureand imbedded in lots of technology. All of these issuescontribute to the complexities and uncertainties of pro-ject selection which is a great challenge for managersin virtually every segment of industry and government.

They include computer, pharmaceutical, automotive,health care, transportation, and financial businesses, justto name a few. New technologies, especially computersand communications, have radically changed the work-place and transformed our global economy, with focus oneffectiveness, value, and speed. These techniques offermore sophisticated capabilities for cross-functional inte-gration, resources mobility, market responsiveness, andmanagerial decision support, but they also require moresophisticated skills both technically and socially, dealingeffectively with higher levels of conflict, change, risks, anduncertainty. These challenges resulted in a shift in man-agerial focus from functional efficiency to effectivenesswith attention to organizational interfaces, human factors,and overall enterprise performance.

What we know about project evaluation andselectionManagerial decision support for project selection hasbeen known for a long time (Remer et al. 1993). Today,managers have available a large array of tools and tech-niques for project evaluation and selection (NationalScience Foundation NSF 2010) that can be grouped intothree principle classes which are briefly summarized inAppendix 1 of this paper:

1. Primarily quantitative and rational approaches.2. Primarily qualitative and intuitive approaches.3. Mixed approaches, combining both quantitative and

qualitative methods.

While in the past, decisions on project selection fo-cused on quantitative approaches, such as ROI, paybackperiods, and net present value for determining desirabil-ity and potential success, today’s managers take a morebalanced approach between quantitative and qualitativemethods. They cast a much wider net for capturing abroad spectrum of variable that go far beyond the scopeand limitations of traditional financial evaluations, suchas ROI or payback (Kumar 2006).Most managers are keenly aware of the intricate con-

nections of success variables among organizational sys-tems and processes, which often limit the effectivenessof analytical methods. Especially for complex projectsand business processes, managers argue that no singleperson or group within an enterprise has all the smartsand insight for assessing these multi-variable influencesand their cascading effects (Shakhsi-Niaei et al. 2011).Further, no analytical model seems sophisticated enoughto represent the complexities and dynamics of all factorsthat might affect success or failure of a major project(Kavadias and Loch 2004; Zhang et al. 2009; Loch et al. 2001).These managers realize that while analytical methods pro-vide a critically important toolset for project evaluation and


Table 1 Summary of field sample statistics

Project environment Metrics

Total sample population 425

Companies 18

Business units 25

Projects (product developments) 105

Project teams 43

Team membersa 397

Product managers 12

R&D managers 5

Senior managers and directors 11

Average project budget US$28 million

Average project life cycle 18 monthsaTeam = total sample minus product managers, R&D managers and seniormanagers.


selection, these methods also take the collective thinkingand collaboration of all the stakeholders and key personnelof the enterprise and its partners to identify and deal withthe complexity of risks in today’s business environment.As a result, an increasing number of organizations arecomplementing their analytical methods with managerialjudgment and collective stakeholder experiences that in-clude such broad measures as strategic desirability andprojections from past performances (Hadad et al. 2012),hence moving beyond a narrow dependence on just ana-lytical models. In addition, many companies have devel-oped their own ‘systems,’ uniquely designed for dealingwith uncertainties in their specific projects and enterpriseenvironment (Henriksen and Traynor 2002; Kavadias andLoch 2004; Kumar 2006; Larson and Gray 2011). Thesesystems emphasize the integration of various tools, oftencombining quantitative and qualitative methods to cast awider net for capturing and assessing risk factors beyondthe boundaries of conventional methods. Examples arewell-known management tools, such as review meetings,Delphi processes, brainstorming, and focus groups, whichhave been skillfully integrated with analytical methods toleverage their effectiveness and improve their reliability.In addition, a broad spectrum of new and sophisticated toolsand techniques, such as user-centered design (UCD), voiceof the customer (VoC), and phase-gate processes, evolvedwhich rely by and large on organizational collaboration andcollective judgment processes to deal with the broadspectrum of risk variables that are dynamically distributedthroughout the enterprise and its external environment.

The missing linkWhile project evaluation and selection methods havebeen studied extensively for several decades (Brenner1994; Cook and Green 2000; Mantel et al. 2011), rela-tively little has been published on the role of collabor-ation across the total enterprise for evaluating potentialproject value (Oral et al. 1991). That is, we know littleabout management processes that involve the broaderproject community in a collective cross-functional wayfor dealing with project selection. The missing link is thepeople side as a cross-functional tool for collectivelyevaluating a project proposal toward potential success,one of the areas that are being examined in this paper.

MethodAn exploratory field study format is used to investigatemanagerial approaches for evaluating and selecting projectproposals. Specifically, this field study looks at both quanti-tative and qualitative models, and reports on their use andeffectiveness for various project types and situations.The work presented in this paper is a continuation of

my ongoing research into project management alignmentwith enterprise strategy (Shenhar et al. 2007) and risk

management in complex project situations (Thamhainand Skelton 2007; Thamhain 2011a,b). The field study re-ported here focuses especially on data collected between2008 and 2012.

DataThe unit of analysis used in this study is the project.Data were captured from 25 technology organizationsbetween 2008 and 2012, as part of an ongoing researchin the area of technology-oriented product developmentand team-based project management. The field studyyielded data from 43 project teams with a total samplepopulation of 425 professionals such as engineers, sci-entists, and technicians, plus their managers, includ-ing 10 supervisors, 53 project team leaders, 12 productmanagers, 5 directors of R&D, 4 directors of marketing,and 11 general management executives at the vice presi-dential level or higher. Together, the data covered morethan 100 projects in 18 companies, as summarized inTable 1. The projects involved mostly high-technologyproduct/service/process developments with budgets aver-aging US$28 million each. All project teams saw them-selves working in a high-technology environment. The18 host companies are large technology-based multi-national companies of the FORTUNE 500 category.The data were obtained from three sources, question-naires, participant observation, and in-depth retrospectiveinterviewing, as discussed in the previous section. Spe-cifically, in stage three, 138 interviews were held withteam leaders, line managers, product managers, mar-keting directors, and general management executives.These discussions provided interesting and usefulinsight into the cross-functional issues and challengesinvolved in project evaluation and selection processesin complex business environment. Content analysishas been used in addition to standard statisticalmethods for evaluating the survey data.



Justification for the exploratory field study formatAll components of this investigation, such as projectmanagement, product development, team work, decisionmaking under uncertainty, technology, and business en-vironment, involve highly complex sets of intricately re-lated variables. Researchers have consistently pointed atthe non-linear, often random nature of these processesthat involve many facets of the organization, its mem-bers, and the environment (Bstieler 2005; Danneels andKleinschmidt 2001; MacCormack, Nellore and Balachandra2001; Thamhain 2008, 2009; Verganti and Buganza 2005).Investigating these organizational processes simultan-eously is not a simple task, making it unlikely to findsimple models appropriate for researching these envi-ronments. Because of these complexities and the stillevolving nature of these components, their theoriesand constructs, an exploratory field research formatwas chosen for the investigation which uses question-naires in addition to two qualitative methods: participantobservation and in-depth retrospective interviewing.

Defining project value, performance, and successAll of these measures are multifaceted. Typically, theyinclude not only technical, but also financial, marketing,social, legal, and ethical dimensions. For most projects,the DNA of success is highly complex, and outcomes aredifficult to predict, especially long term. Special atten-tion must be given to the assessment of these variables,with their many facets and reference points throughoutthe project lifecycle and beyond. Especially, projectsuccess is highly judgmental and difficult to determineat the outset, but often only years after project com-pletion, as we can see in aerospace, pharmaceutical,automobile, infrastructure or government service pro-jects. Yet, in spite of these challenges, senior man-agers, collectively, seem to have a good sense of thepotential success and value of a proposed project. Wevalidated this fact in previous studies by testing theagreement among senior managers from various partsand different levels of the enterprise regarding theirjudgment of project performance (Kruglianskas andThamhain 2000; Thamhain 2005, 2006). These agree-ments were measured via Kruskal-Walles analysis of vari-ance by ranks which provided evidence of the strongagreement among the managers on the degree of successexpected for a given project under evaluation. Therefore, Iused this finding as a basis for arguing the benefits of col-lective multifunctional judgment in selecting candidateprojects, a discussion presented later in this paper. Yet,the topic of ‘measuring success’ and its underlying metricsis an area that needs additional study. Suggestions for fu-ture research include the relationship between short-termproject performance and long-term project success, andunderlying conditions and evaluation criteria that might

be useful for predicting future project performance andsuccess.

ResultsBased on the findings from this field study, an overviewis first given on how companies evaluate and select com-plex projects, followed by specific recommendations foreffectively managing the process. The first segment in-cludes a discussion on the effectiveness of some of themost popular quantitative and qualitative methods avail-able to managers, and an assessment of the situationalvalue of quantitative versus qualitative methods.

How do companies evaluate and select projectsBased on the findings from this field study, supported byadditional observations on hundreds of projects duringmy action research, a framework for effective projectevaluation and selection is summarized first and thenfollowed by specific recommendations for effectivelymanaging the process. While any decision model needsto be fine tuned toward specific application, the basicframework and management philosophy of evaluatingand selecting project proposals seem to be appropriatefor most projects, in spite of their differences in com-plexities, technologies, organizational structure, and cul-ture among companies.

Evaluation dimensions and decision phasesProject opportunities must be analyzed relative to theirpotential value, strength, and importance to the enter-prise. Four major dimensions should be considered forproject evaluation and compared to available alternativesas graphically shown in Figure 1: (1) added value of thenew project, consistent with the organizational objec-tives; (2) resource requests such as cost, personnel, andfacilities needed to complete the new project; (3) readi-ness and ability of the enterprise to execute the project;and (4) managerial belief and desire. A well-organizedproject evaluation and selection process provides theframework for systematic data gathering and informeddecision making toward resource allocation. Typically,the decision process can be broken into phases whichare often overlapping and executed concurrently, suchas shown in Figure 1 and described below:

Phase I. Deciding initial feasibility: screening andfiltering, quick decision on the viability of an emergingproject for further evaluation;Phase II. Deciding strategic value to enterprise:identifying alternatives and options to the proposedproject;Phase III. Deciding detailed feasibility and value:determining the specific value and chances of successfor a proposed project;


Φ

Φ

Φ

Φ

Figure 1 Project selection decision model.

Table 2 Typical criteria considered for project evaluationand selection

The criteria relevant to the evaluation and selection of a particularproject depend on the specific project type (i.e., product, serviceor process), business situation, industry, and market. Typically,evaluation procedures include the following criteria and measures:

● Cash flow, revenue, and profit ● Project cost

● Consistency with business plan ● Project duration (PLC)

● Cost-benefit ● Resource availability

● Impact on other businessactivities

● Return on investment

● Market share ● Risk

● Organizational readiness andstrength

● Sales volume

● Product lifecycle (deliverables) ● Strategic value

● Project business follow on ● Technical complexity andfeasibility

Each criterion is based on a complex set of parameters, variables,and assumptions.


Phase IV. Deciding project go/no-go: committingresources for a project implementation.

While the process seems simple, logical, and straight-forward, developing meaningful support data is a com-plex undertaking. It is also expensive, time consuming, andoften highly eclectic. Typically, decision making requiresthe following inputs: (1) specific resource requirements;(2) specific implementation risks; (3) specific benefits,i.e., economics, technology, markets, etc.; (4) bench-marking and comparative analysis of threats and oppor-tunities; and (5) strategic perspective, including long- andshort-term value assessment.Estimating cost, schedules, risks, and benefits, such as

those shown in Table 2, is always challenging. However,these estimates are relatively straightforward in compari-son to predicting project success. The difficulty is in de-fining a meaningful aggregate indicator for project valueand success. Methods for determining success rangefrom purely intuitive to highly analytical. No method isseen as truly reliable in predicting success, especially forcomplex and technologically intensive projects. Yet,some companies have a better track record in selecting‘winning’ projects than others. They seem to have the abil-ity to create a more integrated picture of the potentialbenefits, costs, and risks for the proposed project relativeto the company’s strength and strategic objectives.

Methods of project evaluation and selectionProducing a comprehensive picture of future value thatintegrates the large array of potential costs and benefitsis both a science and an art. During field interviews,managers confirm the changing paradigm of projectevaluation that has been reported in the literature for

some time (Baker 2012; Kavadias and Loch 2004; Kumar2006; Shakhsi-Niaei et al. 2011). That is, not too longago, in the 1990s and before, rational evaluationprocesses prevailed by and large for supporting pro-ject selections. However, in today’s more complex and dy-namic business environment, managers point out thatpurely rational analytical processes apply only to a limitednumber of situations. Most of our technologically complexbusiness scenarios require the integration of both analyticaland judgmental techniques to capture the broad spectrumof variables affecting project success or failure, necessaryfor predicting success and making the best choice.



Yet, in spite of the shift toward including judgmentalmethods more specifically into the selection process, sys-tematic information gathering and standardized methodsare at the heart of any project evaluation process and pro-vide the best assurance for reliably predicting project out-come and repeatability of the decision process. In thesample of 105 projects of my field study, managers reportthe following breakdown of approaches used for projectevaluation and selection:

� Primarily quantitative and rational approaches: usedfor 18% of all projects.

� Primarily qualitative and intuitive approaches: usedfor 24% of all projects.

� Mixed approaches (quantitative/qualitativecombination): used for 58% of all projects.

Given the fact that projects of the field sample con-sisted of complex, largely high-technology projects, itmight look surprising that a relatively large number ofproject evaluations are based on either primarily quanti-tative or qualitative methods. That is, one might expectmore mixed approaches. Follow-up interviews provideadditional insight. Of the sample’s projects, 20% were verylarge undertakings of over US$100 million each, many ofthem defense contracts or alike. While these programevaluations included considerable qualitative inputs andjudgment from focus groups, industry experts and man-agement, the final go decision was strongly influenced byquantifiable data and highly rational methods of decisionmaking. As a result, more than half of these ‘very large’projects were judged as being selected by ‘primarily quan-titative and rational approaches’. Similarly, we found spe-cial situations in the ‘primarily qualitative and intuitive’category. Approximately one half of the projects in thiscategory were of R&D or exploratory nature with verylimited quantifiable data, especially in the area of futurevalue (i.e., future sales, revenue, market share, etc.).While all of these projects went through detailed tech-nical feasibility, budgeting, and project planning processes,the final ‘go decision’ was based on primarily qualitativeand intuitive approaches for approximately 60% of theseprojects.However, regardless of the primary nature (quantitative,

qualitative, or combined focus), all project evaluations in-cluded some quantitative and qualitative assessments.Equally important, we observed highly interactive interdis-ciplinary effort among the various resource groups of theenterprise and its partners during the evaluation and se-lection processes for all projects, regardless of their type,size, and classification. That is, while management mayjudge the process leading to the final go/no-go decision as‘primarily’ one of the three approaches, all decisions in-cluded both quantitative and qualitative inputs to some

degree. Often, many meetings were needed even beforethe formal project evaluation process started, just to gaina basic picture of potential benefits, costs, and risks in-volved, and to determine the type of data needed. All ofthese activities occurred regardless of primary evaluation/selection focus.Yet, in all situations, there is the risk of relying too

heavily on quantitative data. This was pointed out espe-cially by senior managers. Most concerned were thosemanagers and project leaders who lived through a disap-pointing project rollout or delivery, or outright projectfailure, after careful feasibility study and selection proce-dures at the front end. These managers observed inretrospect that many of their front-end decision pro-cesses relied too much on analytical data and modelswith assumptions that did not hold or forecasts thatproved unreliable, while not casting a wide enough netfor scanning the business environment. They felt thatmore experiential information had to be captured fromsenior management and technology, market and usercommunities, to complement and validate the analyticalmodels. The issues and potential remedies to these chal-lenges will be explored in the next section.

Going beyond simple formulas and quantitative methodsWhile quantitative methods of project evaluation havethe benefit of producing relatively quickly a measure ofmerit for simple comparison and ranking, they also havemany limitations, as summarized in Table 3. Yet, in spiteof the limitations inherent to quantitative evaluation andthe increased use of qualitative approaches, virtually,every organization supports its project selections withsome form of quantitative measures; the most popular areROI, cost-benefit, and payback period (see Appendix 1 and 2).However, driven by the growing complexity of the businessenvironment, quantitative decision models are becomingless capable of capturing the full spectrum of variables asso-ciated with project success or failure. As pointed out byone senior manager in our field study, “Numbers like favor-able ROI and projected revenue are great selling points to-ward project approval. However, often I’m not comfortablewith the data. The numbers don’t always seem to reflectmarket realities, customer reactions, changing technologiesand dozens of other uncertainties and assumptions. Brain-storming the issues and defining critical success factorswith experts from the areas critical, and getting inputs fromuser groups, often gives us a better sense of potential suc-cess or failure and how to plan for success that a coldnumber that is based on a complex financial model orprojection that we don’t really trust”. These types of issuesand inherent limitations of analytical methods concernmanagers across all industries, who started increasingly toaugment quantitative methods and to explore alternatives.As observed in this field study and my own project work,


Table 3 Comparison of quantitative and qualitative approaches to project evaluation

Quantitative methods Qualitative methods

Benefits Benefits

Clear and simple comparison, ranking, and selection Search for meaningful evaluation metrics

Repeatable process Broad-based organizational involvement

Encourages data gathering and measurability Understanding of problems, benefits, opportunities

Benchmarking opportunities Problem solving as part of selection process

Programmable Broadly distributed knowledge base

Useful input to sensitivity analysis and simulation Multiple solutions and alternatives

Connectable to many analytical and statistical models Multifunctional involvement leading to buy-in and risk sharing

Limitations Limitations

Many success factors are not quantifiable Complex, time-consuming process

Probabilities and weights may change Biases introduced via organizational power and politics

True measures do not exist Difficult to procedurize or repeat

Analyses and conclusions are often misleading Conflict and disagreement over decision/outcome

Masking of hidden problems and opportunities Does not fit conventional decision processes

Stifle innovative decision making Intuition and emotion may obscure facts

Lack people involvement, buy-in, commitment Used for justifying ‘wants’

Ineffective in dealing with multifunctional issues, non-linearitiesand dynamic situations

Lead to more fact finding than decision making

May mask hidden costs and benefits Temptation for unnecessary expansion of fact finding

Temptation for acting too quickly and prematurely Process requires effective managerial leadership


and as a management consultant to hundreds of compan-ies, there is an increasing trend for companies to supple-ment quantitative results with additional information fordetermining future cost-benefits and project success.Many of these contemporary decision making methodsrely to a large degree on qualitative, judgmental decisionmaking as summarized in Appendix 1. These data gather-ing methods cast a wider net and consider a broaderspectrum of factors than those methods limited to quanti-tative measure. They can include fuzzy variables and deci-sion parameters such as related to strategy and businessethics that may be difficult to describe or quantify, but areimportant in gaining overall perspective and a more com-prehensive picture on potential benefits, risk, and chal-lenges of the proposed project.

Discussion and recommendationsEffective evaluation and selection of project opportun-ities involve many variables of the organizational andtechnological environment, reaching often far beyondcost and revenue measures. While economic modelsprovide an important dimension of the project selectionprocess, most situations are too complex to use simplequantitative methods as the sole basis for decisionmaking. Many of today’s project evaluation proceduresinclude a broad spectrum of variables and rely on acombination of rational and intuitive processes for defin-ing the value of a new project venture to the enterprise.

The better organizations understand their business pro-cesses, markets, customers, and technologies, the betterthey will be able to evaluate the value, risks, and chal-lenges of a new project venture. Further, manageability ofthe evaluation process is critical to its results, especially incomplex situations. The process must have a basic struc-ture, discipline, and measurability to be conducive to theintricate multivariable analysis. One method of achievingstructure and manageability calls for grouping the evalu-ation variables into four categories: (1) consistency andstrength of the project with the business mission, strategy,and plan; (2) multifunctional ability to produce the projectdeliverables and objectives, including technical, cost, andtime factors; (3) success in the customer environment;and (4) economics, including profitability. Modern phasemanagement, such as Stage-Gate® processes provide man-agers with the tools for organizing and conducting projectevaluations in a systematic way. The following sectionsummarizes suggestions that can help managers in effect-ively evaluating and selecting projects toward successfulimplementation.

Seek out relevant informationMeaningful project evaluations require relevant qualityinformation. The four sets of variables related to thestrategy, results, customer, and economics, as identifiedabove, can provide a framework for establishing theproper metrics and detailed data gathering.



Ensure competence and relevancyEnsure that the right people become involved in the datacollection and judgmental processes.

Take top-down look first, detail comes laterDetail is less important than information relevancy andevaluator expertise. Do not get hung-up on missing dataduring the early phases of the project evaluation. Evalu-ation processes should be iterative. It does not makesense to spend a lot of time and resources on gatheringperfect data to justify a ‘no-go’ decision.

Select and match the right peopleWhether the project evaluation consists of a simple eco-nomic analysis or a complex multifunctional assessment,competent people from functions critical to the overallsuccess of the project should be involved.

Define success criteriaWhether deciding on a single project or choosing amongalternatives, evaluation criteria must be defined. Theycan be quantitative, such as ROI, or qualitative, such asthe chances of winning a contract. In either case, theseevaluation criteria should cover the true spectrum of fac-tors affecting success and failure of the project(s). Thesuccess criteria should be identified by seasoned enterprisepersonnel. In addition, people from outside of the com-pany, such as vendors, subcontractors, and customers, areoften included in this expert group and critical to the de-velopment of meaningful success criteria.

Strictly quantitative criteria can be misleadingBe aware of evaluation procedures based on quantitativecriteria only (ROI, cost, market share, MARR, etc.). Theinput data used to calculate these criteria are likelybased on rough estimates and are often unreliable.Furthermore, a reliance on strictly quantitative dataconsiders only a narrow spectrum of factors affectingproject success or failure, thus ignoring many otherimportant factors, especially those that influence pro-ject success in a dynamic or non-linear way, typicalfor many complex technologically sophisticated under-takings. Evaluations based on predominately quantita-tive criteria should at least be augmented with someexpert judgment as a ‘sanity check’.

Condense criteria listCombine evaluation criteria, especially among the judg-mental categories, to keep the list manageable. As a goal,try to stay within the 12 criteria for each category.

Gain broad perspectiveThe inputs to the project selection process should in-clude the broadest possible spectrum of data from the

business environment that affect success, failure, andlimitations of the new project opportunity. Assumptionsshould be carefully examined.

Communicate across the enterpriseFacilitate communications among evaluators and func-tional support groups. Define the process for organizingthe team and conducting the evaluation and selectionprocess.

Ensure cross-functional representation and cooperationPeople on the evaluation team must share a strategic vi-sion across organizational lines. They also must have thedesire to support the project if selected for implementa-tion. The purpose, goals, objectives, and relationships ofthe project to the business mission should be clear to allparties involved in the evaluation/selection process.

Do not lose the big pictureAs discussions go into detail during the evaluation, theteam should maintain a broad perspective. Two globaljudgment factors can help focus on the big picture ofproject success: (1) overall cost-benefit perspective and(2) overall risk of failure assessment. These factors canbe recorded on a ten-point scale, −5 to +5. This alsoleads to an effective two-dimensional graphic display forcomparing competing project proposals.

Do your homework between iterationsProject evaluations are usually conducted progressivelyin iterative cycles. Therefore, the need for more informa-tion, clarification, and further analysis surfaces betweeneach cycle. Necessary action items should be properlyassigned and followed up to enhance the evaluationquality with each consecutive iteration.

Take a project-oriented approachPlan, organize, and manage your project evaluation/selection process as a project. Proposal evaluation andselection processes require valuable resources that mustbe justified and carefully managed.

Resource availability and timingDo not forget to include in your selection criteria theavailability and timing of resources. Many otherwise suc-cessful projects fail because they cannot be completedwithin a required time period.

Use red-team reviewsSet up a special review team of senior personnel. This isespecially useful for large and complex projects withmajor impact on overall business performance. This re-view team examines the decision parameters, qualitativemeasures, and assumption used in the evaluation process.



Limitations, biases, and misinterpretations that may other-wise remain hidden can often be identified and dealt with.

Stimulate creativity and candorSenior management should foster an innovative risk-shared ambience for the evaluation team. Especially, theevaluation of complex project situations involves intri-cate sets of variables. Criteria for success and failure arelinked among many subsystems, such as organization,technology, and business, associated with a great deal ofrisks and uncertainty. Innovative approaches are re-quired to evaluate the true potential of success for theseprojects. Risk sharing by senior management, recogni-tion, visibility, and a favorable image in terms of high pri-ority, interesting work, and importance of the project tothe organization have been found strong drivers towardattracting and holding quality people on the evaluationteam and toward gaining their active and innovative par-ticipation in the process.

Manage and leadThe evaluation team should be chaired by someone whohas the trust, respect, and leadership credibility with theteam members. Senior management can positively influ-ence the work environment and the process by providingguidelines, charters, visibility, resources, and active sup-port to the project evaluation team.

ConclusionsIn summary, effective project evaluation and selectionrequires a broad scanning process across all segments ofthe enterprise and its environment to deal with the risks,uncertainties, ambiguities, and imperfections of dataavailable for assessing the value of a new project venturerelative to other opportunities. No single set of broadguidelines exists that guarantees the selection of success-ful projects. However, the process is not random! A bet-ter understanding of the organizational dynamics thataffects project performance and the factors that drivecost, revenue, and other benefits can help in gaining abetter, more meaningful insight into the future value ofa prospective new project. Seeking out both quantitativeand qualitative measures incorporated into a combinedrational judgmental evaluation process often yields themost reliable predictor of future project value and de-sirability. Equally important, the process requiresmanagerial leadership and skills in planning, organiz-ing, and communicating. Above all, the leader of theproject evaluation team must be a social architect whocan unify the multifunctional process and its people. Theleader must be able to foster an environment, profession-ally stimulating and conducive to risk sharing. It also mustbe effectively linked to the functional support groupsneeded for project implementation. Finally, organizational

strategy must be aligned and integrated with theevaluation/selection process, early and throughout itsevaluation cycle. Senior management has an import-ant role in unifying the evaluation team behind themission objectives and in facilitating the linkages tothe stakeholders and ultimate user community. Seniormanagement should further help in providing overallleadership and in building mutual trust, respect, andcredibility among the members of the proposal evaluationteam, all critical drivers toward a strong partnership of allteam members and the basis for an effective enterprise-wide decision-making system. Taken together, this is theenvironment conducive for cross-functional communica-tion, cooperation, and integration of the intricate variablesneeded for effective evaluation and selection of projectproposals in complex business environments.

AppendicesAppendix 1Summary of project evaluation and selectiontechniquesSome of the popular project evaluation and selectiontools, techniques, and approaches are summarized inthis Appendix, grouped into three classes:

1. Primarily quantitative and rational approaches.2. Primarily qualitative and intuitive approaches.3. Mixed approaches, combining both quantitative and

qualitative methods.

Quantitative approaches to project evaluationand selectionQuantitative approaches are often favored to supportproject evaluation and selections if the decisions requireeconomic justification. They are also commonly used tosupport judgment-based project selections. One of thefeatures of quantitative approaches is the generation ofnumeric measures for simple and effective comparison,ranking, and selection. These approaches also help es-tablish quantifiable norms and standards and lead to re-peatable processes. Yet, the ultimate usefulness of thesemethods depends on the assumption that the decisionparameters, such as cash flow, risks, and the underlyingeconomic, social, political, and market factors, can actu-ally be quantified and reliably estimated over the projectlife cycle. Therefore, quantitative techniques are effectiveand powerful decision support tools, if meaningful esti-mates of cost-benefits, such as capital expenditures andfuture revenues, can be obtained and converted into netpresent values for comparison. Because of their import-ance, quantitative methods have been discussed in theliterature extensively, ranging from simple return on in-vestment (ROI) calculations to elaborate simulations ofproject scenarios. Many companies eventually developed



their own project evaluation/selection models, custom-ized to their specific needs. However, the backbone formost of these customized models is a set of economic/financial measures which tries to determine the cost-benefit of the proposed venture, usually for some pointin the future. Specifically, five measures are especiallypopular:

1. Net present value (NPV)2. ROI3. Cost-benefit (CB)4. Payback period (PBP)5. Pacifico and Sobelman project ratings

The calculation and application of these measures toproject evaluation/selection will be illustrated by caseexamples. Specifically, four project proposals (describedin Table 4) will be evaluated in this chapter, using theabove measures. The results are summarized in Table 5.

Net present value comparisonThis method uses discounted cash flow as the basis forcomparing the relative merit of alternative project op-portunities. It assumes that all investment costs and rev-enues are known and that economic analysis is a validbasis for project selection. We can determine the NPV ofa single revenue, or stream of future revenues, or costs ex-pected in the future. Two types of presentations are com-mon: (1) present worth (PW) and (2) net present value.

Present worthThis is the single revenue or cost (also called annuity A)which occurs at the end of a period n, subject to theprevailing interest rate i. Depending on the manage-ment philosophy and enterprise policies, this interest ratecan be (1) the internal rate of return (IRR) realized by the

Table 4 Description of four project proposals

Projectproposal

Description

Projectoption P1

Management does not accept any new project proposal.Hence, neither investment capital is required nor is anyrevenue generated.

Projectoption P2

This opportunity requires a US$1,000 investment at thebeginning of the first year and generates a US$200revenue at the end of each of the following 5 years.

Projectoption P3

This opportunity requires a US$2,000 investment at thebeginning of the first year and generates a variablestream of net revenues at the end of each of the next5 years as follows: US$1,500, US$1,000, US$800, US$900,and US$1,200.

Projectoption P4

This opportunity requires a US$5,000 investment at thebeginning of the first year and generates a variablestream of net revenues at the end of each of the next5 years as follows: US$1,000, US$1,500, US$2,000, US$3,000, and US$4,000.

company on similar investments or (2) the minimumattractive rate of return (MARR) acceptable to companymanagement, or the prevailing discount rate. The presentworth is calculated as

PWðAj i; nÞ ¼ PWn ¼ A 11þ ið Þn:

For the examples used in this chapter, we consider theIRR (defined as the average return realized on similar in-vestments) to be the prevailing interest rate.

Net present valueThe net present value is defined as a series of revenuesor costs, An, over N periods of time at a prevailing inter-est rate i:

NPVðAnji;NÞ ¼XNn¼1

An1

ð1þ iÞn ¼XNn¼1

PWn:

Three special cases exist for the net present value cal-culation: (1) for a uniform series of revenues or costs overN periods, NPV(An|i, N) =A[(1 + i)N − 1]/i(1 + i)N; (2) foran annuity or interest rate i approaching zero, NPV =A ×N; and (3) for the revenue or cost series to continueforever, NPV =A/i. Table 5 applies these formulas to thefour project alternatives described in Table 4, showingthe most favorable 5-year net present value of US$3,192for project option P3.

Return on investment comparisonPerhaps one of the most popular measures for projectevaluation is the ROI:

ROI ¼ Revenue Rð Þ−Cost Cð ÞInvestment Ið Þ :

ROI calculates the ratio of net revenue over invest-ment. In its simplest form, the stream of cash flow is notdiscounted. One can look at the revenue on a year-by-year basis, relative to the initial investment. For example,project option 1 in Table 3 would produce a 20% ROIeach year, while project option 2 would produce a 75%ROI during the first year, 50% during the second year,and so on. In a somewhat more sophisticated way, wecan calculate the average ROI per year over a given rev-enue cycle as shown in Table 3:

�ROIðAn; InjNÞ ¼XNn¼1

RevenueRð Þn− CostCð ÞnInvestmentIð Þn

" #= N½ �:

We can then compare the average ROI to the MARR.Given a MARR of 10% for our project environment, allthree project options P1, P2, and P3 compare favorable,with project P3 yielding the highest average return oninvestment of 54%. Although this is a popular measure,


Table 5 Cash flow and net value calculations of four project options or proposals

Given cash flow

Do-nothing option (P1) Project option (P2) Project option (P3) Project option (P4)

End of year

0 0 −1,000 −2,000 −5,000

1 0 200 1,500 1,000

2 0 200 1,000 1,500

3 0 200 800 2,000

4 0 200 900 3,000

5 0 200 1,200 4,000

Calculations

Net cash flow (Σ · P) 0 0 +3,400 +6,500

Net present value at the endof year 5 (NPV|N = 5)

0 −242 +2,153 +3,192

Net present value for revenueto continue ∞ (NPV|N = ∞)

0 +1,000 +9,904 +28,030

Average annual return on investment (ROI|N = 5) 0 20% 54% 46%

Cost Benefit (CB = ROINPV|N = 5) 0 76% 108% 164%

Payback period for MARR = 10% (NPBP|i = 10) 0 8 1.8 3.8

Payback Period for MARR = 0% (NPBP|i = 0) 0 5 1.5 3.3

MARR of i = 10% is assumed. Given for all four project proposals, (1) a single investment is being made at the beginning of the project life cycle (e.g., at the endof year 0), and (2) the internal rate of return (IRR) or the minimum attractive rate of return (MARR) is 10%.


it does not permit a meaningful comparative analysisof alternative projects with fluctuating costs and reve-nues. Furthermore, it does not consider the time value ofmoney.

Cost-benefitAlternatively, we can calculate the net present value ofthe total ROI over the project lifecycle. This measure,known as cost-benefit (CB) is calculated as the presentvalue stream of net revenues divided by the presentvalue stream of investments. It is an effective measurefor comparing project alternatives with fluctuating cashflows:

CB ¼ ROINPVðAn; Inji;NÞ ¼

XNn¼1

NPV Anji;Nð Þ" #

XNn¼1

NPV In i;Njð Þ" # :

In our example of four project options (Table 3), pro-ject proposal P4 produces the highest cost-benefit of164% under the given assumption of i =MARR = 10%.

Payback period comparisonAnother popular figure of merit for comparing projectalternatives is the PBP. It indicates the time period ofnet revenues required to return the capital investment

made on the project. For simplicity, undiscounted cashflows are often used to calculate a quick figure for com-parison, which is quite meaningful if we deal with an ini-tial investment and a steady stream of net revenue.However, for fluctuating revenue and/or cost steams, thenet present value must be calculated for each periodindividually and cumulatively added up to the ‘break-even point’ in time, NPBP, when the net present valueof revenue equals the investment. Mathematically,

NPBP occurs whenXNn¼1

NPVðAnjiÞ≥XNn¼1

NPV In iÞ:jð

In our example of four project options (Table 3), pro-ject proposal P3 produces the shortest, most favorablepayback period of 1.8 years under the given assumptionof i =MARR = 10%.

Pacifico and Sobelman project ratingsThe previously discussed methods of evaluating projectsrely heavily on the assumption that technical and com-mercial success is assured, and all costs and revenuesare predicable. Because these assumptions do not alwayshold, many companies have developed their own specialprocedures and formulas for comparing project alterna-tives. Two examples illustrate this special category ofproject evaluation metrics.



The project rating factorThis measure was originally developed by Carl Pacificofor assessing chemical products and predicting commer-cial success:

PR ¼ pT � pC � RTC

:

Pacifico’s formula is in essence an ROI calculation ad-justed for risk. It includes probability of technical suc-cess (0.1 < pT < 1.0), probability of commercial success(0.1 < pC < 1.0), total net revenue over project lifecycle(R), and total capital investment for product develop-ment, manufacturing set-up, marketing, and relatedoverheads (TC).

Product development figure of meritThe formula developed by Sobelman

z ¼ P � TLCð Þ− C � TDð Þ

represents a modified cost-benefit measure which takesinto account both the development time and commerciallifecycle of the product. It also includes average profitper year (P), estimated product lifecycle (TLC), averagedevelopment cost per year (C), and years of development(TD).

Qualitative approaches to project evaluation andselectionWhile quantitative methods provide an important tool-set for project proposal evaluation and selection, there isalso a growing sense of frustration, especially amongmanagers of complex and technologically advanced un-dertakings, that reliance on strictly quantitative methodsdoes not always produce the most useful or reliable inputsfor decision-making, nor are all methods equally suitedfor all situations. Therefore, it is not surprising that forproject evaluations involving complex sets of business cri-teria, narrowly focused quantitative methods are oftensupplemented with broad scanning, intuitive processes,and collective, multifunctional decision making such asDelphi, nominal group technology, brainstorming, focusgroups, sensitivity analysis, benchmarking, and UCD. Eachof these techniques can either be used by itself to deter-mine the best, most successful, or most valuable option, orthese techniques are integrated into a comprehensive ana-lytical framework for collective multifunctional decisionmaking, which is being discussed next.

Collective, multifunctional evaluationsThis process relies on subject experts from various func-tional areas for collectively defining and evaluating broadproject success criteria, employing both quantitativeand qualitative methods. The first step is to define the

specific organizational areas critical to project successand to assign expert evaluators. For example, a product orservice development project may typically include organi-zations such as R&D, engineering, testing, manufacturing,marketing, product assurance, and customer/field services.The function experts should be given the time and re-sources necessary for the evaluation. They also shouldhave the commitment from senior management for fullorganizational support. Ideally, these evaluators should bemembers of the core team ultimately responsible for pro-ject implementation.

Evaluation factorsEarly in the evaluation process, the team defines the fac-tors which appear critical to the ultimate success of theprojects under evaluation and arranges them into a listwhich includes both quantitative and qualitative factors.A mutually acceptable scale must be worked out for scor-ing the evaluation criteria. Studies of collective multi-functional assessment practices show that simple scalesare most effective for leading to actionable team decisions.The four most popular and robust scales for judging situ-ational outcomes are shown below:

1. Ten-point judgment scale: This scale ranges from+5 (most favorable) to − 5 (most unfavorable).

2. Three-point judgment scale: +1 (favorable),0 (neutral or cannot judge), −1 (unfavorable).

3. Five-point judgment scale: A (highly favorable),B (favorable), C (marginally favorable),D (most likely unfavorable), F (definitely unfavorable).

4. Five-point Likert scale: 1 (strongly agree), 2 (agree),3 (neutral), 4 (disagree), 5 (strongly disagree).

Weighing of criteria is not recommended for most ap-plications as it complicates and often distorts the collect-ive evaluation. Perspective and judgment are part of thestrength and value of these qualitative methods, whichcan be lost by forcing too much of a quantitative frame-work on the qualitative evaluation.

The evaluation processEvaluators first assess and then score all of the successfactors they feel qualified to judge. Then, collective dis-cussions follow. Initial discussions of project alternatives,their markets, business opportunities, and technologiesinvolved are usually beneficial, but not necessary for thefirst round of the evaluation process. The objective ofthis first round of expert judgments is to get calibrated onthe opportunities and challenges presented. Further, eachevaluator has the opportunity to recommend (1) actionsthat could improve the quality and accuracy of the projectevaluation, (2) additional data needed, and (3) suggestionsfor increasing project success. Before meeting at the next



group session, agreed-on action items and activities forimproving the decision process should be completed. Theevaluation process is enhanced with each iteration by pro-ducing more accurate, refined, and comprehensive data.Typically, between 3 and 5 iterations are required before ago/no-go decision can be reached for a given project.

Mixed approaches, combining both quantitativeand qualitative methodsMixed approaches are the most common method ofevaluating and selecting projects in today’s complexbusiness environment. Virtually all evaluations of projectproposals include some form of quantitative and qualita-tive methods. However, to qualify as a mixed approach,the project evaluation and selection process has to con-tain a fairly balanced array of both classes of analyticaland judgmental tools and techniques of the various typesdiscussed above.

Appendix 2The following is a summary description of terms, vari-ables, and abbreviations used in this chapter:

� Cross-functional involves actions which spanorganizational boundaries.

� Phase management involves breaking of projects intonatural implementation phases, such as development,production, and marketing, as a basis for projectplanning, integration, and control. Phase managementalso provides the framework for concurrent engineeringand Stage-Gate® processes.

� Project success is a comprehensive measure,defined in both quantitative and qualitative termswhich includes economic, market, and strategicobjectives.

� Stage-Gate® process is a framework originallydeveloped by R Cooper and S Edgett for executingprojects within predefined stages with measurabledeliverables (at gates) at the end of each stage. Thesegates also provide the review metrics for ensuringsuccessful transition and integration of the projectinto the next stage.

� Weighing of criteria is a a multiplier associatedwith specific evaluation criteria.

� Annuity (A) is the present worth of a revenue orcost at the end of a period n.

� Cost benefit (CB) is the net present value of allROIs in dollars.

� Prevailing interest rate (i).� Investment (I).� Internal rate of return (IRR) is the average return

on investment realized by a firm on its investmentcapital.

� Minimum attractive rate of return (MARR) on newinvestments acceptable to an organization.

� Net present value (NPV) of a stream of futurerevenues or costs.

� Payback period (PBP) is the time period needed torecover the original investment.

� Project rating factor (PR) is a measure developed byCarlo Pacifico for predicting project success.

� Present worth (PW, also called annuity) is thepresent value of a revenue or cost at the endof a period n.

� Return on investment (ROI)� Project rating factor (z) is a measure developed by

Sobelman for predicting project success.

Competing interestsThis author declares that he has no competing interests.

Authors’ informationDr. Hans Thamhain is a Professor of Management and Director of MOT andProject Management Programs at Bentley University, Boston/Waltham. Dr.Thamhain, held management positions with Verizon, General Electric and ITT,has written over seventy research papers and six professional referencebooks. He received the IEEE Engineering Manager Award in 2001, PMI’sDistinguished Contribution Award in 1998 and PMI’s Research AchievementAward in 2006. He is profiled in Marquis Who's Who in America and certifiedas NPDP and PMP.

Received: 10 October 2012 Accepted: 13 November 2013Published:

ReferencesBaker NR (2012) R&D project selection models: an assessment. R&D Manag

5(1):105–111Brenner M (1994) Practical R&D project prioritization. Res Technol Manag

37(5):38–42Bstieler L (2005) The moderating effects of environmental uncertainty on new

product development and time efficiency. J Prod Innov Manag 23(3):267–284Cicmil S, Williams T, Thomas J, Hodgson D (2006) Rethinking project

management: researching the actuality of projects. Int J Proj Manag24(8):675–686

Cook WD, Green RH (2000) Project prioritization: a resource-constrained dataenvelopment analysis approach. Socioecon Plann Sci 34(2):85–99

Danneels E, Kleinschmidt EJ (2001) Product innovativeness from the firm’sperspective. J Prod Innov Manag 18(6):357–374

El Emam K, Koru A (2008) A replicated survey of it software project failures.Software (IEEE) 25(5):84–90

Gulla S (2012) Seven reasons why IT projects fail. IBM Systems Magazine.http://www.ibmsystemsmag.com/mainframe/tipstechniques/applicationdevelopment/project_pitfalls. Last accessed 22 Nov 2013

Hadad Y, Keren B, Laslo Z (2012) A decision-making support system module forproject manager selection according to past performance. Int J Proj Manag31(4):532–541

Henriksen AD, Traynor AJ (2002) A practical R&D project-selection scoring tool.IEEE Trans Eng Manag 46(2):158–170

Kavadias S, Loch CH (2004) Project selection under uncertainty: dynamicallyallocating resources to maximize value. Kluwer, Norwood, MA, USA

Kruglianskas I, Thamhain H (2000) Managing technology-based projects inmultinational environments. IEEE Trans Eng Manag 47(1):55–64

Kumar PD (2006) Integrated project evaluation and selection using multiple-attribute decision-making technique. Int J Prod Econ 103(1):87

Larson E, Gray C (2011) “Organization strategy and project selection,” Chapter 2 inProject Management: The Management Process. McGraw-Hill, New York, pp 22–63

Lemon WF, Bowitz J, Burn J, Hackney R (2002) Information systems project failure:a comparative study of two countries. J Glob Inf Manag 10(2):28–39

26 Nov 2013

http://www.ibmsystemsmag.com/mainframe/tipstechniques/applicationdevelopment/project_pitfalls

http://www.ibmsystemsmag.com/mainframe/tipstechniques/applicationdevelopment/project_pitfalls



Loch CH, Pich MT, Terwiesch C, Urbschat M (2001) Selecting R&D projects atBMW: a case study of adopting mathematical programming models.IEEE Trans Eng Manag 48(1):70–80

Mantel S, Meredith J, Shafer S, Sutton M (2011) “Selecting projects to meetorganizational objectives,” Chapter Section 1.5 in Project ManagementPractice. Wiley, Hoboken, NJ, USA, pp 10–22

National Science Foundation (NSF) (2010) The 2010 user-friendly handbook forproject evaluation. Division of Research and Learning in Formal and InformalSettings. National Science Foundation, Arlington, VA, USA

Oral M, Kettani O, Lang P (1991) A methodology for collective evaluation andselection of industrial R&D projects. Manag Sci 37(7):871–881

Remer DS, Re Stokdyk SB, Van Driel M (1993) Survey of project evaluationtechniques currently used in industry. Int J Prod Econ 32(1):103–115

Shakhsi-Niaei M, Torabi S, Iranmanesh S (2011) A comprehensive framework forproject selection problem under uncertainty and real-world constraints.Compt Ind Eng 61(1):226–237

Shenhar A, Milosevic D, Dvir D, Thamhain H (2007) Linking project managementto business strategy. Project Management Institute (PMI) Press, Newtown,PA, USA

Shore B (2008) Systematic biases and culture in project failures. Proj Manag J39(4):5–16

Standish Group (2013) Chaos Tuesday. http://blog.standishgroup.com/. Lastaccessed 22 Nov 2013

Thamhain H (2005) Team leadership effectiveness in technology-based projectenvironments. IEEE Eng Manag Rev 33(2):11–25

Thamhain H (2006) Optimizing innovative performance of R&D teams intechnology-based environments. Creativity Res J 18(4):435–436

Thamhain H (2008) Team leadership effectiveness in technology-based projectenvironments. IEEE Eng Manag Rev 36(1):165–180

Thamhain H (2009) Leadership lessons from managing technology-intensiveteams. Int J Innov Technol Manag 6(2):117–133

Thamhain H (2011a) Critical success factors for managing technology-intensiveteams in the global enterprise. Eng Manag J 23(2):30–36

Thamhain H (2011b) Evaluating and selecting technology-based projects. In:Engineering measurements encyclopedia. Edited by Kutz M. Wiley, New York

Thamhain H, Skelton T (2007) Success factors for effective R&D risk management.Int J Technol Intell Plann (IJTIP) 3(4):376–386

Verganti R, Buganza T (2005) Design inertia: designing for life-cycle flexibility ininternet-based services. J Prod Innov Manag 22(3):223–237

Zhang D, Zhang J, Lai KK, Lu Y (2009) An novel approach to supplier selectionbased on vague sets group decision. Expert Syst Appl 36(5):9557–9563

Cite this article as: Thamhain: Contemporary methods for evaluatingcomplex project proposals. Journal of Industrial EngineeringInternational

10.1186/2251-712X-9-34

2013, 9:34

Submit your manuscript to a journal and benefi t from:

7 Convenient online submission

7 Rigorous peer review

7 Immediate publication on acceptance

7 Open access: articles freely available online

7 High visibility within the fi eld

7 Retaining the copyright to your article

Submit your next manuscript at 7 springeropen.com

http://blog.standishgroup.com/


Documents

ORIGINAL RESEARCH Open Access Contemporary methods for ... · ORIGINAL RESEARCH Open Access Contemporary methods for evaluating complex project proposals Hans J Thamhain Abstract