Embed Size (px)
Citation preview
Electronic copy available at: https://ssrn.com/abstract=3064353
1
Designing Leadership Development Programs for High Impact in Emerging Economies:
The case of Spanish Multinationals in Latin America
Author Details (please list these in the order they should appear in the published article)
Author 1 Name: Dr. Camelia Ilie
University/Institution: INCAE Business School
Country: Costa Rica
Author 2 Name: Dr. Guillermo Cardoza
University/Institution: INCAE Business School
Country: Costa Rica
Author 3 Name: Dr. Schon Beechler
University/Institution: INSEAD
Country: France
Author 4 Name: Dr. Jaume Hugas
University/Institution: ESADE Business School
Country: Spain
Corresponding author: Dr. Camelia Ilie
Corresponding Author’s Email: [email protected]
Electronic copy available at: https://ssrn.com/abstract=3064353
2
Abstract
Traditional learning methods for executive education have been based primarily on lectures
and case studies. However, these can no longer provide the type of management development
training that companies need to compete effectively in volatile, uncertain and complex
business environments. Providers of Leadership Development Programs (LDPs) need to
ensure that both the knowledge transmitted and the competencies developed align with
organizations´ purpose, strategies, and cultures. This paper aims to link the key elements of
LDPs and their impact on the participants and their organizations, by identifying the relation
between the LDPs elements and key learning, behavior and performance indicators. Thus, we
propose a new conceptual model that links the program with its intrinsic impact (participants’
attitudes, behavior and knowledge) and its extrinsic impact (organizational indicators of
promotion, retention, and role changes following the program). In addition, we analyze the
model’s hypothesis by conducting quantitative research based on data collected from Spanish
multinational corporations with activities in Latin America.
The results of our research can contribute significant insights to LDP design, development
and evaluation that can provide tangible measures of their impact on organizational
indicators. These findings can be valuable as a measure of the strategic roles of all human
resources and talent development areas—especially in emerging economies, where
companies’ human resources and talent development departments are not yet viewed as
strategic partners for top executives.
Keywords: Leadership Development Programs; Talent Development; Executive
Education; Adult Learning; Emerging Economies; Impact of Training; Latin America
3
Designing Leadership Development Programs for High Impact in Emerging Economies:
The case of Spanish Multinationals in Latin America
"It is in changing that things find purpose."
Heraclitus
INTRODUCTION
The shortage of management and technical staff with specialized skills poses significant
challenges to multinational corporations' (MNCs) ability to do business in complex, uncertain
and dynamic environments characterized by widespread globalization, emerging knowledge-
based industries, disruptive technological change and keen competition (Cardoza, 2001;
Dalakoura, 2010; Grossman, Salas, Pavlas & Rosen, 2013; Gurdjian, Halbeisen & Lane,
2014).
According to the Corporate Learning Factbook (2014), spending on corporate training grew
by more than $130 Billion worldwide. This report mentions that 60 percent of all companies
indicate that the leadership gap is their top business challenge, and, therefore, they invest
35% of their entire training budget (35 cents of every training dollar) on LDPs. Also, a recent
survey by Deloitte (2016) reports that 89 percent of executives rated organizational
leadership as an important priority, and more than half of the respondents mentioned that
their organizations are not ready to meet leadership needs.
Though these studies reveal that HR managers agree that executive training is vital to
developing leadership and increasing employee commitment and productivity, critics such as
Beer et al. (2016) argue that learning does not have the expected impact because participants
in executive programs return very quickly to older ways of working and making decisions. In
4
their opinion, participants in LDPs, once they return to their jobs, very often cannot apply
new knowledge and tools because the culture and organizational systems remain the same.
Gurdjian et al. (2014) also observe that LDPs tend to fail in the first place because they
overlook the context, decouple reflection from real work, underestimate mindsets, and fail to
measure results. Also, budgetary restrictions frequently force firms to manage corporate
spending on leadership development more efficiently. Moreover, firms are asked to
demonstrate the added value and impact of investments in executive education on achieving
corporate objectives and goals (Dragoni, Tesluk, Russell & Oh, 2009; Eiter & Halperin,
2010; Gurdjian et al., 2014; Kirkpatrick & Kirkpatrick, 2009; Mattioli, 2009).
Furthermore, measuring the impact of LDPs in an organization is challenging because these
programs are intended to produce results in the long term and to prepare talent to assume
leadership responsibilities in the future (Perez, 2014). In fact, seeing the effect of LDPs on
job performance is improbable in the short term. On the other hand, it is hard to isolate the
effect of training on a leader’s development and unrealistic to expect that an executive would
become a great leader merely by attending LDPs (Day, 2014). Even though, as Charlton and
Osterweil (2005) argue, "measuring ROI is like seeking the Holy Grail," a reliable LDP
methodology is needed to ensure a better return from growing corporate investments in
executive training.
In response to these requests, during recent years, LDPs have shifted from a focus on
teaching best management practices to an emphasis on training managers to be leaders,
capable of successfully designing and implementing business strategies through unified
leadership styles and corporate cultures (Hugas, 2007; Ilie, 2009). To do so, LDPs are also
5
gradually incorporating new active and experiential learning approaches, including team-
based computer simulations, corporate case studies, role-playing, 360-degree feedback,
executive coaching and mentoring, high-involvement experiences, action learning projects,
and demonstration-based learning. Similarly, LDPs’ curricula are increasingly covering
subjects associated closely with globalization, sustainable development, global trade, change
management, corporate social responsibility, ethical and responsible leadership, cross-cultural
management, and diversity management (Akins, 2013; Dalakoura, 2010; Day, 2000; De
Meyer, 2012; Grossman et al., 2013; Klimoski and Amos, 2012; Magni et al., 2013; Quigley,
2013; Schneider et al., 2010; Smith, 2013; Strebel and Keyes, 2005; Thomas, 2013). To
incorporate the necessary complexity to make program design effective, LDPs are also
expected to correctly align development plans and career paths with corporate strategic
interests (Chen, Chang & Yeh, 2004; Cranfield-ESADE Report, 2008). They are also
expected to be linked to the identification of high-potential employees (Corporate University
Xchange, 2007) and the design of corporate succession plans (Cohn, Khurana & Reeves,
2007). Additionally, to enhance the transfer of learning and its impact, the LDPs should
create the appropriate organizational climate and manage the learning process for successful
organizational change (Yorks, Beechler & Ciporen, 2007).
Prior research on leadership development focuses mainly on developing comprehensive
frameworks that include factors such as mindsets, attributes, personal growth, leadership
roles, team skills, and best practices to train leaders (Cacioppe, 1998; Conger & Benjamin,
1999). The literature also identifies the sequence of processes of instructional systems
development (ISD) in a training method (Molenda, 2003). More recently, leadership
education studies have begun to focus on the factors and the context that determine
leadership development (Dalakoura, 2010); the development of multilevel evaluation
6
frameworks (Clarke, 2012); the use of different pedagogical approaches, categories and
instructional features to improve leadership training and ensure the transfer of learning
(Conger, 2013; Grossman et al., 2013; Klimoski & Amos, 2012; Jenkins, 2013; Scott &
Webber, 2008); and leader competencies and their impact on business performance (Tompson
& Tompson, 2013).
Although the studies on leadership development have contributed to the better understanding
of multiple factors that affect the development of leaders, as well as to the identification of
the best pedagogical approaches, to achieve an even greater impact, LDP design should take
more into account the business context and the barriers to organizational change. Moreover,
in addition to ignoring, to some degree, the challenges facing organizations and their
industries, leadership programs fail to maximize their impact because they lack a link
between program methodologies and their impact on organizational indicators. The present
study moves in this direction by proposing a comprehensive and dynamic model for
developing efficient methods for the LDPs. This model offers educators and practitioners a
practical tool for linking program design and methodologies with organizational indicators.
Another important factor is that the majority of research on leadership and LDPs has been
conducted for organizations in developed countries, and, therefore, the adoption of these
methodologies in emerging economies faces limitations due to differences in economic,
cultural, leadership and management capabilities. With the aim of helping to fill these gaps
in the literature, the present study uses data collected from a focus group and quantitative
research with Spanish MNC executives responsible for leadership development in their Latin
American operations. The purpose of this article is to review the contributions in the field of
LDPs, to identify key elements and to propose a process-based model to increase the impact
7
of LPDs on business results, and to suggest potential future research. Building on previous
research on LDPs, this study implements an analysis of managers' perceptions of LDPs’
impact on corporate performance. It also introduces the concept of the Virtuous Circle of
LDPs as an integrative platform to identify and manage the critical factors in designing and
delivering these kinds of programs.
The paper is structured as follows. The first section provides a general overview of the main
scholarly contributions to the theory of LDPs. To simplify and organize the analysis, we
structure this section around the different value chain stages of the programs: diagnostic &
design; methods & delivery; and follow-up & evaluation. The second section proposes an
integrated model that links different aspects of LDPs with organizational impacts. The third
section presents the research hypothesis related to different stages and components of the
LDPs and the key factors that potentially increase their impact and discusses the research
results. The final section presents the conclusions of the study, the main findings and
recommendations.
LITERATURE REVIEW
A comprehensive review reveals that the literature questions the role of business schools in
training leaders and managers (Crossan et al., 2013). The literature also indicates an ongoing
debate among academics around the effectiveness of executive education programs in
developing leadership and teaching sound management practices. For instance, Pfeffer and
Fong (2002: 80) contend that there is no evidence that the training offered in business schools
improves students’ careers and that the research conducted by academicians lacks relevance
and, therefore, has little impact on organizations. Other authors also criticize the effectiveness
8
of the context of classroom-based management education (Allio, 2005; Doh, 2003; Pettigrew
et. al, 2016; Raelin, 1993). Allio (2005) argues that traditional leadership programs promote
leadership literacy but not leadership competency. Some authors go even further, claiming
that business schools are becoming more and more irrelevant since they are teaching the
wrong content with the wrong methods to the wrong students (Bennis & O´Toole, 2005;
Mintzberg, 2005).
Researchers also claim the existence of a disconnection between theory and practice, or
between research and practice, in business schools (Pfeffer and Fong, 2002; Raelin, 2007).
This concern about the missing linkages between theory and practice is not new, and critics
continue to ask for both rigor and relevance. For instance, observing this problem, Raelin
(2007) advocates developing a theory of practice using both theory—to challenge
management assumptions—and experience—by using real problems that participants
encounter in their jobs. In his words, “We need to move beyond the acquisition of formal
logic to reasoning and sense making that is concurrent with ongoing practice” Raelin (2007:
513). All of these critics emphasize the need to provide context-relevant learning experiences
linked to day-to-day organizational challenges to ensure relevant and impactful results.
Another interesting discussion going on in business schools relates to leadership training.
One of the main points of these discussions relates to what methods would be most
appropriate and efficient in teaching leadership (Booth, 2010; Doh, 2003; Snook, Nohria &
Khurana, 2013). This discussion is not new; for instance, Nevins and Stumpf (1999) claim
that traditional teaching and learning approaches in management education fail to link
knowledge, skills, and concepts to leadership practice within organizations. In particular, they
believe that these methods fail to provide lifelike situations and practice opportunities for
9
skill development, and they suggest developing contexts favoring active and experiential
learning styles.
To address the shortcomings of current executive education methods—in particular, the
problems of relevance and effectiveness; the need to link training to real learner's roles and
performance; and company challenges—a more integrative approach to executive education
may be helpful. This study proposes the use of an LDP's Virtuous Circle, a process-based
model that links the main steps in developing leadership and teaching sound management
practice with real organizational challenges and opportunities. Deconstructing the executive
education process into its primary value chain phases may have two important benefits: it
may help to integrate and manage the pre-program, program, and post-program LDPs'
activities; and it may introduce and evaluate experiential learning approaches that ensure the
efficient transfer of knowledge and skills that benefit the organization's goals.
A VIRTUOUS CIRCLE OF LEADERSHIP DEVELOPMENT: AN INTEGRATED,
SYSTEMIC AND DYNAMIC APPROACH
Considering that leadership development is a dynamic and longitudinal process and that
LDPs must fulfill certain needs and expectations to maximize their impact, leadership
development should be integrated and managed as a Virtuous Circle combining three
interrelated and consecutive stages.
Stage 1 is the pre-program: training needs assessment, program co-design, and preparation of
participants and organizations for learning and change. Stage 2 is the program: program
delivery and learning methodologies. Stage 3 is the post-program: follow-up and evaluation.
The Virtuous Circle of LDPs (Figure 1) should be built on a learning platform that integrates
all the steps and activities involved in the value-added chain of leadership development. As
10
presented in Figure 1, this approach facilitates the management of the whole process, from
training needs identification through the transfer of knowledge and applications, including
the expected impact. The following section uses this model to build research arguments and
to formulate the hypotheses.
Figure 1. Virtuous Circle of Leadership Development
Phase 1. Pre-Program: Diagnostic & Design
During the first phase of the LDP value chain, program developers must perform a training
needs assessment to create an inventory of the organization's existing knowledge and skills.
They must also gather relevant data for identifying the necessary training requirements to
develop the talent base that the organization needs to meet its goals. As Kirkpatrick and
Kirkpatrick (2009) state, LDPs are designed to meet organizational objectives and goals;
11
therefore, executive education providers should clearly define the knowledge and behaviors
(skills and attitudes) necessary to achieve the desired results.
As McClelland (1993) explains, the diagnostic phase is critical in allocating financial
resources for training and leadership development so that organizations can clearly define
“where they will have the greatest impact[,] thereby providing a positive return on the firm's
investment." To this end, McClelland (1993) presents a systematic and applications-oriented
process to determining and analyzing training needs and proposes four types of training needs
assessment (TNA) and data-gathering methods: survey questionnaires, individual interviews,
focus groups, and on-site observations. To better identify and address organizational training
needs, McClelland (1993) recommends using a combination of these types of data-gathering
and analysis methods.
Learning theory (Kolb, 1984; Kolb & Kolb, 2005) is another approach that provides a highly
effective diagnostic tool for identifying managers' development needs and the most
appropriate content and methodologies for a given group of participants. According to this
theory, executive education providers should take into account the four-stage learning
cycle—experiencing, reflecting, thinking and acting (Kolb, 1984)—and the different learning
and leadership styles (Kolb, 1984; De Vries & Korotov, 2007). At the same time, executive
programs should offer opportunities to question priorities and beliefs and to experiment with
different learning and leadership styles (De Vries & Korotov, 2007).
In addition, learning methods should be fairly broad and diverse so that the participants can
develop themselves intellectually, emotionally and practically, and, thus, the LDPs can
increase their impact on corporate challenges (Adler, Hooijberg, Kemanian, Keys, Lane &
12
Strebel, 2005; Eiter & Halperin, 2010; Ilie, 2009; Jenkins, 2013; Strebel & Keyes, 2005;
Yorks et al., 2007). For instance, to ensure the transfer of learning, Conger (2013) proposes
that LDPs include learning experiences that use different pedagogical approaches, such as
conceptual frameworks, skill-building experiences, feedback, and reflection on personal
growth, including participants’ emotions and values.
In fact, to add to the complexity of effective executive training design, developers also need
to focus on emotional processes to ensure that they positively impact personal, organizational
and social change. Other researchers also argue that emotional intelligence can help to
enhance team performance and positively impact financial performance (Goleman, Boyatzis
& McKee, 2001; Goleman, Boyatzis & McKee, 2002; McEnrue, Groves & Shen, 2007;
Mirvis, 2008; Siegling, Nielsen & Petrides, 2014). They mention that high levels of
emotional intelligence “create climates in which information sharing, trust, healthy risk-
taking, and learning flourish” and advise that LDPs start with self-knowledge tools and then
move on to the knowledge and understanding of others. In a similar vein, to develop socially
responsible behavior in leaders, addressing personal traits and values is necessary to build
social awareness and social consciousness (Schneider et al., 2010; Smith, 2013).
Phase 2. Program: Learning Methods & Delivery
Recent studies on executive education trends reveal a growing demand for more-customized
programs based on specific knowledge of the participants’ own organizations. Involving
managers, faculty, and facilitators during the pre-program activities permits them to better
appreciate organizational dynamics, to understand strategic challenges, and to participate
13
actively in the design and delivery process (Boiney & Mallinger, 2002; Lloyd & Newkirk,
2011; Strebel & Keys, 2005; Weldon E., 2012).
The use of real case studies based on company data and questionnaires related to business
priorities are highly recommended for developing high-impact LDPs. To ensure
transferability and use of knowledge and learning, Dover, Perkins and Wylie (2009)
recommend using customized cases since they offer a framework in which to articulate the
different education program modules, provide a historical perspective, and offer shared
knowledge for the discussions.
To ensure that action-learning-based LDPs align with corporate needs, program developers
should involve companies' senior leaders when designing the programs since they can
provide a broad outline of a company's strategy and the precise nature of the problems and
challenges their businesses are facing. Involving top managers in the early stages of diagnosis
and design facilitates the understanding of what is applicable and relevant to the company. It
also enables a better selection of participants and better team composition; assignment of
companies' internal sponsors and mentors; identification of strategic challenges; and, most
importantly, preparation of the conditions for the transfer of knowledge and verification of its
application (Tushman, O’Reilly, Fenollosa, Kleinbaum & McGrath, 2007).
However, considering that case studies only partially meet participants’ needs, LDPs should
also incorporate experience-learning methods to increase participants’ understanding and
management of personal and organizational change. As Lombardo and Eichinger (2001)
explain, the learning rule of thumb for managers is 70-20-10: managers usually acquire 70%
14
of their new knowledge and practices from experiential (on-the-job) learning, 20% from
coaching and mentoring activities, and 10% from formal education programs.
Authors and practitioners recommend organizational practices and learning tools such as
simulations, narrations, reflexive diagnostics, experience sharing, debates in small groups,
peer or group coaching, outdoor “ropes-type” activities, 360-degree feedback, job
assignments, executive coaching and mentoring, among others (Doh & Stumpf, 2007;
Tushman et al., 2007; Buchel & Antunes, 2007; Day et al., 2014). In particular, several
authors highlight the importance of action-learning projects to enhance participants’ learning
and ensure knowledge and skills transfer to the work place (Buchel & Antunes, 2007; De Déa
Roglio & Light, 2009; Raelin, 2007; Thorsell et al., 2012; Tushman et al., 2007).
Since the knowledge acquired in classrooms is disconnected from the workplace, several
authors find experiential learning based on real-time work problems a powerful approach to
leadership development programs (Buchel & Antunes, 2007; Conger, 2013; De Déa Roglio &
Light, 2009; Fulford, 2013; Phoocharoon, 2013; Raelin, 2007; Thorsell et al., 2012; Tushman
et al., 2007; Yulmartin & Riyanto, 2013). Raelin (2007: 513), in particular, calls for an
“epistemology that transforms learning from the acquisition of the objective rules of wisdom
to one that appreciates the wisdom of learning in the midst of action itself.” He argues that
action learning is a pedagogical approach that “seeks to generate learning from human
interaction arising from engagement in the solution of real-world work problems” Raelin
(2007: 511).
Similarly, building on Schön´s (1983) concept of the “reflective practitioner,” De Déa Roglio
and Light (2009: 169) propose that executive training programs should integrate experiential
15
learning that allows participants to transform their capabilities in real time. Along the same
line of reasoning, Yulmartin and Riyanto (2013: 499) argue that action learning facilitates
development at three levels—personal, teams and organization—and, thus, can become a
valuable tool for organizational learning strategy.
In fact, considering that learning designs based on current situations have enormous appeal
and utility in solving real problems and producing change, research and pedagogy should be
driven by action-learning projects related to the crucial issues and challenges of participants'
organizations (Boiney & Mallinger, 2002; Buchel & Antunes, 2007; Garratt, 2012; Tushman
et al., 2007; Yulmartin & Riyanto, 2013).
In conclusion, the above-cited investigations suggest the importance of considering three key
aspects—intellectual, emotional and experiential—in order to offer a transformational LDP
(DiStefano et al., 2005; De Vries & Korotov, 2007). Incorporating these three aspects into an
LDP will lead to the development of participating executives by impacting several intrinsic
capacities: their knowledge, attitudes and behavior.
In turn, the development of new knowledge, as well as a change in attitude and behavior,
should lead to better performance in the organization and to an impact on extrinsic aspects,
such as better retention, promotions, or changes in roles requiring different responsibilities.
Phase 3. Post-Program: Follow-up & Evaluation
Follow-up and Evaluation, the third part of the LDP's Virtuous Circle is a critical stage in the
training process, particularly since the recent financial crisis is forcing companies to manage
corporate spending on executive training more efficiently and to demonstrate how executive
16
education is contributing to the organization’s objectives and goals (Dragoni et al., 2009;
Eiter & Halperin, 2010; Kirkpatrick & Kirkpatrick, 2009; Mattioli, 2009; Hannum & Craig,
2010). Avolio et al. (2010) recommend that organizations manage investments in leadership
training as they would any other investment that they make in anticipation of a future benefit.
Models for evaluating programs have been designed from various perspectives. Assessment
models include participant satisfaction with the program, the transfer of learning to the
workplace, and training payback (Kirkpatrick, 1979; Kirkpatrick and Kirkpatrick, 2009;
McClelland, 1993). Indeed, the evaluation of training programs is the subject of many
studies, with both researchers and managers often using the multilevel model as their
benchmark (Kirkpatrick, 1979). This model is structured in four successive levels: (1)
satisfaction of trainees; (2) knowledge acquired; (3) behavioral change; and (4) impact on
organizational performance.
According to Dolliver (1994), the evaluation process should be split into three stages. The
first involves the use of "happy sheets" immediately after each session to measure
participants’ reaction to the program (Kirkpatrick's first level). The second stage is applied six
months after the training to measure the application of skills and knowledge acquired during
training (Kirkpatrick’s second and third levels). The third stage is the long-term evaluation,
measuring sustained changes in both group and individual results (level four in Kirkpatrick's
model).
Tejada and Ferrández (2007) argue that impact evaluation should be carried out at two levels:
1) individual, which measures learning and knowledge and skill transfer to the job position;
and 2) organizational, which measures the impact on the company’s results. Although the
17
final objective for any organization investing in training is to improve its managers'
performance, a direct relationship between results and the training program is hard to
determine. The effects of training on managers' performance are difficult to isolate because
jobs possess many intangible aspects. It is also important to highlight that Kirkpatrick-based
evaluation models usually involve increased cost and complexity and require more time and
greater analytical rigor as they progress through the successive levels (Winfrey, 1999). For
this reason, companies make little use of impact measurement of training programs on
organizational performance.
Defining evaluation objectives at the program-design stage (during the initial phase of
training needs assessment) is recommended to obtain good program assessments and to
achieve better results (Dolliver, 1994; Kirkpatrick & Kirkpatrick, 2009; Yorks et al., 2007).
This approach provides a practical tool for measuring both the knowledge acquired and the
behavioral changes (levels 3 and 4 in Kirkpatrick's model) and, therefore, also gives human
resources and learning and development managers the information they need to design
subsequent training programs for the same groups of participants. In this way, a more
integrated process (pre-program/program/post-program) can be adopted, leading to the
Virtuous Circle of LDPs.
The post-program follow-up to measure LDP’s success at the participant and the
organizational levels is an important part of the evaluation process. Buchel and Antunes
(2007: 410) suggest that executive education providers become much more involved in the
post-program stage by applying measures of the program’s outcomes in terms of behavioral
and cognitive transformations. Also, follow-up actions are keys to check whether the
decisions made during the program were efficiently implemented (Tushman et al., 2007).
18
In summary, treating evaluation as a process throughout all the value chain stages (diagnosis,
design, development, and final assessment) helps maximize the LDP’s impact by ensuring
that the program stays aligned with its objectives. Yorks, Beechler, and Ciporen (2007: 318)
argue that the incorporation of the program assessment as an integral part of the program
design is proof that the learning is the central focus and that that the program assessment
works as the link among the program design, delivery, and outcomes.
Figure 2 presents a model with various impact levels that depicts the different stages of LDPs
and their potential impact on the participants.
Figure 2. Relation between LDPs and Program Impact. Source Authors.
Two definitions of impact are also proposed as follows:
• Extrinsic impact: related to organizational aspects, as retention, promotion, changes in roles,
increased productivity or other better results.
Emotional training
Attitude
Retention
Intellectual training
Knowledge
Results
Experiential learning
Behaviour
Promotion
+
+
+
+
+
+
+
19
• Intrinsic impact: related to the personal development of participants in terms of knowledge,
attitude and behavior learning.
The following six research hypotheses are based on the arguments presented in this section:
HYPOTHESES RELATED TO THE EXTRINSIC IMPACT – RETENTION, PROMOTION,
PRODUCTIVITY (RESULTS)
H1. The impact of LDPs on the organization is directly correlated with the direct involvement
of top management in all stages of LDPs: diagnostic and design, implementation and
evaluation.
H2. The impact of LDPs on the organization is directly correlated with the inclusion of
strategic information in the program design.
H3. The impact of LDPs on the organization is directly correlated with needs assessment
surveys among potential participants during the diagnostic stage.
HYPOTHESES RELATED TO THE INTRINSIC IMPACT – CHANGE IN KNOWLEDGE,
ATTITUDE AND BEHAVIOR
H4. The impact of LDPs on the knowledge, attitudes and behaviors of the participants is
directly correlated with the use of intellectual, emotional and experiential methodologies
during the program.
H5. The impact of LDPs on the participants is directly correlated with the link between the
training program and their career development plans (promotion, retention or change of
roles).
H6. The changes in attitude and behavior are directly correlated with the involvement of top
management in the program and the use of personalized methodologies focused on individual
development, such as coaching or mentoring.
20
RESEARCH METHOD
We first wanted to confirm that the research question—what are the key elements of LDPs
that can enhance their impact on participants and organizational key performance
indicators—and the hypotheses are relevant, clear and well-formulated. To accomplish this,
we organized a focus group with experts.
Focus Group
The focus group relied on the participation of a mixed group of executive education clients of
a business school: Chief Learning Officers (CLOs), Human Resources Managers, and
Business Unit Managers from Spanish multinational corporations responsible for business
expansion in Latin America. The focus group was designed to identify the key factors that
maximize the impact of LDPs; to understand managers' concerns about the variables
analyzed; and to explore the impact of these programs on participants’ corporations. The
focus group was composed of ten directors from Spanish multinationals with activities in
Latin America, including Telefonica, Gas Natural, Union Fenosa, Grupo Santander, BBVA,
and Indra. Six of the directors were human resources and development experts; two were
from functional areas with executive development responsibilities; and two were this paper’s
authors, from the executive education sector, who also moderated the group.
As an information-gathering tool, the focus group is currently the most commonly used
instrument, especially when conducting "applied" research in the fields of marketing or
market studies, when analyzing the impact of mass media communications, and when
discussing relevant issues affecting specific human groups (Sandoval, 1996).
21
The focus group is the most recommended tool to conduct an exchange of experiences among
a group of people. The purpose is to foster discussion around a common topic, which leads to
the development of new ideas that would otherwise be very hard to develop (Hartman, 2004;
based on Morgan, 1988).
The recommendation for this interview technique, which can take between one and two
hours, is to form groups of six to ten people. Participants must be a homogeneous group of
people, with common occupations or interests, who will be interviewed by panel moderators.
Participants listen to everyone else's responses and may add comments to their answers. The
group does not need to achieve consensus.
The goal is to gather high-quality information in a setting in which people can consider their
points of view and compare or reconsider them, based on other people's perspectives (Quinn
Patton, 1987; Krueger, 1994). Topics to be discussed must be limited (focus), and the group
dynamics must encourage an almost ‘disorderly’ exchange of ideas, to generate new
dimensions for analysis (Hartman, 2004; based on Herndon, 2001).
To achieve the recommended dynamics in a focus group, researchers must rely on some tools
for gathering information in an organized way so that it can be used to analyze results. We
used two tools to develop the current focus group: the affinity diagram and the relationship
diagram.
During the first stage of the discussion group, when the exchange of ideas was going to be
more disorderly, the affinity diagram proved to be a useful tool to identify and structure an
issue when the situation was indeterminate. The affinity diagram involves collecting several
22
verbally-stated facts, opinions, or ideas and to synthesize them in a simple diagram, grouping
them by natural affinity. The procedure for the affinity diagram is as follows:
• Each opinion is written on a single card.
• Cards with similar opinions are grouped.
• A synthesis of cards expressing similar views on the same issue is written on a
separate card.
• Another summary is made of the second-tier cards into a single card. This single card
will hold the general idea of the shared views.
In the second stage of the focus group, once ideas were grouped using the affinity diagram,
we used the relationship diagram to help clarify causal relationships in the identified
problems. The relationship diagram, used in the second stage of the focus group, is useful for
systematizing work or ideas, and it provides an efficient pathway to choose the most
important factors from a broad mass. The procedure to elaborate a relationship diagram is as
follows:
• The issue is stated in writing.
• A list of probable causes of the problem is made.
• Next, group members identify an effect corresponding to each cause, which is written
down and circled.
• Causes are related to effects using arrows.
• When an effect is the reason for another effect, an arrow is drawn from the initial
effect-cause to the next corresponding effect.
Finally, the relationship diagram was used in the second stage of the focus group because, in
addition to organizing initially collected ideas, it is recommended for deeper understanding of
23
the issue and to formulate assumptions that will be verified later with a quantitative study
(Calduch, 1994). In the case of the present research, ideas collected during the focus group
(See Annex and Table 1 for a detailed explanation of the focus group organization and
findings) were used to improve questions in the quantitative questionnaire. Below is an
explanation of the design of the quantitative instrument and of the data-gathering process.
The remaining part of this paper presents an analysis of the quantitative study results, the
conclusions and future lines of investigation.
Quantitative research
The quantitative questionnaire was used to measure the main research question—that is, to
identify the key elements that increase LDPs’ impact on corporate strategies. The
questionnaires were formatted using Ashridge’s (International University Consortium for
Executive Education, UNICON, 2005) study as a model. A five-point Likert-type scale was
used, and managers who had taken part in the focus group validated the questionnaires for
content, clarity, thematic order, number of questions, and completion time. The survey of
human resources managers included 113 items, of which 56 were five-point Likert questions,
with response categories varying from “strongly disagree” to “strongly agree,” and 57 were
Yes/No questions. A shorter version of the questionnaire—sent to business unit managers—
consisted of 79 questions, 50 of which were five-point Likert scale questions, while 29 were
of the Yes/No type. The long and short versions of the questionnaire had 76 questions in
common.
The questionnaire was divided into five sections: 1) strategic aspects; 2) diagnostic and
design aspects; 3) content aspects; 4) evaluation aspects; and 5) questions covering the
respondent's profile. After the questionnaires were constructed, a website specializing in
24
online surveys hosted the two documents. A link to the site was sent by e-mail to a database
of contacts, and responses were also gathered online. It is important to mention that given the
lack of information on the training sector (economic data, market segmentation, training
proposals to companies and other strategic, confidential data), the empirical study was based
on business managers’ and human resource managers’ perceptions of training impact.
Sample definition
We were granted access to the corporate database of our business school for the quantitative
study. The database was segmented using the following key words: human resources
managers; training managers; leadership development managers; business area and business
unit managers; and Spanish multinationals in Latin America. The result was a database of
1,071 individuals who were asked to take part in the research. Of these, 107 responded: 61
human resources managers and 46 business unit managers. Eighty-six percent of participants
were from private companies. Although the response rate was only 10%, the respondents
were experts, and the aim of the study was to obtain the maximum number of answers to
meet the various criteria for sample size, which indicated a minimum sample size of 100
(Kline, 1994).
Regarding company size, 33.7% of participants came from companies with fewer than 500
workers; 30.8% were from companies with between 501 and 5,000 workers; and 32.7% were
from corporations with over 5,000 employees. The rest of participants did not indicate the
number of staff. A factor in the subsequent analysis was that the three groups were similar in
size in terms of the number of participants.
FINDINGS, CONCLUSIONS AND FUTURE RESEARCH
25
The objective of the analysis of the results was to identify the relationship among pre-
program assessments, senior executives' participation in the various LDP stages, and the
LDP's impact at the organizational level. For this purpose, we first analyzed the data by
applying t-tests, Pearson correlation coefficient tests, and standard deviation measures to
estimate the statistical significance of these relationships. Following this first analysis, we
applied regression models and present the analysis results in this section.
First, we found some empirical evidence supporting the first hypothesis, which proposed that
direct involvement of executives in different stages of the LDP is positively related to the
LDPs’ later impact on the company.
Concretely, involvement of top management in the selection of LDP participants showed to
be moderately and significantly correlated, both to the total impact of LDPs (r=0.381,
p<0.001) and to their intrinsic (r=0.299, p=0.002) and extrinsic impact (r=0.295, p=0.002).
Furthermore, committee participation in content also showed positive correlation with
extrinsic impact (r=0.213, p=0.033).
However, this empirical evidence is not as conclusive as one might have expected. For
instance, the expected significant association between the impact of LDPs and top
management involvement in diagnostics and design was not found; nor did we find a
significant association between impact and executives’ involvement in the program
evaluation, for that matter.
26
We found support for Hypothesis 2. Statistically significant differences in the mean values
were found when applying the t-test to estimate whether or not the use of information on
business strategy in the pre-program increased LDPs’ organizational impact (t(100)=2.34,
p=0.022). A significant correlation was also found between the strategic content of the
program (alignment of the content with the business strategy) and the program’s impact.
In terms of the Hypothesis 2, we found evidence showing that LDP effectiveness is related to
the inclusion of strategic data in the program design. In fact, inclusion of such strategic data
showed a low, but significant correlation with the extrinsic impact of the programs (r=0.198,
p<0.05).
Hypothesis 3 was also supported. A statistically significant correlation was found between the
use of information obtained from training needs assessments during the pre-program and the
LDP's impact on the organization (r=0.213, p=0.033). A variable that is also particularly
relevant to LDP development and success is linking information on business strategy to the
LDP design during the pre-program (the alignment of the program with the business
strategy).
As shown in Table 2, Hypothesis 4 was partially confirmed, as experiential learning methods
(action learning, case studies, workshops, mentoring and development of competencies) were
the most valued methodologies and proved to be statistically significant in predicting LDPs'
impact on the organization.
Contrary to our expectations, narrations—a subtler experiential method—was not rated as
highly as better-known experiential learning methods. Perhaps the novelty of this approach in
27
the field of LDPs could be the reason for the lower score, although additional data collection
is required to confirm this conjecture.
Table 2
Learning Methodologies. Source: Authors
Valid
answers
Minimum Maximum Average
Standard
deviation
Theory 103 2 5 3.67 .759
Simulators 102 1 5 4.20 .745
Roles 103 2 5 4.19 .768
Self-evaluation 101 1 5 4.20 .762
Case studies 103 2 5 4.38 .628
Development of
competencies
103 2 5 4.25 .637
Action learning 101 3 5 4.57 .517
Mentoring 102 2 5 4.27 .733
Workshops 102 3 5 4.33 .586
Related projects 102 2 5 4.21 .665
Graphics 102 1 5 3.53 .829
Outdoor training 102 1 5 3.58 .814
Managers 102 2 5 4.12 .775
Narrations 102 1 5 3.11 .757
Psychological tests 101 1 5 3.40 .928
Other managers 103 1 5 3.77 1.031
28
A positive correlation was also found between the impact of the LDPs and the degree to
which multiple aspects were evaluated throughout the LDPs’ value chain (r=0.322, p=0.012).
We found a greater impact when using the competencies evaluation to assess the results of
the program (t(58)=2.02, p=0.047) and when a questionnaire of organizational climate was
used before and after the program (t(58)=2.08, p=.042). Additional information related to the
impact of different kinds of evaluation done throughout the program appears in Table 3.
Table 3
Correlations between different types of evaluations and the impact
generated by the LDPs. Source: Authors.
Impact
Individual sessions Pearson Correlation ,304(*)
Sig. (two-tailed ) ,002
N 102
Self-evaluation Pearson Correlation ,023
Sig. (two-tailed) ,819
N 102
Self follow-up Pearson Correlation ,132
Sig. (two-tailed) ,185
N 102
Directive committee Pearson Correlation ,016
Sig. (two-tailed) ,872
N 100
*Correlation is significant at the 0,01 level (2-tailed)
Concerning Hypothesis 5, there was significant empirical evidence, although of low
magnitude, in favor of a correlation between the intrinsic dimension of LDPs and top
management’s influence on career plans (r=0.227). Moreover, extrinsic and intrinsic
dimensions of LDPs’ impact were significantly correlated with each other (r=0.210),
reinforcing the previous analysis in the theoretical model (See diagram 36, pg. 148).
29
With regard to Hypothesis 6, we observed evidence of a correlation between personalized
follow-up methodologies and intrinsic impact of LDPs. The correlation between the use of
mentoring and intrinsic impact was shown to be moderate and significant (r=0.291).
In order to further analyze the specific elements that affect the impact of the LDPs, we
applied a general regression model that proved to be significant (F(7,58)=5.25, p<.001). The
program impact was defined as a dependent variable and seven LDP descriptors (design, level
of selecting directors, content richness, methodological diversity, strategic character of the
program, level of evaluation, and program audit) as predictors.
Predictors account for 33.9% of variability. Table 4 presents significance values associated
with each variable, showing that level of selecting directors, content, strategic character of
the program, and level of evaluation are significant predictors of program impact.
Table 4
Value of LDPS descriptors as predictors of program impact. Source: Authors
Model
Non-standardized
coefficients
Standardized
coefficients t Sig.
B
Typical
error Beta B
Typical
error
1 (Constant) .965 .704 1.370 .177
Design -.120 .221 -.076 -.543 .590
Level of selecting
directors .715 .353 .252 2.025 .048
Content richness .313 .128 .315 2.446 .018
Methodological
diversity .086 .182 .062 .472 .639
Strategic character
of the program .261 .111 .353 2.340 .023
Level of
evaluation .487 .213 .278 2.286 .026
Program audit -.388 .311 -.176 -1.248 .218
30
Figure 3 shows the general prediction model schematically, as follows.
Figure 3. Prediction model for LDP descriptors and program impact
Source: Authors.
In order to ascertain whether core LDP features predict their general impact, we conducted a
multiple regression analysis using the Introducing method, with the seven LDP descriptive
variables as predictors and program impact of LDPs as the criterion. The model proved
significant (F(7,98)=5.4, p<0.001), accounting for 23.9% of the variability of program impact
(corrected R2=0.239).
Table 5 marks predictors that proved significant in boldface. Level of evaluation and the
number of directors involved in the selection were shown to be significant predictors of
program impact, while other variables did not reach significance.
PROGRAM IMPACT
Design Level of selecting directors Content richness Methodological diversity Strategic character of the program Level of evaluation Program audit
0.34
31
Table 5.
Multiple regression model Source: Authors.
Model
Non-standardized
coefficients
Standardize
d
coefficients
t Sig. B
Typical
error Beta
1 (Constant) -.059 .192 -.306 .760
Diagnostic and
design .007 .066 .011 .103 .918
Content .020 .047 .051 .416 .678
Method .064 .055 .130 1.176 .243
Strategic
character of the
program
-.003 .033 -.009 -.078 .938
Selectors .313 .101 .298 3.089 .003
Action Learning
Projects .001 .039 .003 .034 .973
Level of
evaluation .296 .086 .339 3.464 .001
Note. Dependent variable: program impact
On the other hand, a successive regression model, considering first the intrinsic impact and
then the extrinsic one, was applied to analyze whether the intrinsic impact was a mediator
between the LDP and the extrinsic impact. Once the effect of the intrinsic impact was
discounted, several LDP factors were still significantly related to the extrinsic impact
(F(7,50)=4.15, Change in R2=0.26).
Figure 4 shows a summary of the regression analysis, for both the intrinsic and extrinsic
impact.
Figure 4. Synthesis of the regression analysis. Source: Authors.
INTRINSIC IMPACT
Design Level of Selecting Directors Content Methodology Strategic Plan Evaluation level Auditing
0.21
EXTRINSIC IMPACT
0.35 (0.24)
0.28
32
From a general perspective, as predicted, many of the factors that make up the LDP system
have shown a link to program impact. In particular, two factors—the number of directors
involved in participant selection and the degree to which different aspects of the program are
evaluated—have shown significant and relevant correlations with impact of the programs,
both intrinsic and extrinsic.
Other factors have shown significant correlation with only a concrete type of impact. For
instance, rich content and diversity of the methodology used in LDPs have shown significant
correlation only with intrinsic impact. Once again, this finding strengthens the assumption of
the theoretical model (Figure 3), in which training has a direct impact on intrinsic variables.
Furthermore, an alignment is observed within the company between intrinsic (knowledge,
attitudes, and behavior) and extrinsic (results, promotion, and retention) variables. If
companies were to use stronger connecting mechanisms between these two types of
variables, the expectation is that impact would be higher.
The factor referred to as “the degree of evaluation of different aspects of LDPs” warrants
special comment because the number of intrinsic aspects evaluated correlates moderately to
highly with the intrinsic impact of the programs, while the number of extrinsic elements
assessed correlates, to the same magnitude, with extrinsic impact.
Empirically, this outcome seems to support the maxim that evaluation is needed for
improvement, so that the aspect evaluated is improved. Moreover, it has been shown that
33
intrinsic aspects tend to be evaluated more than extrinsic ones, which is consistent with
intrinsic impact being higher than the extrinsic impact generated.
On the other hand, some factors showed significant correlation only with extrinsic impact.
This was the case with the presence of a business plan and of the strategic character of LDPs,
which showed a correlation only with extrinsic impact. Finally, factors having to do with the
diagnosis and design, as well as the importance given to evaluation, have not shown any
significant correlation with impact.
Thus, these results suggest that intrinsic impact and extrinsic impact are related, although
they constitute partially independent LDP success factors. Their correlation has been low to
moderate (r=0.210), with both variables sharing 4.4% variance. Therefore, the theoretical
structure predicting a relation between the two types of impact, shown in Figure 3, is only
partially support.
Intrinsic impact significantly exceeds indicators of extrinsic impact, with the former being a
significant predictor of the latter. However, the intrinsic impact generated can account for
only a small part of the extrinsic impact, underlining other factors that influence only
extrinsic impact. Therefore, these results suggest that, to a considerable extent, both impacts
are independent.
Another fundamental outcome of the present investigation involves the empirical evidence in
support of the ability of different systemic factors studied to predict program impact. The
regression models that we used suggest that 23.4% of impact variance is explained by the
program’s design and diagnostic, development, evaluation, auditing, and strategic character
34
factors. Significant predictors, regardless of their impact, were found to be the number of
aspects evaluated and the number of top managers selecting participants.
This result supports the concept of the Virtuous Cycle of LDPs, which proposes that the
training program must be based on a systemic approach, with the variables diagnosis, design,
deployment, evaluation, auditing, and strategic character all correlated with one another.
Other results deserve particular mention. First, our data were on the most commonly used
procedures (content, methodology, program format, evaluation method), and second,
correlations were calculated between these procedures and the impact generated by the
programs.
The present study confirms that more-integrative approaches to leadership development
training need to be adopted to address the shortcomings of current executive education
methodologies—in particular, the problems of their relevance and their impact on business
strategies. We propose the concept of LDP's Virtuous Circle to integrate the main steps in
developing leadership and teaching management programs that have impact.
Breaking down the executive education process into its primary value chain phases will help
with the integration and management of the entire LDP—from training needs identification to
program design and evaluation—for the benefit of an organization's goals. As argued here,
the Virtuous Circle model offers an integrative and dynamic process to effectively identify
and manage the critical factors in designing and delivering successful LDPs.
35
The results of the focus group discussion also support the opinions of theoreticians on the
importance of aligning LDPs with the business. At the same time, they cast light on the poor
existing relationship between companies’ top management and leadership development
policies. Variables with minimal difference between desirability and practicality are
commonly used factors in designing programs: identifying the executive profile, measuring
the programs, and improving internal communication.
Although the members of the focus group stated that they would like to get more information
about business objectives, in the end, they concluded that they were forced to find different
ways to get this information. On the other hand, although they considered the identification
of improvement areas only a minor determinant of program impact, they rated this variable as
easy to achieve and, therefore, ended up using it as one of the program assessment needs
variables.
Finally, the focus group rated commitment of top management as the most desirable
variable—but also the hardest to get—to maximize the impact of LDPs on the company
strategy.
The first part of our research leads to some concerning conclusions, as it shows clearly that
the inputs used to design the LDPs are not the most impactful, but the easiest to find. The
lack of personal involvement of top management, beyond the approval of a budget for LDPs,
affects the programs’ impact significantly. Therefore, the training budget is spent, but with no
clear results and with diminished impact.
36
The literature indicates that, to ensure that LDPs are aligned with corporate needs, program
designers should involve the companies' senior leaders to get a broad outline of the
companies' strategies and to clarify the nature of the problems and challenges their businesses
are facing. However, the results of our investigation show that HR and program designers do
not have easy access to top management and the necessary strategic information, so they
cannot count on this critical input.
The literature review also highlighted the importance of connecting experience-based
learning methods with the organization's real problems in order to help participants
understand and manage personal and organizational change and to ensure that knowledge and
skills transfer to the workplace. Nevertheless, the result of our investigation indicates that the
LDPs may have direct impact on the promotion, the changing roles or the retention of
participants, without the appropriate measurement of changes in knowledge, behavior and
attitudes. The successive regression model shows that only by attending LDPs can the
participants be included in the pipeline of leadership, whether or not they’ve learned or
changed. This finding raises concerns and is a caution to companies that invest millions of
dollars in LDPs.
Another important outcome of the present research involves the empirical evidence implying
that the different systemic factors considered can predict the impact of the programs—an
aspect that has typically been missing from the research and studies on LDPs due to the
difficulty in isolating the effect of training in organizations. The regression models employed
suggest that 23.4% of LDPs’ impact variability is explained by the following program
factors: diagnostic and design, implementation, evaluation, program audit, and strategic
37
character of the program. Significant predictors were shown to be the number of aspects
evaluated and the number of directors selecting participants.
Finally, the research results suggest that conducting an auditing process during the different
stages of the programs, instead of doing only the post-program assessment, increases the
programs’ impact on the organization. In summary, treating the evaluation of LDPs as an
ongoing process during all stages of the value chain (diagnosis, design, development, and
implementation) helps maximize the organizational impact by ensuring that the program
stays aligned with the orgnaization’s objectives throughout. While this process is costlier and
requires the assignment of additional resources (people, time, technology), the impact of the
program will increase, and, consequently, the return on investment will presumably be higher.
While the present research yields useful and practical insights and findings, the study is still
exploratory since the problem of impact assessment of executive education programs has not
been clearly defined at the present stage in either the literature or in practice. The present
study explores different relationships already discussed in the literature, but in a new
conceptual framework, and its primary goal is to link the LDPs’ key factors with their
intrinsic impact on participants their and extrinsic impact on organizational indicators.
Therefore, the study design is not without flaws. One of its main limitations is that we
measure managers’ perceptions rather than objective measures.
As we mentioned at the beginning of the paper, one of the main limitations of executive
education training is that it is complicated to isolate its impact from that of other
organizational measures that can affect outcomes. Therefore, future research should consider
the longitudinal nature of leadership development (Day, 2011) and try to further establish the
38
linkages between LDPs’ factors and their impact on medium- and long-term business results.
Nevertheless, the results of our research can provide significant insight into LDP design,
development and evaluation that can increase their impact, especially in Latin America,
where the human resources and talent development areas are not yet viewed as strategic
partners for companies’ top executives.
Finally, based on the research conducted in this paper, we highlight three main
recommendations for leadership training providers in order to increase the impact of the
programs on organizations: 1) adopt a more integrative, systemic and dynamic model such as
the Virtuous Circle LDPs; 2) define the intrinsic and extrinsic impact indicators during the
design stage; and 3) monitor the process and measure the outcomes, from design to delivery
to evaluation, by auditing the LDPs during each stage. By doing so, both the intrinsic and
extrinsic impacts of LDPs likely will be higher, and participants will be able to contribute in a
more significant way to their jobs and their organizations' results.
39
REFERENCES
Adler, G., Hooijberg, R., Kemanian, V., Keys, T., Lane, N., & Strebel, P. 2005. 3-
dimensional learning. Mastering Executive Education: How to combine content with
context and emotion – the IMD Guide: 68-83. Pearson Education Limited, Harlow: FT
Press.
Akins, R., Bright B., Brunson T. & Worthamet, W. 2013. Effective Leadership for Sustainable
Development. Journal of Organizational Learning and Leadership, 11(1): 29.
Allio, R. J. 2005. Leadership development: teaching versus learning. Management Decision,
43(7/8): 1071-1077.
Avolio, B. J., Avey, J. B., & Quisenberry, D. (2010). Estimating return on leadership
development investment. The Leadership Quarterly, 21(4), 633–644.
Beechler, S. 2009. Executive Education Trends and Adult Learning Practices. Consulting
report. ESADE Business School.
Bennis, W., G., & O' Toole J. 2005. How business schools lost their way. Harvard Business
Review, 83(5): 96-104, 154.
Blackler, F., & Kennedy, A. 2004. The design and evaluation of a leadership program for
experienced chief executives from the public sector. Management Learning, 35(2): 181-
203.
Boiney, L., & Mallinger, M. 2002. Integrating the Curriculum: Executive Education in the
21st Century. International Business & Economics Research Journal 1(2): 93-99.
Booth, C. 2010. Leadership and management education: An analysis of the executive
education experiences of an Australian group of managers, International Review of
Business Research Papers, 6(6): 36-47.
40
Brown, G., Ichijo, K., Lehtivaara, P., Kemanian, V., Keys, T., Maznevski, M., Strebel, P.,
Tobias, A., & Walsh, J. 2005. Real World Context. Mastering executive education: How
to combine content with context and emotion: the IMD Guide: 57-67. Pearson
Education Limited, Harlow: FT Press.
Buchel, B., & Antunes, D. 2007. Reflections on Executive Education: The User and
Provider’s Perspectives. Academy of Management Learning & Education, 6(3): 401-
411.
Cacioppe, R. 1998. An integrated model and approach for the design of effective leadership
development programs. Leadership & Organization Development Journal, 19(1): 44-53
Cardoza, G. 2001. Higher Education and Research and Development (R&D) in East Asia and
Latin America: Different Perspectives? Journal of Institutional Research, 10(2).
Charlton K and C Osterweil. 2005. Measuring Return on Investment in executive education:
a quest to meet client needs or pursuit of the Holy Grail? 360o The Ashridge Journal,
Autumn 2005, Ashridge Business School UK.
Chen, T., Chang, P., & Yeh, C. 2004. Research note: A study of career needs, career
development programs, job satisfaction and the turnover intentions of R&D personnel.
Career Development International, 9(4/5): 424-37.
Clarke, N. 2012. Evaluating Leadership Training and Development: A Levels-of-Analysis
Perspective. Human Resource Development Quarterly, 23(4): 441-460.
Cohn, J., Khurana R., & Reeves, L. 2007. El desarrollo del talento en la cultura de la
empresa. Harvard Deutsche Business Review, 140: 6-17.
Conger, J. 2013, Mind the gaps: what limits the impact of leadership education. Journal of
Leadership Studies, 6(4): 77-83.
Conger, J., & Xin, K. 2000. Executive education in the 21st century. Journal of Management
Education, 24: 73-101.
41
Cordero, C., Haour, G., Kahwayjy, J.L., Kemanian, V., Keys, T., Meehan, S., Robertson, D.,
Sjöblom, L., Strebel, & P., Vollmann, Th. 2005. Program Scripting. Mastering Executive
Education: How to combine content with context and emotion – the IMD Guide: 85-
102. Pearson Education Limited, Harlow: FT Press.
Corporate Learning Factbook. 2014. The Corporate Learning Factbook 2014: Benchmarks,
Trends, and Analysis of the U.S. Training Market. Available at
http://marketing.bersin.com/corporate-learning-factbook-2014.html
Corporate University Xchange. 2007. Leadership 2012: Research Report.
http://www.corpu.com/leadership/leadership2012/
Crossan, M., Mazutis, D., Seijts, G., & Gandz, J. 2013. Developing leadership character in
business programs. Academy of Management Learning & Education, 12: 285–305.
Cranfield ESADE Report. 2008. Gestión Estratégica de Recursos Humanos. Working paper.
Barcelona: Instituto de Estudios Laborales de ESADE.
Culbertson, S., & Owen, K. 2012. Demonstrating the Value of Executive Education. People
& Strategy, 35(3): 44-51.
Cullen, J., & Turnbull, S. 2005. A meta-review of the management development
literature. Human Resource Development Review, 4(3): 335-355.
Dawes, J. 2008. Do data characteristics change according to the number of scale points used?
International Journal of Market Research, 50(1): 2-10.
Day, D. V. 2000. Leadership development: A review in context. The Leadership Quarterly,
11(4): 581-613.
Day, D. 2011. Integrative perspectives on longitudinal investigations of leader development:
From childhood through adulthood. The Leadership Quarterly, 22(3), 561–571.
Day, D. et al. 2014. Advances in leader and leadership development: A review of 25 years of
research and theory, The Leadership Quarterly 25 (2014) 63–82.
42
Deloitte. 2016. Global HumanCapital Trends report. 2016. The new organization: Different
by design, Deloitte University Press. Available at:
https://www2.deloitte.com/content/dam/Deloitte/global/Documents/HumanCapital/gx-
dup-global-human-capital-trends-2016.pdf
De Déa Roglio, K. & Light, G. 2009. Executive MBA programs: The development of the
reflective executive. Academy of Management Learning & Education, 8(2): 156-173.
De Meyer, A. 2012. Environmental influences. Reflections on the globalization of
management education. Journal of Management Development, 31(4): 336-345.
De Vries, K. M. & Korotov, K. 2007. Creating transformational Executive Education
programs. Academy of Management Learning & Education, 6(3): 375-387.
Dalakoura, A. 2010. Differentiating leader and leadership development. A collective
framework for leadership development. Journal of Management Development, 29(5):
432-441.
DiStefano, J., Kemanian, V., Keys, T., & Strebel, P. 2005. Mastering Executive Education:
How to combine content with context and emotion – the IMD Guide: 42-54. Session
scripting. Pearson Education Limited, Harlow: FT Press.
Doh, J.P. 2003. ¿Can leadership be taught? Perspectives from management educators.
Academy of Management Learning and Education, 2(1): 54-67.
Doh, J.P., & Stumpf, S.A. 2007. Executive Education: A View from the Top. Academy of
Management Learning & Education, 6(3): 388-400.
Dolliver, S. 1994. The missing link: Evaluating training programs. Supervision, 55:11.
Dover, P., Perkins, S., & Wylie, D. 2009. The role of custom case materials in action-based
executive education programs. Journal of Management Development, 28(4): 285-300.
Dragoni, L., Tesluk, P. E., Russell J. E. A., & Oh, I. S. 2009. Understanding managerial
development: Integrating developmental assignments, learning orientation, and access to
43
developmental opportunities in predicting managerial competencies. Academy of
Management Journal, 52(4): 731-743.
Duke CE´s Client Study. 2009. Learning and Development in 2011: A Focus on the future.
Duke Corporate Education. http://www.dukece.com/papers-
reports/documents/FocusFuture.pdf
Eguiguren, M. 2000. Aspectos económicos de la formación en la empresa: una metodología
para el control de gestión de la función de formación en la empresa en Cataluña.
Doctoral thesis. Departamento de Organización de Empresas. Universitat Politècnica de
Catalunya. http://www.tesisenred.net/TDX-1010101-082513
Eiter, M. 2009. Investigating Our Custom Clients’ Evolving Needs. A UNICON Research
Study. www.uniconexed.org
Eiter, M., & Halperin, R. 2010. Investigating Emerging Practices in Executive Program
Evaluation. A UNICON Research Study. September. www.uniconexed.org
ESADE. 2008, 2009, 2010. Corporate University Forum conclusions. Working document.
ESADE Business School.
ESADE. 2010. Custom Programs data base. Working document. ESADE Business School.
Madrid: Executive Education.
Fulford, MD. 2013. Practice What You Preach: Using an Experiential Learning Approach to
Teach Leadership. Journal of Leadership, Accountability and Ethics, 10(2): 81-86.
Garratt, B. & Pedler, M. (Ed.). 2012. The Power of Action Learning in Action Learning in
Practice. Henley Business School, UK.
Goleman, D., Boyatzis, R., & McKee, A. 2001. Primal leadership: the hidden driver of great
performance. Harvard Business Review: 43-51.
Goleman, D., Boyatzis, R., & Mckee, A. 2002. Primal Leadership. Boston: Harvard
Business School Publishing.
44
Greenwood, D., & Levin, M. 2001. Pragmatic action research and the struggle to transform
universities into learning communities, in Reason, P. Bradbury, H. (Eds.), Handbook of
Action Research: 103-113. London. Sage Publications.
Grossman R., Salas, E., Pavlas, D., & Rosen, M.A. 2013. Using Instructional Features to
Enhance Demonstration-Based Training in Management Education. Academy of
Management Learning & Education, 12(2): 219-243.
Gurdjian, P., Halbeisen, T., & Lane, K. 2014. Why leadership-development programs fail.
McKinsey Quarterly, 1: 121-126.
Hannum, K. M., & Craig, S. B. (2010). Introduction to special issue on leadership
development evaluation. The Leadership Quarterly, 21(4), 581–582.
Hartman, J. 2004. Using focus groups to conduct business communication research. Journal
of Business Communication, 41(4): 402-410.
Heifetz, R. 1994. Leadership without easy answers. Cambridge, MA: Harvard University
Press.
Herndon, S. L. 2001. Using focus group interviews for preliminary investigation.
Qualitative research: Applications in organizational life. Cresskill, NJ: Hampton Press.
Holton III, E. F. 1996. The flawed 4-level evaluation model. Human Resource Development
Quarterly, 7: 5-21.
Hugas, J. 2007. Executive Education: LDP leadership development program. Diploma de
Estudios Avanzados. Programa de Doctorado en Innovation Management, R&D and
Technology Evaluation. Departamento de Organización, Gestión Empresarial y Diseño de
Productos. Universidad de Girona.
Hugas, J. 2007. Liderazgo: “del arte a la ciencia”, Diploma de Estudios Avanzados.
Programa de Doctorado en Innovation Management, R&D and Technology Evaluation.
45
Departamento de Organización, Gestión Empresarial y Diseño de Productos. Universidad
de Girona.
Hughes, G. D. 2006. How business education must change. MIT Sloan Management
Review, 47(3): 88.
Ilie, C. 2009. El impacto de los programas de desarrollo directivo en la estrategia
empresarial. Doctoral thesis. Universidad Comillas—ICADE. Madrid.
https://www.educacion.es/teseo/mostrarSeleccion.do
International University Consortium for Executive Education, UNICON. 2005. Executive
Education: Evaluating the return on investment. Bringing the client voice into the debate.
Study by Ashridge for Unicon. www.uniconexed.org
Jenkins, DM. 2013. Exploring instructional strategies in student leadership development
programming. Journal of Leadership Studies, 6(4): 48-62.
Kahwajy, J., Kemanian, V., Keys, & T., Strebel, P. 2005. Emotional Highs. Mastering
Executive Education: How to combine content with context and emotion – the IMD
Guide: 13-20. Pearson Education Limited, Harlow: FT Press.
Kirkpatrick, D. L. 1979. Techniques for evaluating training programs. Training and
Development Journal, 33(6): 78-92.
Kirkpatrick, D. 1994. Evaluating training programs. San Francisco, CA: Berrett-Koehler
Publishers.
Kirkpatrick, D. 2006. Training and performance appraisal - are they related? T+D: 44-45.
Kirkpatrick, D. L., & Kirkpatrick J. D. 2009. Evaluating Training Programs. Berrett-
Koehler Publishers.
Kirkpatrick, S. & Locke, E. A. 1991. Leadership, do traits matter? Academy of Managerial
Executive, 5(2): 48-60.
Kline, P. 1994. An easy guide to factor analysis. Newbury Park: Sage.
46
Knowles, M. 1978. The Adult Learner: A neglected species. Houston: Golf Publishing.
Knowles, M., & Malcom, S. 1989. Learning to be authentic. Training & Development
Journal, 43(10): 42.
Kolb, D. A. 1984. Experiential learning: Experience as a source of learning and
development. Englewood Cliffs NJ: Prentice-Hall Inc.
Kolb, A. & Kolb, D. A. 2005. The Kolb Learning Style Inventory —Version 3.1 2005
Technical Specifications. HayGroup Experience Based Learning Systems, Inc.
Kraiger, K., McLinden, D., & Casper, W. 2004. Collaborative planning for training impact.
Human Resource Management, 43(4): 337-351.
Krueger, R. 1994. Focus groups: A practical guide for applied research. Second Edition.
Newbury Park, CA: Sage.
Linder, J., & Smith, H. 1992. The complex case of management education. Harvard
Business Review, 70(5): 16-17.
Lloyd, F. & Newkirk, D. 2011. University-based executive education markets and trends.
Research paper. http://uniconexed.org/2011/research/UNICON-whitepaper-markets-
trends-Lloyd-Newkirk-08-2011.pdf
Lombardo, M., & Eichinger, R. 2010. Eighty-Eight Assignments for Development in Place
(French Canadian). CCL Press.
Magni, M. et al. 2013. Diving Too Deep: How Cognitive Absorption and Group Learning
Behavior Affect Individual Learning Academy of Management Learning & Education,
12(1): 51-69.
Mazza, C. 2004. Rethinking management education for the 21st century. Management
Learning, 35(3): 349-352.
Mattioli, D. 2009. Despite Cutbacks, Firms Invest in Developing Leaders, The Wall Street
Journal. http://online.wsj.com/news/articles/SB123395874246058397
47
McClelland, S. B. 1993. Training Needs Assessment: An Open-systems Application. Journal
of European Industrial Training, 17(1): 1993.
McClelland, S. 1994. A model for designing objective-oriented training evaluations.
Industrial and Commercial Training, 26(1): 3-9.
McEnrue, M.P., Groves, K. S., & Shen, W. 2007. Emotional intelligence development:
Leveraging individual characteristics. Journal of Management Development, 28(2): 150-
174.
Miller, D., & Desmarais, S. 2007. Developing your talent to the next level: Five best
practices for leadership development. Organization Development Journal, 25(3): 37-43.
Mintzberg, H. 2005. Managers Not MBAs: A Hard Look at the Soft Practice of Managing
and Management Development. Berrett-Koehler Publishers.
Mirvis, P. 2008. Executive Development through Consciousness-Raising Experiences.
Academy of Management Learning & Education, 7(2):173-188.
Molenda, M. 2003. In Search of the Elusive ADDIE Model. Performance Improvement,
42(5).
Morgan, D. L. 1988. Focus groups as qualitative research. Qualitative Research Methods
Series: 16. Newbury Park: Sage.
Mumford, A. 1995. Effective Learning. Institute of Personnel and Development. London:
CIPD Publishing.
Nevins, M. D., & Stumpf, S. A. 1999. 21st-Century Leadership: Redefining Management
Education. Strategy+Business (3rd Quarter): 1-11.
Patton, M. Q. 1987. How to Use Qualitative Methods in Evaluation. Newbury Park: Sage.
Perez, S. 2014. The ROI of Talent Development. UNC Kenan-Flagler Business School, UNC
Executive Development 2014. Available at: http://www.kenan-
48
flagler.unc.edu/~/media/Files/documents/executive-development/unc-white-paper-the-
ROI-of-talent-development.pdf
Pettigrew, Andrew, and Ken Starkey. "From the Guest Editors: The Legitimacy and Impact of
Business Schools—Key Issues and a Research Agenda." Academy of Management
Learning & Education 15.4 (2016): 649-664.
Pfeffer J. & Fong C. 2002. The End of Business Schools? Less success than meets the eye.
Academy of Management Learning and Education. 1(1): 78-95.
Pfeffer, J. 2009. Leadership development in business schools: An agenda for change.
Research Paper No. 2016. Graduate School of Business, Stanford University.
Phillips, J. 1996. ROI: The search for best practices. Training & Development, 50(2): 42.
Pledger, C. 2007. Building manager effectiveness by combining leadership training and
organization development. Organization Development Journal, 25(2): 71-76.
Quigley, N. 2013. A Longitudinal, Multilevel Study of Leadership Efficacy Development in
MBA Teams Villanova University. Academy of Management Learning & Education,
12(4): 579-602.
1. Quinn Patton, M. (1987). How to use qualitative methods in evaluation. Newbury Park:
Sage.
Raelin, J. A. 2007. Toward an Epistemology of Practice. Academy of Management Learning
& Education, 6(4): 495-519.
Rowden, W. R. 2005. Exploring methods to evaluate the return-on-investment from training.
Business Forum, 27(1): 31-36.
Sandoval, C. 1996. Métodos de investigación cualitativa. Bogotá: ICFES.
49
Sauquet, A., & Bonet, E. 2003. Implications of national cultural impacts for conflict
resolution and team learning in Spain: Observations from a comparative case study.
Advances in Developing Human Resources, 5(1): 41-63.
Schneider, S. C., Zollo, M., & Manocha, R. 2010. Developing Socially Responsible
Behaviour in Managers: Experimental evidence of the effectiveness of different
approaches to management education. Journal of Corporate Citizenship, 39: 21-40.
Schön, D. A. 1983. The reflective practitioner. How professionals think in action. Basic
Books.
Scott, S. & Webber, C. F. 2008. Evidence-based leadership development: the 4L framework.
Journal of Educational Administration, 46(6): 762-776.
Siegling, A. B., Nielsen, C., & Petrides, K. V. (2014). Trait emotional intelligence and
leadership in a European multinational company. Personality and Individual
Differences, 65, 65-68.
Smith A. 2013. Responsible leadership development through management education: A
business ethics perspective. Afr J Bus Ethics, 7: 3-9.
Snook, S., Nohria, N., & Khurana, R. N. (Eds.). 2013. The handbook of teaching leadership:
Knowing, doing and being. Thousand Oaks, CA: Sage.
Solé Parrellada, F., Eguiguren, M., Llínás, X., & Pons, O. 2006. Los aspectos económicos—
organizativos de la formación: una aproximación al caso de Cataluña. Universia Business
Review-Actualidad económica, 9: 28-41.
Strebel, P. & Keys, T. 2005. High-impact learning. Mastering Executive Education: How to
combine content with context and emotion – the IMD Guide: 1-8. Pearson Education
Limited, Harlow: FT Press.
50
Tejada Ferández, J., & Ferrández Lafuente, E. 2007. La evaluación del impacto de la
formación como estrategia de mejora de las organizaciones. Revista Electrónica de
Investigación Educativa, 9(2). http://redie.uabc.mx/vol9no2/contenido-tejada2.pdf
Thomas J.R. 2013. Developing Tomorrow’s Global Leaders. MIT Sloan Management
Review, 55(1): 12-13.
Thomas, H., & Wilson, A. 2009. An analysis of the environment and competitive dynamics of
management research. Journal of Management Development, 28(8): 668-684.
Thorsell, J., Bridge, J., & Gardner, F. 2012. Executive development: Evolutionary revolution.
Global Focus: The EFMD Business Magazine, 6(3): 34.
Tompson, H.B. & Tompson, G. 2013.The Focus of Leadership Development in MNCs.
International Journal of Leadership Studies, 8(1): 67-75.
Tushman, M. I., O'Reilly, C. A., Fenollosa, A., Kleinbaum, A. M., & McGrath, D. 2007.
Relevance and Rigor: Executive Education as a Lever in Shaping Practice and Research.
Academy of Management Learning & Education, 6(3): 345-362.
Weldon, E. 2012. Developing successful executive education faculty: a review of the
literature from 2001-2011. A UNICON Research Report.
Werner, J. M., O Leary-Kelly, A. M., Baldwin, T.T., & Wexley, K. N. 1994. Augmenting
behaviour-modelling training: Testing the effects of pre- and post-training
interventions. Human Resource Development Quarterly, 5(2): 169.
Winfrey, E.C. 1999. Kirkpatrick's Four Levels of Evaluation. In B. Hoffman (Ed.),
Encyclopedia of Educational Technology. www.coe.sdsu.edu/EET.
Yorks, L, Beechler, S. and Ciporen, R. 2007. Enhancing the Impact of an Open Enrollment
Executive Program Through Assessment. Academy of Management Learning &
Education, 6(3): 310-320.
51
Yulmartin, E. B. H. & Riyanto A. 2013. Improving Leadership Training Effectiveness
through Action Learning Program. Proceedings of the European Conference on
Management, Leadership & Governance: 498.
FIGURE LEGENDS
Figure 1. Virtuous Circle of LDPs. Source: Authors.
Figure 2. Relation between LDPs and Program Impact. Source: Authors.
Figure 3. Prediction model for LDP descriptors and program impact. Source: Authors.
Figure 4. Synthesis of the regression analysis. Source: Authors.
TABLE LEGENDS
Table 1. Critical variables for impact of LDPs on company strategy. Source: Authors
Table 2. Learning Methodologies. Source: Authors
Table 3. Correlations between different types of evaluations and the impact generated
by the LDPs. Source: Authors
Table 4. Value of LDPS descriptors as predictors of program impact. Source: Authors
Table 5. Multiple regression model. Source: Authors
52
APPENDIX 1
Focus group definition and organization
After group member introductions, the purpose of the meeting was explained, and the debate
was opened, engaging them in conversation about the following issues:
- How their training plans and LDPs are designed.
- Which executive level is involved in the design of the LDPs.
- How LDPs are evaluated and their impact on the company.
- Whether there is any relation among professional career development, succession
plans, and LDPs.
- How the potential leaders are identified.
During this first stage, some common aspects were observed during the discussion:
- Most participants believed that top executives did not clearly share the company
strategy with them, and, thus, they felt weak when designing leadership development
policies for senior managers.
- Another issue highlighted by the group was the uncertainty about their environment
due to a wave of mergers and acquisitions that created constant changes in leadership
development priorities within the company.
- However, in the Human Resources department, most of the participants believed that
processes to identify the leadership pipeline, the competency models, and leadership
development needs were well defined and had been implemented correctly.
After the first stage of general reflection, the two moderators introduced the topic, explaining
the goals of the research and stating concrete objectives. Then, they asked the research
question: “What aspects can increase the impact of LDPs on the participants and the
organization?”
Next, each group member was asked to choose three variables that could increase or weaken
the impact and to write them on a piece of paper. Then, the two moderators selected similar
variables, by choosing papers with similar concepts, and created a reduced number of groups
with the same issues, which they placed in the same area of the work panel.
Once the variables were grouped, each participant voted on them again, rating them from 1
(least important) to 3 (most important). The selected variables that can increase or weaken the
impact, in order of importance, were:
- Link between the program and the business strategy (15)
- Commitment from top management (9)
- Customization of the program (6)
- Participant selection (5)
- Link between the program and human resources policies (4)
- Commitment of top management with the programs (4) [How is this different from
“Commitment from top management”?]
- Impact evaluation (3)
- Alignment and planning of the programs with the executives’ and top management’s
53
agenda (2)
Thus, during the first stage of the focus group, the two variables with the most votes (which
were also highlighted by the theoretical review) were: alignment of LDPs with the company
strategy and commitment of top management.
In the second stage of the focus group, the top-voted variable was selected. The moderators
asked the group to discuss the following question: "How should the LDPs be linked to the
company business strategy?”
As in the first stage, each participant chose three variables that they considered critical to
connecting LDPs with the business strategy.
Afterwards, variables were grouped again, and participants voted on those that, from an ideal
(desirable) viewpoint, were most important to achieve a link between the program and the
business strategy. One point was given to the least significant variable and three points to the
most important.
Finally, another voting round was conducted, this time from a pragmatic viewpoint (easiest to
achieve). As in the first stage, each participant picked the three most important variables to
link programs and business strategy. Variables and scores are shown in Table 1.
54
Table 1
Critical variables for impact of LDPs on company strategy
Source: Authors
Variables Total Desirable Real
(Practical)
Difference
Having a clear executive profile 29 15 14 -1
Training measurement 3 2 1 -1
Having information on business
objectives
23 13 10 -3
Getting commitment from top
management
19 16 3 -13
Identifying improvement areas 10 0 10 +10
Proposing integrated leadership
development actions
9 2 7 +5
Improving communication 3 0 3 +3
