Embed Size (px)
Citation preview
Integrated Project
The Virtual Team Challenge: Is it time for training?
By: Lance Poehler
EMGT 590
Rose-Hulman Institute of Technology, Terre Haute, IN 47803 USA
Spring 2007
Abstract
In today’s environment, organizations deal with increasing levels of complex
changes and environmental uncertainties [5]. These changes appear to favor teams with
high diversity, where the best talent can be leveraged [14], allowing the team to identify
global efficiencies [10], [56]. There have also been concurrent changes in information
technologies. Patricia Wallace believes that the Internet has acted like a catalyst for
innovation, allowing employees to access work at anytime from anywhere These Net-
centric technology changes have altered the ways people are capable of communicating,
creating the opportunity, and encouraging management to build and manage virtual teams
[57][29]. Virtual teams can provide the ability to respond rapidly to the current business
challenges, which is why the use of these teams is rapidly expanding [29]. Researchers
investigating Virtual Teams describe problems these teams encounter, assert that training
for virtual team assignments is necessary, and offer suggestions on the issues that such
training should address.
The Virtual Team Challenge has been developed to provide training that can
reduce or eliminate common problems within virtual teams. This simulation will provide
the participants with experience in managing problems and team dynamics in an
authentic learning environment [25]. The simulation is composed of two unique and
contrasting viewpoints. Throughout this simulation, companies that represent each of
these viewpoints will employ the participants. The player will need to adapt to the
cognitive framework employed by each company. The game design causes the player to
see a project from two different perspectives in a process I have termed ‘forced empathy’.
The intent of this simulation is that the players gain a better understanding of what
actions to take in order to make a virtual team successful.
- i -
Table of Contents
THE IMPORTANCE OF VIRTUAL TEAMS............................................................................................... 1 CONCLUSIONS FROM PREVIOUS RESEARCH ..................................................................................... 1 VIRTUAL TEAM PITFALLS .......................................................................................................................... 8 THE LEARNING CHALLENGE................................................................................................................... 11
WHY SIMULATIONS ......................................................................................................................................... 11 FORCED EMPATHY .......................................................................................................................................... 13 SIMULATION ADVISOR .................................................................................................................................... 14 LEARNING (INTERFACES & MESSAGE)........................................................................................................... 15 WHAT ARE THE LEARNING CHALLENGES ..................................................................................................... 16
WHAT ARE THE BENEFITS ........................................................................................................................ 17 THE ‘PAIRED-GAME’ MODEL BENEFIT .......................................................................................................... 17 BENEFITS FROM DESIGN.................................................................................................................................. 19
DEVELOPMENT PLATFORM ..................................................................................................................... 27 THE VIRTUAL TEAM CHALLENGE: TEST RESULTS ....................................................................... 28 POTENTIAL AREAS OF INTEREST.......................................................................................................... 34 THE CURRENT PLAYING FIELD .............................................................................................................. 35 CONCLUSION .................................................................................................................................................. 37 REFERENCES................................................................................................................................................... 37 ADDENDUMS ................................................................................................................................................... 44
REVOLUTION ENTERPRISE .............................................................................................................................. 44 REVOLUTION: BENEFITS ................................................................................................................................. 45 REVOLUTION MEDIA AND STUDIO FEATURES ............................................................................................... 47 REVOLUTION - FAQ ........................................................................................................................................ 53 DETAIL ANALYSIS OF TWO-SAMPLE TEST OF MEANS: SMALL SAMPLES .................................................... 56 DETAIL ANALYSIS OF THE KOLMOGOROV-SMIRNOV TWO-SAMPLE TEST .................................................. 57 FISHER GROUP TRAINING PROGRAMS: THE DISTANCE MANAGER .............................................................. 59
- ii -
Table of Figures
Figure 1: Complexity of Knowledge vs. Nature of Interactivity [48] .............................................................. 12 Figure 2: A thermodynamic metaphor for Frame of Reference shift [45] ...................................................... 14 Figure 3: Learning phases found in using simulations [48] ............................................................................ 16 Figure 4: Screenshot of the introduction page of the Virtual Team Challenge .............................................. 21 Figure 5: Screenshot of advisor from the Virtual Team Challenge ................................................................. 21 Figure 6: Screenshot of scheduling meeting from the Virtual Team Challenge ............................................. 22 Figure 7: Screenshot of networking from the Virtual Team Challenge ........................................................... 22 Figure 8 Screenshot of a face-to-face meeting from the Virtual Team Challenge.......................................... 23 Figure 9: Screenshot for video conference from the Virtual Team Challenge................................................ 23 Figure 10: Screenshot of phone call from the Virtual Team Challenge .......................................................... 24 Figure 11: Screenshot of detailed calendar from the Virtual Team Challenge .............................................. 24 Figure 12 Screenshot of PERT chart from the Virtual Team Challenge ......................................................... 25 Figure 13: Screenshot of time allocation from the Virtual Team Challenge .................................................. 25 Figure 14: Screenshot of team meetings from the Virtual Team Challenge.................................................... 26 Figure 15: Screenshot of team member variables from the Virtual Team Challenge .................................... 26 Figure 16: Screenshot of the finish screen from the Virtual Team Challenge ................................................ 27 Figure 17: Part 1 of 2 of Questionnaire ............................................................................................................ 29 Figure 18: Part 2 of 2 of Questionnaire ............................................................................................................ 30
- iii -
Table of Tables
Table 1: Participants Critique ............................................................................................................................ 31 Table 2: Statistical Analysis................................................................................................................................ 33
- 1 -
The Importance of Virtual Teams
Virtual teams, virtual organizations, remote teams, and cross-site teams are terms
in the literature that have been used to describe the concept of geographically dispersed
teams. The virtual teams concept can be described as a group of employees located in
different locations who must collaborate to accomplish tasks. These teams can vary
along a continuum from highly to minimally virtual [29]. On one end of the continuum
the team may never meet face-to-face, teams at the other end of the spectrum may meet
face-to-face every day and also conduct other dispersed activities. Virtual teams have the
potential for high performance [10] mainly because diversity can provide a competitive
advantage [14]. According to Walter Wriston, the former Citibank CEO, “the person
who figures out how to harness the collective genius of the people in his or her
organization is going to blow the competition away” [52].
Conclusions from Previous Research
When virtual team members understand their strengths and weaknesses as well as
those of their teammates it allows the team to work effectively and creates the
opportunity for success [9]. Bruce Tuckman’s 1965 standard “forming, storming,
norming, performing” is still relevant in virtual teams [54]. The team needs to
communicate [26], especially face-to-face [14] at the beginning of the project. Teams
also need to understand and agree upon their goals from the start of the project [9]. One
of the best ways to accomplish this is by having a team orientation meeting. Many
authors believe that having a team orientation will help to reduce future conflict within
the team [4], [13], [14], [26], [44]. During this meeting it is important to develop the
- 2 -
team norms [14], roles, goals [4], as well as all of the team ground rules [44]. Multi-
cultural teams may run into scheduling problems with some of the different cultural
holidays and religious activities [14], [56]. In addition, different cultures have different
views of time [14]. Some cultures view time as very precious while other cultures view
time as part of life [23], [47]. For these reasons and many others, Deborah L. Duarte
believes that it is important to offer a culture-training course during the initial kick off
meeting so that the team may learn about national and cultural differences. Differences
in time zones, holidays, and work hours, how these differences may affect team norms, as
well as how different cultures view personal praise or reward in front of the group are just
a few of the concepts that are important for the team to understand in order to master
virtual teams [14].
Virtual teams that develop frequent and predictable communication schedules
(norms) will be much more successful than teams without norms [26], [44], [47]. These
types of communication behaviors help to develop trust [26]. It is important to
understand that even though trust can begin to develop early in the project it takes time
for trust to develop fully [41] and the first meeting sets the tone [14]. Building trust can
be the most difficult challenge that virtual teams face [30]. While reaching out and
communicating with other team members is important, the quality and timing of the
responses are even more imperative [13], [26]. Team members are able to make
corrections to their work when they receive quality feedback from the team [29].
Cascio, Shurygailo, Jarvenpaa, and Leidner believe that each team member is
responsible for regular communication [4], [26]. Teams that conveyed enthusiasm and
optimism have been found to be much more successful than teams that do not [26]. Also,
- 3 -
inspiration and encouragement have been shown to cause higher levels of learning [15],
which in turn enhances the performance of the team [29], [58]. When achievements are
made or tasks completed by team members, both the management and the team members
should provide recognition [4]. This being said, it is important that this recognition is
directed to the team and not to an individual [14] to avoid problems associated with
differing culture norms.
Social interaction is another communication variable that needs to be considered
when looking at successful teams [8], [26]. Small talk about the weekend, family,
personal hobbies, or simply sharing experiences can all help to create a sense of
belonging [14] as well as building trust [26], [41]. It is important to understand that the
appropriate level of small talk varies from one culture to another. For example, in Japan
this type of interaction may be an important courtesy but probably not so much for New
Yorkers. Jarvenpaa found that high levels of trust help members to cope with technical
and task uncertainty [26]. Focusing on deadlines has been shown to eliminate most
social communication [47] and for this reason may need to be kept to a minimum.
Deborah Duarte also believes that it is important to budget for a social gathering at the
end of the project. Providing closure for the project is very important [14]. This will
allow all of the team members to discuss their new assignments [14] as well as
documenting all of the lessons learned [17].
One thing that virtual team members should always understand is that there is
more than one way to communicate. It has also been found that virtual teams should not
rely solely on written communication because this tends to cause unnecessary
documentation [4], [14]. The more often that the team can communicate using video and
- 4 -
audio the easier it will be for the team to overcome the difficulties caused by distance and
time [14]. Distributing detailed meeting notes can also help teams with language or
accent problems. In addition, it is important for teams to know that when relationship (or
emotional) problems occur, or simply when there is a large amount of ambiguity
surrounding an issue, then face-to-face communication is an effective way to repair these
problems [14], [56]. Face-to-face meetings are considered difficult to replace, if not
impossible, as reported by Jarvenpaa and Leidner, when it comes to building trust and
repairing trust [26], [42], [43]. Although trust may be important in traditional teams, it is
vital for the success of virtual teams [4].
Romahn and Hartman describe three different types of trust: competence, ethical,
and emotional trust. Competence trust is the idea that you trust someone to be
knowledgeable enough to complete their tasks. Ethical trust is the idea that one person
believes that another person will make choices that according to their expectations.
Ethical trust can be improved with open communication and team norms. Emotional
trust, as expected, is the most difficult to improve. This type of trust has to do with one
team member trusting another simply because the team members like each other. High
level of each of these types of trust is ideal [46]. Trust is very important for the success
and performance of the team [14]. Without team trust it is virtually impossible to
develop strong mental models1, a shared vision, or maximize performance [14]. As the
level of trust within the team increases, the team will begin to communicate more openly
which will allow the team to uncover possible task issues earlier. Trust is a key factor in
1 Senge describes mental models as “deeply ingrained assumptions, generalizations, or even pictures or images that influence how we understand the world and how we take action” [52].
- 5 -
team performance.
It has been shown that all humans have the need for a sense of belonging [24]. It
may be possible to fulfill that need by having a sense of belonging to a community, even
if the community is a virtual team [24]. One of the ways to enrich this community is to
develop a shared vision. Having strong mental models are important for teams with time
constraints [13] because it provides both the focus and energy for learning [52]. It is
important to be aware that shared mental models decrease over the life of the project.
This is due to team member roles becoming more focused and, therefore, spending more
time working on the project and less time interacting with the team. Having positive and
consistent communication norms can slow this weakening of a shared mental model [34].
According to Peter M. Senge “few if any, forces in human affairs are as powerful as
shared vision” [52].
Teams will also need to learn throughout the project’s life cycle. Team learning
can increase the performance of the team [14], [29], [58]. When problems occur, teams
that have failed to learn are more likely to experience aggravation, conflict, and distrust
within the team [58]. One of the ways to increase the team learning is by utilizing local
responsiveness [58], otherwise known as social networking [10]. The management and
team leader should not only support but also encourage the team members to build social
networks [10]. Deborah Duarte found that effective virtual teams often spend most of
their networking time during the first few weeks of the project [14]. Social networking
will allow the team to have access to information and ideas independent from the team.
Social networking can improve performance by helping to reduce some of the uncertainty
in a changing environment. This uncertainty is reduced because team members gather
- 6 -
and interpret technological and market information from their social network that
otherwise may never have been considered [10]. Another step that should be taken to
improve the team learning is to establish knowledge management norms or procedures.
This facilitates team learning because it provides the team members with another source
for information [58].
Due to the nature of virtual teams, the level of diversity has the potential to be
higher in virtual teams than in traditional teams. There are two main categories of
diversity that tend to effect virtual teams: educational and national diversity [11], [14].
Zellmer-Bruhn and Gibson have found that exposure to diversity tends to stimulate
problem solving [58]. Diversity in expertise has been shown to influence a team’s
innovativeness, and have more of an effect on the range, depth, and integration of
innovation than national diversity, which has been shown to interfere with team
performance [11]. With the help of the cultural training and social interaction, the team
members should be able to identify and acknowledge each other’s circumstances as well
as cultural holidays [56]. It is important for the team leaders to be perceptive to each of
the team members’ unique culture [4]. With some work the appearance of differences in
national origin will tend to fad away over time if the team members have come to know
one another though positive and consistent social interaction [11].
When team conflicts arise, virtual teams will have to engage in conflict resolution
in order to avoid harmful effects on the team performance [13], [26], [40]. Gray,
Susman, and Ren found that one of the ways to handle conflict is to have a broker play a
key role as the conflict handler. The broker has a unique and important position because
they have the ability to see conflicts from both points of view. People in this position are
- 7 -
also less likely to see the conflicts as detrimental to the team when compared to the
perspectives of the team members involved [21]. It is also important to understand that
managing conflict by avoidance has a negative effect on the team performance [40].
Team members avoid criticism and challenges through email because they can easily be
misconstrued. Instead the team should collaborate together face-to-face to come up with
the best possible solution for the problem [14]. There are certain conflict situations that
are even more difficult to solve. For example, a team member located in a different
location is told from their boss that, at present, something else is more important. Dr.
Chenoweth (personal communication, April 24, 2007) believes that “these organizational
problems tend to be team splitters.” Team leaders should discuses the importance of the
project with the team member’s boss and work to find a solution to the situation.
Teams that lack a sense of empowerment have the tendency to distrust each other,
hold information, and are reluctant to take risks. There are a few things that both
management and the team leader can do to improve the team empowerment and,
therefore, the performance of the team [44]. First, it is important not to micromanage the
team members but to concentrate on motivating, encouraging [14], and trusting [24] –
basically treating the team members like partners [44]. The empowerment of the team
members is dependent upon the organization’s characteristics, their job satisfaction, and
the performance of the management or team leader [29]. Frequent meetings can also help
to overcome the lack of team empowerment [29]. In essence, the performance of the
team can be improved when the team leader encourages, coaches, and simply empowers
the team [29], [58].
- 8 -
Virtual Team Pitfalls
Along with the rapid increase in deployment of virtual teams has come an
awareness of problems that can occur during the life of a virtual project [26]. O’Hara-
Devereaux and Johansen [43] found that problems such as low team member
commitment, role overload or ambiguity, absenteeism, or even social loafing, all of
which may be troublesome to traditional teams, become even more disruptive in virtual
teams.
There is a strong possibility that a diverse team will begin to form subgroups [11].
Team members with a similar background or culture may form cliques. This
phenomenon is augmented in many virtual team situations because there are often
culture-based misunderstandings between culturally different team members [56].
Although diversity brings more perspectives, which is positive to the team, it also makes
it difficult for the team to communicate and synchronize their activities [10], [11]. One
observation is that as interpersonal diversity increases, there is a corresponding increase
in the inadequacy of Information Theory as a model for communication. Simply
exchanging bits of information does not support communication when the parties do not
share sufficient common cultural referents. Schumacher describes the growing relevancy
of the Constructivist approach to communication with greater diversity [50]. Schumacher
[45] found the following:
In this model, greater time and attention is necessary to establish shared context in
order that exchanged symbols can convey intended meanings. The obvious
paradox is that the limited bandwidth supported by information technologies, and
the limited meeting time inherent in most virtual teams, substantially reduces
- 9 -
opportunities to establish those shared referents.
Other suggestions in the Virtual Team literature include support for an egalitarian
approach. For example, team leaders need to offer the same type of career development
for all members on the team and not just the ones in close proximity [14]. It is also
judged important that team members with similar skills and duties during the project are
paid equally. When external team members get paid less for the same type of job it
creates tension within the organization due to an economic threat [12], which may lower
the performance of the team. Internal team members may feel like their job may be at risk
because there are external employees completing similar duties while receiving lower
compensation.
Although face-to-face communication is an effective way to repair relationship
problems, all of the people involved must be willing to talk. Glinow, Shapiro, and Brett
found that sometimes when team members are experiencing strong emotions they can’t or
won’t talk. In addition, they also found that emotional conflicts, which are often caused
by cultural differences, are inevitable in virtual teams. For situations where the team
members can’t or won’t talk, aesthetic activities are great alternatives. This could include
music, outdoor events, city festivals, art exhibits, or even volunteer public services. For
virtual teams that don’t or rarely meet face-to-face this becomes more difficult. These
type of teams will need to spend some extra time socializing, preferably during a video
conference, to overcome these issues. These activities have been shown to increase the
level of empathy that each member feels to one another while reducing some of the
original negative feelings [56].
Communication of information and ideas are often more difficult when using
- 10 -
electronic technology [10]. E-mail, telephone, fax, and voicemail offer more restricted
bandwidth than face-to-face communication [24]. Awareness of cultural differences as
well as e-mail etiquette issues is extremely important when communicating by e-mail
[57]. E-mail messages are extremely easy to misinterpret, with both information and
meaning subject to distortion or partial loss [14]. Videoconferences are capable of
transferring much more information (facial expressions, gestures, tonality, etc.) but they
still lack the touch of face-to-face communication.
Another critical issue to consider is the amount of management communication.
The lack of management communication can reduce the vision as well at the productivity
of virtual teams. It has been shown that the success of virtual teams is critically
dependent on the support of upper management [19].
It is important that all members of the virtual team understand the relationships
between their time zones when arranging real-time communications (phone calls or video
conferences) with other members of the team. Nobody will find a meeting at 3:00 AM
very motivating. Maslow’s hierarchy of needs explains why team members that are
required to be off their normal sleep hours may under perform [37]. The team leader
should make sure that face-to-face meetings are not always held at one location [44].
This is so the burden of traveling can be shared between all members of the team. Due to
the differences in time zones and extra time it takes to travel, it is vital that the team
leader should always send out notices of the meetings well in advance [4], [14]. It is also
important to send out the minutes of the meeting to all of the team members involved
[14]. As long as the team has created awareness and discussed these differences during
the development of the team norms, the team should be able to manage these differences
- 11 -
effectively [47].
Finally, an important potential pitfall of virtual teams is that the organization
needs to make sure that they do not become a mere “broker of contract” [24], a situation
in which the competitive advantage leaves the company [1]. In essence, if the model of
the company becomes something that other businesses can recreate, i.e. exclusively hire
external employees, then the future of the company can not be guaranteed. If the
competitive advantage is contracted out, the company will have lost a significant part of
the exchange and as stated by Charles Handy “the search for wealth will have destroyed
wealth” [24].
The Learning Challenge
Why Simulations
Ilias Karasavvidis states that from Socrates’ point of view, teaching is basically
the transfer of knowledge from teacher to student [28]. From this perspective, when
answering the question, ‘Why Simulations?’, it is important to understand some of the
different ways to transfer information.
Figure 1 illustrates how different media technologies relate to interactivity and the
complexity of the knowledge being represented [48]. Traditional educational methods,
primarily text and discussion, are organized using linear flow. Although hypertext allows
the author to expand the content structure, the information presented is still very linear.
For example, one can click on different links on a website and learn the information
within each page in many different sequences, but the information learned on each page
is still very structured and does not provide feedback. The user has a choice of the
- 12 -
sequence, but any path selected is linear.
Simulations, on the other hand, provide feedback to the user. Transforming inputs
(or decisions) into outputs (or feedback) is one of the basic ideas of a simulation. These
outputs are the feedback that completes the learning cycle and these are not available in
other types of media.
Figure 1: Complexity of Knowledge vs. Nature of Interactivity [48]
It has been shown that learning activities that recreate real world experiences
encourage more learning [55]. Gütl and Pivec found that problems can be solved more
quickly and with less work when using multimedia with virtual tutors rather than with
traditional literature. They also found that interactivity promotes active learning [22].
Multimedia also has the ability to provide real time feedback to the student. Schumacher
believes that in order to learn a strategy the player must be able to see their results [1].
Michael Grace-Martin found that multimedia for educational purposes can increase the
mental engagement of the student with the information. It has also been shown that the
- 13 -
more engaged the student is, the more likely the information will be absorbed [20].
Multimedia products have been found to be more useful in developing critical thinking
skills than text-based products [16]. These are just a few of the reasons for developing a
training simulation for virtual teams.
Forced Empathy
The multimedia design used for this virtual team training simulation has been
described as a ‘paired-game’ design [48]. Schumacher developed this design to present
players with two distinct and contrasting strategies. This design guides the player to
experience the beliefs and interpretations of two contrasting strategies as presented by the
two different frames of references. It was shown to be effective in creating employee
attitude change. Bennett has discussed the concept of different frames of references. He
feels that his empathy model, experiencing another frame of reference, is an effective
communication strategy [3]. As shown in Figure 2, this simulation puts the player in the
‘A’ frame of reference in game 1 and puts the player in the ‘B’ frame of reference in
game 2. All of the positions in Figure 2 other than ‘A’ and ‘B’ are unstable. If the
player’s choices in game 1 are not in-line with the ‘A’ frame of reference the advisor will
push and guide the player back to the initial stable frame of reference. A similar effect
will happen in game 2 if the player makes a choice more in line with ‘A’ instead of ‘B’,
the advisor will resist and guide the player back to the stable frame of reference.
Schumacher [48] found that this type of simulation encourages the participants to learn
the alternate reality based on the cues presented by the advisor of the simulation.
Schumacher found that people often adopt familiar frames and seldom try contrasting
- 14 -
frames, which is implied with point C in Figure 2. The design of this simulation has the
ability to overcome this situation by establishing different stable frames of references to
match the vision of both game A and B.
Figure 2: A thermodynamic metaphor for Frame of Reference shift [45]
Simulation Advisor
It has been shown that the mere presence of a simulation advisor (virtual tutor)
can increase learning [7]. This simulation advisor has also been shown to increase both
the interest and motivation of the player [7]. Gütl and Pivec also found that simulation
advisors provide the player with assistance in problem-solving [22].
When students are unable to make the appropriate choice there is a negative
impact on learning [7]. Chen, Toh, and Fauzy suggest four types of coaching in
constructivist learning environments: providing motivational comments, monitoring and
regulating the players’ performance, provoking reflection, and asking questions [6]. The
simulation advisor in the Virtual Team Challenge simulation presents all of these
coaching types.
These authors have shown that having an advisor in the simulation will help the
- 15 -
student learn more efficiently. This is directly in-line with the fact that research has
shown that real world virtual teams need advisors in order to be successful.
Learning (Interfaces & Message)
Schumacher has identified four learning phases inherent in multimedia
simulations, as shown in Figure 3, and progression through these stages is an important
goal to guide the student to deep learning of the content [49]. A good design will allow
for deep learning (phase four). The Virtual Team Challenge has been designed to
minimize phase 1 so that true learning can begin. The simulation has arrows, flashing
buttons, and tutorials that are all intended to reduce or eliminate much of the learning
difficulties that are inherent with most simulations interfaces.
Another point is the importance of the developer understanding that player
cognition is distributed along the dimensions of both material and social [28]. The
material dimension refers to both mental and physical factors, while the social dimension
refers to the involvement with others, otherwise known as the player’s social network
[28]. While The Virtual Team Challenge does not allow the player to interact with other
players, the simulation does allow the player to build a social network. The second frame
of reference presented in paired-game simulations reinforces the need for the player to
develop a strong and broad social network.
- 16 -
Figure 3: Learning phases found in using simulations [48]
What Are The Learning Challenges
There are many unique challenges in facilitating learning with the use of a
multimedia simulation. Ferdig and Mishra found that humans tend to treat computers as
social actors and, therefore, may feel betrayed or even angry at them. They believe that
developers that lack this understanding may be unknowingly hindering the goals of the
simulation [18]. Anger can be counter-productive to learning. Ferdig and Mishra also
cite six different journal articles that explain the importance of designing the simulation
so that it is perceived as having positive traits. These traits include being polite,
trustworthy, competent, empathic, responsive, honest, cooperative, and demonstrating
emotion [2], [18], [31], [33], [1], [38], [39]. These traits are portrayed through the
player’s advisor and virtual team members.
Grace-Martin states that it is a commonly accepted principle in psychology that
humans have a limited working memory capacity and attention span [20]. Software that
places a large cognitive load on the player may cause the player to become overwhelmed
- 17 -
and their performance will decline [20]. It is important to challenge the player without
overwhelming [20]. Therefore, it is important that the interface to the simulation not
have a steep learning curve.
Another potential learning challenge to overcome is participants’ native biases.
Jose, Varez-Torres, Mishra, and Zhao found that participants unknowingly remembered
more content when they believed that a native of their own country rather than a non-
native programmer developed the multimedia even if the content was identical [27].
Their study showed that even multimedia simulations are prone to stereotyping. Since
the Virtual Team Challenge is composed of four different automated characters
representing four different cultures, it was critical to make sure that problems that occur
throughout the simulation don’t happen to only one of the characters. The simulation has
been developed so that potential problems are spread out equally among the different
characters. This will help to prevent the simulation from creating the appearance of one
culture being less productive than another.
What are the Benefits The ‘Paired-Game’ Model Benefit
The ‘Paired–Game’ concept employed for this project has participants play two
games in sequence. These games incorporate the same simulation model, the method that
transforms participants’ decisions into outputs. What is different is that participants
adopt contrasting cognitive frameworks, one in each game. These cognitive frameworks
are delivered via several simulation features, for example, a stream of recommendations
from a game character, who acts as the participant’s advisor, graphs of simulation
- 18 -
parameters such as productivity, and messages from simulated team members.
Participants’ interpretations of simulation goals, events, and feedback are filtered
through their cognitive frameworks. The cognitive frameworks presented in the
simulation are designed to present contrasts among key issues and participants’
experiences are, thus, different in the two games. This juxtaposition of contrasting
perspectives produces an opportunity for reflection on the attitudes and assumptions
participants adopt when engaged in the simulated situation [45]. The simulation model
and the two different point of views experienced by the participant work as a system to
present questions regarding their pre-existing beliefs, and ultimately to convey a deep
understanding of the content represented in the simulation.
The goal of this simulation is to show players two different perspectives of virtual
teams with the intent that they will walk away with a better understanding of what makes
a successful virtual team. The general model concept established within the Virtual
Team Challenge is similar to the idea of the Yin-Yang. This simulation utilizes two
contrasting viewpoints: Alpha and Beta. Each of these contrasting viewpoints are needed
for deep learning to occur. The first time through the simulation, the player has the
perspective from an ‘Alpha’ type company. This Alpha company doesn’t quite
understand the true benefits and hurdles of virtual teams and, therefore, doesn’t know
how to maximize the potential of the virtual team. The Alpha company may not know if
the player has made a poor choice and, therefore, will not be able offer much guidance.
The second time through the simulation, the player has the perspective from a ‘Beta’ type
company. This company understands how to maximize the benefits of virtual teams and
will help the player reach these goals.
- 19 -
Benefits from Design
The Virtual Team Challenge has been designed both to show the user many of the
difficulties that have been identified with virtual team as well as to provide guidance on
how to solve these difficult situations. A major design constraint for this Virtual Team
Training simulation was to keep total playing time under 3 hours. This allows the
simulation to be used more easily in academic settings. This simulation also has a
training video that the player watches before starting the simulation. This video walks
the player through interface of the simulation, thereby lowering the interface learning
curve.
One of the main design decisions for this simulation was to have completely
animated characters with vocalization in the dialogues. This helps to immerse the player
in the simulation and, therefore, increases the mental engagement and the richness of the
players’ experience. This also helps to speed up game play by simplifying the interface.
Having animated characters also helps to raise the level of enjoyment of the player,
which has been shown to increase learning. Lee and O’Leary concluded, “…that student
enjoyment is absolutely necessary to achieve the more profound kinds of learning
objectives addressed by simulation games [32].”
At the start of this simulation (Figure 4) the player enters information such as
their technical skills and interests and then begins work on a new project as the project
leader of a virtual team. The player has an advisor (Figure 5) along with 4 team
members. The player has the ability to schedule meetings (Figure 6), network with other
departments (Figure 7), and work on tasks. The meeting types from which the player can
- 20 -
choose are face-to-face meetings (Figure 8), video conferences (Figure 9), and phone
calls (Figure 10). The player has several tools to help him/her manage the project and
team, including detailed calendars with all local times (Figure 11), a project PERT chart
(Figure 12), and graphs of team member’s time allocation (Figure 13) meeting allocation
(Figure 14), and social parameters (Figure 15). After each of the two games the player
receives a short summary of their results (Figure 16). This will allow the player to step
back and see how utilizing different approaches can affect the outcome of the project.
This simulation presents the difficulties of virtual teams, such as the different
bandwidths for different meeting types or how some cultures may be more reluctant to
share concerns or problems. Different problem types, based on the documented pitfalls of
virtual teams, may occur during the simulation. Player success depends on how they
respond to problem situations.
- 21 -
Figure 4: Screenshot of the introduction page of the Virtual Team Challenge
Figure 5: Screenshot of advisor from the Virtual Team Challenge
- 22 -
Figure 6: Screenshot of scheduling meeting from the Virtual Team Challenge
Figure 7: Screenshot of networking from the Virtual Team Challenge
- 23 -
Figure 8 Screenshot of a face-to-face meeting from the Virtual Team Challenge
Figure 9: Screenshot for video conference from the Virtual Team Challenge
- 24 -
Figure 10: Screenshot of phone call from the Virtual Team Challenge
Figure 11: Screenshot of detailed calendar from the Virtual Team Challenge
- 25 -
Figure 12 Screenshot of PERT chart from the Virtual Team Challenge
Figure 13: Screenshot of time allocation from the Virtual Team Challenge
- 26 -
Figure 14: Screenshot of team meetings from the Virtual Team Challenge
Figure 15: Screenshot of team member variables from the Virtual Team Challenge
- 27 -
Figure 16: Screenshot of the finish screen from the Virtual Team Challenge
Development Platform
The development platform chosen for this simulation was Revolution Enterprise.
Cross-platform compatibility, object-based scripting, along with having a single code and
test environment are just a few of the reasons why this development platform was chosen.
Revolution Enterprise allowed me to code once and then simultaneously to easily
create standalone applications for Mac OS X and Classic, Windows, Linux, and Unix.
This dramatically reduced the amount of time required for the development and
debugging stages of this project.
One of the other very helpful features of Revolution Enterprise was the fact that
the scripts are compiled the second the user is done writing script. Revolution will send
- 28 -
the user an error message on the spot if there are compiling errors. This allowed me to
see syntax problems before I even left the script editor.
The Revolution environment has the ability with the click of a mouse switch
between development mode and testing mode. This feature helped me to identify easily
bugs as well as ways to improve the user interface.
All of the coding for this simulation was completed on Mac OS X. Another
feature that I thought was really helpful was the ability to change the appearance of the
simulation to mimic the look and feel of different operating systems.
The Virtual Team Challenge: Test Results
This simulation was presented to different classes at both Rose-Hulman Institute
of Technology and Indiana State University. Each of the participants was given a
questionnaire, as shown in Figure 17 and Figure 18, to fill out before the simulation. The
participants were only allowed a few minutes to fill out the questionnaire before the start
of the simulation. After this simulation the participants were given the exact same
questionnaire to fill out before leaving. The same amount of time was given for the
questionnaire before and after the simulation.
- 29 -
Figure 17: Part 1 of 2 of Questionnaire
- 30 -
Figure 18: Part 2 of 2 of Questionnaire
- 31 -
This simulation was presented to the participants with the use of a class projection
screen. The class as a whole vicariously became the project leader in this simulation.
My role was simply to facilitate their choices into the simulation. This allowed the entire
class to participate without the need to spend any time learning the interface of the
training simulation.
As the developer of this simulation there are many things that I may overlook.
There can be concepts that may be are clear to me but very confusion to everyone else.
For this reason, in addition to simply participating in the simulation, I also asked each of
the participants to critique the simulation. I asked them to let me know what they liked
and disliked about the simulation. The goal of this technique was to gather information
from the participant’s point of view so that in the future I may improve the quality of the
learning experience. Table 1 is a summary of the comments suggested by the
participants.
Comments 1) Audio levels of the speakers are inconsistent / level somehow 2) Hissing on the S's could be removed with audio editing software 3) It would be nice to be able to go back to schedule meeting instead of (inbox, network, or
work) 4) Volume on voices should be normalized 5) Let user skip audio? Some sounds were heard over and over. 6) More help tips (hover over socializing should explain what the benefits are / why
knowing their skills is good) 7) Some emails seemed useless. If they have uses please explain, lots of guesswork 8) Is there supporting documentation about the tips given by Dr. Beta – like, where do they
look if they’ve never heard of a Raci chart before? Maybe some of this could be a part of the programmed product, say, a table comparing Alpha’s advice and Beta’s advice, with things you can click on to get more explanation.
Table 1: Participants Critique
The data gathered for the analysis for this simulation was attained from 23
- 32 -
participants from three different presentations, one at Rose-Hulman Institute of
Technology and two at Indiana State University. For the analysis of this data I needed to
use methods that could handle small samples of data. The two methods employed were
the Two-Sample Test of Means [36] and The Kolmogorov-Smirnov Two-Sample Test
[53].
While 23 may be less than ideal, I was still able to uncover some statistical
changes. Table 2 shows the results from this analysis2.
2 X = Two-Sample Test of Means: Small Samples and Y = The Kolmogorov-Smirnov Two-Sample Test. Please see addendums ‘Detail Analysis of Two-Sample Test of Means: Small Samples’ and ‘Detail Analysis of The Kolmogorov-Smirnov Two-Sample Test’ for more information about the analysis results
- 33 -
Statistically Significant Changes
Questions X Y 1) I have NO experience or knowledge regarding virtual teams. Yes No 2) Developing trust among VIRTUAL team members is a critical success factor. Yes Yes 3) ENCOURAGING virtual team members to explore SOCIAL NETWORKS is a
recommended practice for team leaders. Yes Yes 4) AWARENESS of cultural DIFFERENCES among team members is a KEY
success factor for virtual teams No No 5) EMAIL is the OPTIMAL way to communicate on a VIRTUAL team. No No 6) When time is running out on a task, VIRTUAL team members should
communicate the situation with the team. No No 7) ALL virtual team members should be INVOLVED in creating and accepting the
project OUTLINE and TIMELINE. Yes No 8) TASK FOCUS in virtual teams will help to PREVENT problems during the
project. Yes Yes 9) It is IMPORTANT to SHARE your social and culture experiences with the
team. No No 10) During the BEGINNING of the project team members should spend MORE
time NETWORKING THAN working on TASK. Yes No 11) Face-To-Face meetings should be held at ONE location to maintain regularity. No No 12) Getting to know the members of the team on a PERSONAL level is important to
team success. Yes No 13) What percentage of time do you think should be spent in meetings during project
initiation? Yes No 14) What percentage of time do you think should be spent in meetings during the
middle of the project? No No 15) What percentage of time do you think should be spent in meetings during the end
of the project? No No
Table 2: Statistical Analysis
- 34 -
The initial results show with high certainty that this simulation does improve
many of the skills that have been shown to be important. It is important to note that
while the ‘Two-Sample Test of Means: Small Samples’ did not show statistically
significant changes on 7 of the 15 questions3, these results may change as more
participants take the Virtual Team Challenge.4
Potential Areas of Interest
There are a few areas that could truly benefit from ‘The Virtual Team Challenge’.
This simulation could be a useful training tool for companies currently using virtual
teams, companies interested in exploring virtual teams, textbook publishers, and the
universities that teach the future real world leaders.
When marketing directly to either companies that currently use virtual teams or
companies that are interested in exploring virtual teams there were two routes that could
be taken. The most direct route would be to hold presentations on-site and present the
simulation to a group of employees at once. This approach is similar to holding a
seminar but with a lot more interaction with the audience. The other route that could be
taken is to allow the companies to have access to the software for a set amount of time.
There are textbooks currently on the market that present information that falls
directly in line with the Virtual Team Challenge. Packaging a software training
simulation along with the textbook could help to leverage the particular textbook over the
3 The change in questions # 8 has been show to be statistically significant but the mean result went in the opposite direction as desired. After investigating the situation by speaking with many participants, I have determined that the question was poorly written and misleading. 4 While the changes that occurred in 6 of the 15 questions were not shown to be statistically significance, 14 of the 15 questions did show an average improvement.
- 35 -
competing textbooks. This method would also help to bring the simulation into another
area of potential interest, universities.
One professor at Rose-Hulman Institute of Technology has decided to use the
Virtual Team Challenge in future project management classes. Other professors may also
find having the Virtual Team Challenge an extremely helpful tool in installing deep
learning in their classes.
This simulation could also be marketed to companies that currently sell other
training simulations. By using this approach, the developers would simply look for one
of the potential competitors to buy them out.
The Current Playing Field
Training seminars, training videos, and interactive e-learning tools are just a few
of the types of methods that are utilized to train individuals on virtual teams. With all of
these methods available comes a plethora of companies that focus on training virtual
teams.
Synergest, Inc.5 and The Fisher Group6 are both companies that utilize training
seminars to teach company employees on virtual team methodologies. Although this
approach may be very effective, it also is very expensive. Synergest Inc. currently
charges from $4600 to $7300 for their 2-day workshop entitled Better Teamwork for
Better Performance: Communication and Group Dynamics. In addition, Synergest Inc.’s 5 Synergest Inc. corporate headquarters are located in Chicago, IL. More information regarding Synergest, Inc. can be found at http://www.synergest.com/ or by calling 888.772.6489. 6 The Fisher Group is located in Portland, OR. For more information on The Fishers Group’s ‘The Distance Manager’ refer to ‘Fisher Group Training Programs: The Distance Manager’ located in the addendum
- 36 -
workshop currently only addresses globally dispersed teams as a side note and not as the
main topic for the workshop. The Fisher Group chargers $8000, $375 per participant,
plus all trip expenses (travel, hotel, car rental, meals, cab fair, and gratuities) for their 2-
day face-to-face program entitled ‘The Distance Manger’. Within this training course
they provide a simulation entitled ‘Virtual Crossbow Simulation’.
Virtual Connection7 is another company that offers a training simulation on
virtual teams entitled Virtual@Work. This product is an interactive e-learning tool with
modules of different types of aspects of virtual teams. The biggest difference with this
simulation and The Virtual Team Challenge is the lack of choices the player is allowed to
make throughout the simulation. This simulation is designed very linear, similar to a
book with questions at the end of each module. This simulation costs $299 per person
per 6-month period.
Diversity Resources, Inc.8 takes a slightly different approach by focusing on
training videos. One of their training video’s is entitled ‘Building the Virtual Team’ has
a play time of 18 minutes and cost between $495 and $595. Although these training
videos may be informative, they do not allow much interaction with the end user.
Another type of potential competitor would be companies that currently have
training simulations but lack a virtual team simulation. This type of company has the
background and potentially the motivation to also develop a training simulation on virtual
teams. Interpretive Solutions is a company that could do just that.
7 For more information and a free trial version on Virtual Connection see http://www.virtualconnection.biz/ 8 For more information regarding Diversity Resources, Inc go to http://www.diversityresources.com/
- 37 -
Conclusion
In today’s environment, companies need to be able to leverage the best talent
regardless of location. It has been shown that these types of teams are prone to a unique
and challenging type of problems. The Virtual Team Challenge simulation has been
developed to provide training that can reduce or eliminate these problems. High
performing teams are the key and The Virtual Team Challenge provides the map to the
door. Are you ready for training?
References
[1] Allen, K. R. (2006). Launching new ventures. 4th ed. Boston: Houghton Mifflin
[2] Bates, J. (1994). “The role of emotion in believable agents.” Communications of the
ACM, 37(7), 122-125
[3] Bennett, Milton 1986. “Towards Ethnorelativism: A Developmental Model of
Intercultural Sensitivity”, in Cross Cultural Orientation: New Conceptualizations
and Applications, Paige, Michael (ed.), University Press of America, 27-69.
[4] Cascio, W. F. & Shurygailo, S. (2003). “E-Leadership and virtual teams.”
Organizational Dynamics, 31, 362-376.
[5] Champy, J. & Nohria, N. (1997). Fast forward: The best ideas on managing
business change. Boston: Harvard Business School Press.
[6] Chen, C. J., Toh, S. C. & Fauzy, W. M. (2004). “The theoretical framework for
designing desktop virtual reality-based learning environment.” Journal of
Interactive Learning Research, 15, 147-167.
[7] Clarebout, G., Elen, J., Johnson, W. L., & Shaw, E. (2002). “Animated pedagogical
- 38 -
agents: An opportunity to be grasped?” Journal of Educational Multimedia and
Hypermedia, 11, 267-286.
[8] Collins, L. (2006, August/September). Build it and they will think? IET Engineering
Management, 14-17
[9] Cook, P. (2006, August/September). Clubbing together. IET Engineering
Management, 9
[10] Cummings, J. N. (2001). Work groups and knowledge sharing in a global
organization. Academy of Management Proceedings, OB, D1-5.
[11] Dahlin, K. B., Weingart, L. R. & Hinds, P. J. (2005). Team diversity and
information use. Academy of Management Journal, 48, 1107-1123.
[12] Desouza. K., Awazu, Y. (2006, August/September). Contingent knowledge workers.
IET Engineering Management, 18-19
[13] Devine, D. J. (2002). A review and integration of classification systems relevant to
teams in organizations. Group Dynamics: Theory, Research, and Practice, 6, 291-
310.
[14] Duarte, D. L., & Snyder, N. T. (2006). Mastering virtual teams. San-Francisco:
Jossey-Bass Publishers.
[15] Edmondson, A. 2002. The local and variegated nature of learning in organizations:
A group-level perspective. Organization Science, 13: 128-146.
[16] Ellis, T. J. (2001). Multimedia enhanced educational products as a tool to promote
critical thinking in adult students. Journal of Educational Multimedia and
Hypermedia, 10, 107-123.
[17] Frame J. D. (1998). Closing out the project. L. J. Pinto (Eds.), Project Management
- 39 -
Handbook (pp. 237-246). San Francisco, CA: Jossey-Bass
[18] Ferdig, R. E., & Mishra, P. (2004). Emotional responses to computers: Experiences
in unfairness, anger, and spite. Journal of Educational Multimedia and Hypermedia,
13, 143-161.
[19] Furst, S. A., Reeves, M., Rosen, B., & Blackburn, R. S. (2004), Managing the Life
Cycle of Virtual Teams, Academy of Management Executive, 13, 6-20.
[20] Grace-Martin, M. (2001). How to design educational multimedia: a loaded question.
Journal of Educational Multimedia and Hypermedia, 10, 397-409.
[21] Gray, B., Susman, G. L., & Ren, H. (2005). Brokers’ roles in knowledge
management in teams: Expertise, influence and schema conflicts. Academy of
Management Best Conference Paper, CM, B1-5.
[22] Gütl, C., & Pivec, M. (2003). A multimedia knowledge module virtual tutor fosters
interactive learning. . Journal of Interactive Learning Research, 14, 231-258.
[23] Hall, Edward T., (1990). The silent language (Rev. ed.). New York, NY: Anchor
Books
[24] Handy, C. (1995). Trust and the virtual organization. Harvard Business Review, 73,
40-50.
[25] Herrington, J., Oliver, R., & Reeves T. C. (2003). Patterns of engagement in
authentic online learning environments. Australian Journal of Educational
Technology, 19, 59-71.
[26] Jarvenpaa, S. L., & Leidner, D. E. (1999). Communication and trust in global virtual
teams. Organization Science, 10, 791-815.
[27] Jose M., Varez-Torres, A., Mishra, P., & Zhao, Y. (2001). Judging a book by its
- 40 -
cover! Cultural stereotyping of interactive media and its effects on the recall of text
information. Journal of Educational Multimedia and Hypermedia, 10, 161-183.
[28] Karasavvidis, I. (2002). Distributed cognition and educational practice. . Journal of
Interactive Learning Research, 13, 11-29.
[29] Kirkman, B. L., Rosen, B., Tesluk, P. E., & Gibson, C. B. (2004). The impact of
team empowerment on virtual team performance: The moderating role of face-to-
face interaction. Academy of Management Journal, 47, 175-192.
[30] Kirkman, et al, (2002), Five Challenges to virtual teams: Lessons from Sabre, Inc.,
Academy of Management Executive, August
[31] Laurel, B. (1997). Computers as theatre. Reading, MA: Addison-Wesley.
[32] Lee, Robert S., and O’Leary, Arlene 1971. “Attitude and Personality Effects of a
Three-day Simulation”, Simulation and Games: An International Journal of Theory,
Design and Research, 2:309-347.
[33] Lepper, M., & Chabay, R. (1987) Socializing the intelligent tutor: Bringing empathy
to computer tutors. In H. Mandl & A. Lesgold, (Eds.), Learning issues for
intelligent tutoring systems, (pp. 242-257). New York: Springer-Verlag.
[34] Levesque, L. L., Wilson, J. M., & Wholey, D. R. (2001). Cognitive divergence and
shared mental models in software development project teams. Journal of
Organizational Behavior, 22, 135-144.
[35] Maes, P. (1997). Intelligent software. Proceedings of IUI ’97: International
Conference on Intelligent User Interfaces, (pp. 41-43).
[36] Mansfield, E. (1983). Statistics for business and economics: Methods and
applications. 2nd ed. New York: W.W. Norton & Company
- 41 -
[37] Maslow, A. (1954). Motivation and personality. New York: Harper.
[38] Mishra, P., & Hershey, K. (2002). A framework for designing etiquette for
educational technology. In C. Miller (Ed.). Etiquette for human-computer work.
Proceedings of the American Association of Artificial Intelligence, Fall Symposium.
Technical Report FS-02-02, (pp. 68-71).
[39] Mishra, P., & Hershey, K. (2004). “Human-computer etiquette: managing
expectation with intentional agents: Etiquette and the design of educational
technology.” Communications of the ACM, 47, 45-49
[40] Montoya-Weiss, M. M., Massey, A. P., & Song, W. (2001). Getting it together:
Temporal coordination and conflict management in global virtual teams. Academy
of Management Journal, 44, 1251-1262.
[41] Nguyen, T. V., Weinstein, M., & Meyer, A. D. (2005). Development of trust: A
study of interfirm relationships in Vietnam. Asia Pacific Journal of Management,
22, 211-235.
[42] Nohria, N., R. G. Eccles. (1992). Face-to-Face: Making network organizations
work. N. Nohria, R. G. Eccles, eds. Network and Organizations. Harvard Business
School Press, Boston, MS 288-308
[43] O’Hara-Devereaux, M., R. Johansen. (1994). Global Work: Bridging Distance,
Culture, and Time. Jossey-Bass San Francisco, CA.
[44] Offshoring to India: Best practices. (2005). Global Skills Update: A Monthly
Publication 32
[45] Poehler, L. M., & Schumacher, T. R. (2007). The Virtual Team Challenge: Is it time
for training? PICMET 2007 Conference proceedings.
- 42 -
[46] Romahn E., F. Hartman. (1999, October). Trust: A new tool for project managers.
Symposium conducted at the annual project management institute, Philadelphia,
Pennsylvania.
[47] Saunders, C., Van Slyke, C., & Vogel, D. R. (2004). My time or yours? Managing
time visions in global virtual teams. Academy of Management Executive, 18, 19-31.
[48] Schumacher, T. R., 1992, “Simulation Design, Role Identification and Attitude
Change in a High Technology Culture”, doctoral dissertation in Systems Science,
Portland State University.
[49] Schumacher, T. (1998). B820 Residential School Observations and Critique. Memo
[50] Schumacher, Terry. (1999). Open University Business School course B823,
Knowledge Management, Book 2 Communication.
[51] Schumacher, T. (2006). Designing educational simulations to teach frames of
reference. Unpublished manuscript.
[52] Senge, P. M. (1990). The leader’s new work: Building learning organizations. Sloan
Management Review, 32, 7-23.
[53] Siegel, S. (1956). Nonparametric statistics for the behavioral sciences. New York:
McGraw-Hill Book Company
[54] Tuckman, B. (1965). Developmental sequence in small groups. Psychological Bulletin,
63, 384-399.
[55] Swanson, R. A. & Holton, E. F. (1999). Results: How to assess performance,
learning, and perceptions in organizations. San Francisco: Barrett-Koehler.
[56] Von Glinow, M. A., Shapiro, D. L., & Brett, J. L. (2004). Can we talk and should
we? Managing emotional conflict in multicultural teams. Academy of Management
Review, 29, 578-592.
- 43 -
[57] Wallace, P. (2004). The internet in the workplace: How new technology is
transforming work. Cambridge: Cambridge University Press.
[58] Zellmer-Bruhn, M., & Gibson, C. (2006). Multinational organization context:
Implications for team learning and performance. Academy of Management Journal,
49, 501-518.
- 44 -
Addendums Revolution Enterprise
- 45 -
Revolution: Benefits
- 46 -
- 47 -
Revolution Media and Studio Features
The features shown below for Revolution Studio are very similar to the features with Revolution Enterprise. While Revolution Enterprise has a few more features, the document below showcases many of the features built into Revolution Enterprise.
- 48 -
- 49 -
- 50 -
- 51 -
- 52 -
- 53 -
Revolution - FAQ
- 54 -
- 55 -
- 56 -
Detail Analysis of Two-Sample Test of Means: Small Samples
The column entitled ‘Results Improved?’ simply answers the questions of whether the average results from the questionnaire before vs. the questionnaire after moved closer to the optimal choice.
Question Observations XA XB SD
Before SD
After SD2
Before SD2
After Variance F statistic Significant Change?
Results Improve?
1 22 4.77 3.27 2.29 1.78 5.23 3.16 4.20 2.43 Yes Yes 2 23 5.48 6.78 1.44 0.42 2.08 0.18 1.13 -4.16 Yes Yes 3 23 4.83 6.35 1.44 0.83 2.06 0.69 1.38 -4.40 Yes Yes 4 23 5.39 6.04 1.64 1.07 2.70 1.13 1.92 -1.60 No Yes 5 23 4.04 3.22 1.77 1.62 3.13 2.63 2.88 1.65 No Yes 6 23 6.17 6.39 1.27 0.72 1.60 0.52 1.06 -0.71 No Yes 7 23 5.04 6.26 2.08 1.01 4.32 1.02 2.67 -2.53 Yes Yes 8 23 4.57 5.57 1.34 1.44 1.80 2.08 1.94 -2.44 Yes No 9 23 5.09 5.83 1.47 1.03 2.17 1.06 1.62 -1.97 No Yes 10 23 4.74 5.91 1.71 1.50 2.93 2.26 2.60 -2.47 Yes Yes 11 23 3.65 2.78 1.85 1.76 3.42 3.09 3.25 1.63 No Yes 12 23 4.96 5.96 1.77 1.02 3.13 1.04 2.09 -2.35 Yes Yes 13 5 4.60 6.40 1.52 0.89 2.30 0.80 1.55 -2.29 Yes Yes 14 5 4.00 4.00 1.22 0.71 1.50 0.50 1.00 0.00 No Yes
15 5 3.20 3.60 1.30 1.14 1.70 1.30 1.50 -0.52 No No Change
