Embed Size (px)
Citation preview
1 Copyright © 2015 by ASME
Proceedings of the ASME 2015 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference
IDETC/CIE 2015 August 2-5, 2015, Boston, MA, USA
DETC2015- 47388
Investigating the Impact of Interactive Immersive Virtual Reality Environments in Enhancing Task Performance in Online Engineering Design Activities
Ajay Karthic B. Gopinath Bharathi Industrial and Manufacturing Engineering
The Pennsylvania State University University Park, PA 16802 Email: [email protected]
Conrad S. Tucker Engineering Design and Industrial and Manufacturing Engineering
Computer Science and Engineering The Pennsylvania State University
University Park, PA 16802 Email: [email protected]
ABSTRACT The objective of this paper is to test the hypothesis that
immersive virtual reality environments such as those
achievable through the head-mounted displays, enhance task
performance in online engineering design activities. In this
paper, task performance is measured by the time to complete a
given engineering activity. Over the last decade, a wide range
of virtual reality applications have been developed based on
non-immersive and immersive virtual reality systems for
educational purposes. However, a major limitation of non-
immersive virtual reality systems is the lack of an immersive
experience that not only provides content to students, but also
enables them to interact and learn in a completely 360 degree
immersive environment. The authors of this work have
developed a replica of a physical engineering laboratory in an
interactive virtual learning environment. This research
measures the difference in task performance between i)
students exposed to an immersive virtual reality system and ii)
students exposed to a non-immersive virtual reality system, in
the interactive virtual environment developed by the research
team. This research seeks to explore whether statistically
significant differences in performance exist between these
groups. Knowledge gained from testing this hypothesis will
inform educators about the value and impact of immersive
virtual reality technologies in enhancing online education. A
case study involving 54 students in a product functional
analysis task is used to test the hypothesis.
1 INTRODUCTION
Virtual reality (VR) is a computer generated virtual
environment which gives users the impression of being
somewhere other than where they actually are. VR can be
considered as an advanced form of human-computer interface
that allows the user to interact with a realistic virtual
environment generated using interactive software and
hardware. A more formal definition of VR was proposed by
Moshell and Hughes, where they define VR “as a real-time
graphical simulation with which the user interacts via some
form of analog control, within a spatial frame of reference and
with user control of the viewpoint’s motion and view direction”
[1]. The primary characteristics of VR which differentiates it
from other means of displaying information are the concepts of
immersion and presence. The VR research community is
divided on its opinion about the definitions of immersion and
presence. Slater and Wilbur defined immersion based on the
description of what any technology could provide [2]. I.e., the
extent to which the computerized system is capable of offering
an inclusive, extensive, surrounding and vivid illusion of
reality to the senses of a user. According to Slater and Wilbur,
“Inclusive indicates the extent to which physical reality is shut
out. Extensive indicates the range of sensory modalities
accommodated. Surrounding indicates the extent to which this
virtual reality is panoramic rather than limited to a narrow field.
Vivid indicates the resolution, fidelity, and variety of energy
simulated within a particular modality”. They also define
presence “as a state of consciousness, the (psychological) sense
of being in the virtual environment”. It is argued that a virtual
environment that produces a greater sense of immersion will
produce higher levels of presence [3]. At times, immersive VR
2 Copyright © 2015 by ASME
environments are confused with the traditional 3D desktop
environments. Van Dam et al. indicate that immersive VR
systems are substantially new media that are different from the
3D desktop environments since “conventional desktop 3D
displays give one the sensation of looking through a window
into a miniature world on the other side of the screen, with all
the separation that sensation implies, whereas immersive VR
makes it possible to become immersed in and interact with life-
sized scenes” [4]. Therefore, immersive VR systems have the
ability to let users “see” things which they could not see with
the desktop 3D graphics.
VR environments can be generated by integrating a
collection of hardware such as a computer, keyboard, mouse,
head-mounted display, headphone or motion sensing gloves,
with necessary software systems. Mujber et al. classify VR
systems based on immersion into three categories namely: non-
immersive (desktop) systems, semi-immersive projection
systems and fully immersive systems as shown in Table 1 [5].
Fully immersive VR systems allow for mobility by capturing
human motion and interaction with objects, and reproducing
the same objects in the virtual 3D environments [6]. Non-
immersive/desktop VR also places the user in a 3D
environment and allows interaction, but only utilizing a
conventional graphic workstation with a monitor, keyboard
and mouse [7]. Semi-immersive VR is another type of VR
system that exists at the intersection between non-immersive
and fully immersive VR systems. Fish tank VR on a monitor
[8], workbenches [9] and single wall project displays [10], all
with head-tracked stereo, are examples of semi immersive VR
systems. In this paper, the term “immersive VR system” is
synonymous to a “fully immersive VR system”. Advancements
in computer hardware technology and internet connectivity
have made VR increasingly popular and less expensive.
According to a recent report [11], the VR market is expected to
grow to $407.51 million and reach more than 25 million users
by 2018. This increased demand is attributed to a wide
spectrum of application areas for these technologies outside of
the gaming world. Today, VR technologies are used in a broad
range of fields such as education, military, healthcare,
entertainment, engineering, sports, and much more [12–17].
For example, engineers at companies like Caterpillar, General
Motors and Ford Motors use VR technology to virtually create
products and test design during the automotive development
[18].
VR technology provides a suitable platform in the
educational settings for the learner to participate in the learning
environment with a sense of being a part of it. Pantelidis
provides several reasons for the use of VR in education, such
as enhanced visualization, motivation, interaction and
collaboration [19]. Mikropoulos and Natsis reviewed 53
empirical studies related to the educational application of VR
and found that several studies showcased the positive attitude
of students and teachers towards the use of VR in educational
settings [20]. They also pointed out that many studies
demonstrated the potential of virtual environments as a tool to
enhance students’ understanding of concepts and reduce
misconceptions. VR systems have been shown to be an
effective and promising tool in education because of their
unique characteristics. One of its unique characteristics is its
ability to allow students to visit virtual environments and
interact with objects and space in real time, which overcomes
the traditional distance, time, or safety constraints [21-22]. VR
can be pedagogically exploited through its special
characteristics such as three dimensional spatial
representations, multisensory channels for user interaction,
immersion and intuitive interaction through natural
manipulations in real time. [23]. There is general consensus
among educational technologists about the need for
interactivity in learning [24].
Table 1. TYPES OF VR SYSTEMS OUTLINED BY MUJBER
ET AL. [5]
VR system
Non-
immersive
VR
Semi-
immersive
VR
Fully-
immersive
VR
Input
devices
Mice,
keyboards,
joysticks,
and
trackballs
Joystick space
balls, and data
gloves
Gloves and
voice
commands
Output
devices
Standard
high-
resolution
monitor
Large screen
monitor, large
screen
projector
system, and
multiple
television
projection
systems
Head
mounted
display
(HMD),
CAVETM
Resolution High High Low–
medium
Sense of
immersion Non-low Medium–high High
Interaction Low Medium High
Price
Lowest
cost VR
system
Expensive Very
expensive
Constructionism is one of the most widely accepted
theories of learning and education in the 21st century. It is
generally recognized in educational domains that an essential
part of the learning process lies in “hands-on" construction.
The constructivism learning theory argues that people produce
knowledge and form meaning based upon their experiences
[25]. The constructivism learning theory suggests that the
construction of an individual’s new knowledge is based on two
concepts namely, assimilation and accommodation.
Assimilation describes how the learners deal with new
knowledge, while accommodation describes how learners
reorganize their existing knowledge [26]. Discovery learning
is a constructive based approach to education which is defined
3 Copyright © 2015 by ASME
as “an approach to instruction through which students interact
with their environment – by exploring and manipulating
objects, wrestling with questions and controversies, or
performing experiments” [27]. Researches have established the
compatibility between characteristics of VR technology and
constructivist learning theory [28-29]. Recent developments in
technology and internet connectivity have dramatically
increased the accessibility to education through Massive Open
Online Course (MOOC) platforms. The ease of accessibility
and quality of the course materials have in turn, increased the
popularity of MOOCs [30]. MOOC predominantly utilizes
videos, texts and images for teaching purposes [31-32]. Tucker
et al. proposed a methodology for mining textual data
generated within MOOCs in order to quantify students’
sentiments pertaining to course topics [33]. Though there are
several advantages of MOOCs, the inability to deliver practical
hands-on experience is seen as a major drawback, especially in
technical courses associated with engineering. The VR
systems, when integrated with such MOOC platforms can
possibly overcome this drawback.
The paper is organized as follows: Section 2 reviews
related research, Section 3 outlines the research hypothesis and
methodology, Section 4 describes the case study, and Section
5 discusses the conclusions and future directions.
2 LITERATURE REVIEW
Since this study involves comparing non-immersive VR
systems with immersive VR systems, the research articles
discussed in this section are limited to such systems.
2.1 Immersive Vs. Non-Immersive Virtual Reality Systems
The non-immersive/desktop VR systems have been
implemented in a variety of applications such as industrial
training [34], preflight navigation training [35], rehabilitation
of functional activities in individuals with chronic stroke [36],
visualizing geographical data [37] and astronomy education
[38]. As explained earlier, the immersive VR systems need a
more sophisticated setup to “immerse” the user in the computer
generated 3D world. Head-mounted display based systems are
one of the oldest types of immersive VR systems where the
virtual environment is displayed via a head-mounted visual
display unit and user interaction with the virtual environment
takes place through a hand held input device. Another type of
immersive VR system is the Immersive Project Technology
(IPT), where the images are projected on single or multiple
screens around the user. CAVETM is an example of such a
system, where the user wearing a Stereoscopic LCD shutter
glasses is kept in a cubical room where the virtual environment
is projected on multiple surfaces, thereby giving the user a 3D
feel of the virtual world. CAVETM systems have been found to
be used in the treatment of acrophobia [39] and understanding
molecular structure in chemistry education [40]. These types of
immersive VR technology have the ability to provide an
interactive and stimulating platform for students and create an
online learning experience similar to the one which they might
experience in an actual university.
2.2 Application of Virtual Reality in Education
Studies have been conducted to investigate the preferences
of students towards digital/virtual and hands-on/tactile learning
in engineering [41-42]. Previous research has shown that three-
dimensional virtual worlds provide a better representation of
the real world and are more effective than text-based or two-
dimensional environments, and the increased degree of realism
can lead to better student engagement in learning activities
[43]. It has also been reported that virtual environments like
computer games can motivate students to engage in learning
activities and achieved promising results [44-45]. Researchers
have investigated the use of VR for educating students of
different ages in various fields and found positive performance
and learning outcomes.
Immersive Virtual Reality Systems in Education:
In the field of biology, Allison et al. created “Virtual
Reality Gorilla Exhibit”, an educational tool for middle school
students wherein the users assumed a character of an
adolescent gorilla to explore a virtual gorilla habitat, to learn
about the gorillas’ interactions, vocalizations and social
structures [46]. In chemistry, Limniou et al. showed that
CAVETM systems enabled students to get a clearer
understanding of molecular structures and their changes during
a chemical reaction when compared to 2D animation on
desktop computers [40].
Non-Immersive Virtual Reality Systems in Education:
In mathematics, VR has shown positive effects on students
learning geometry topics [47-48]. Kim et al. used an education
tool called Virtual Reality Physics Simulation (VRPS) for
visualization of key physics concepts such as wave
propagation, ray optics, relative velocity and electric machines
at high school or college level [49]. The survey results of the
study indicated that students were more satisfied and felt that
VR helped them understand the concepts better. Another
example for the application of VR in physics was demonstrated
by Demaree et al. where three VR-based labs dealing with
linear motion, circular motion and collision were created for
the introductory physics laboratories [50]. VR systems are
being used extensively in the field of astronomy education
since it is not possible for students to visit space to learn
astronomical phenomena. One such VR astronomy system is
the “Virtual Solar System” where undergraduate students build
a solar system using 3D modelling software [51]. During the
process, students were found to gain rich understanding of the
astronomical phenomena. The Harvard Law School has offered
virtual courses in Second Life, in which a specific set of
students could participate in the lectures and interact with the
instructor inside the virtual environment [52]. The overall
response was found to be positive. One of the most important
application of VR is in the education and training of medical
professionals. VR allows medical professionals to recreate the
operating room for ongoing training purposes. Ahlberg et al.
demonstrated that VR training significantly reduced the error
rate of residents during their first 10 laparoscopic
4 Copyright © 2015 by ASME
cholecystectomies [53]. Another similar study concluded that
VR surgical simulation improved the operating room
performance of residents during laparoscopic cholecystectomy
[54].
Though there has been a modest amount of research
conducted using non-immersive VR systems in the educational
setting, the exploration of immersive VR systems has been
limited. Mikropoulos and Natsis performed a ten year (1999-
2009) review of empirical studies related to educational virtual
environments and found that only 16 of the 53 studies used
either immersive or semi immersive VR systems [20]. The
authors of this work investigate the effectiveness of immersive
VR technologies in comparison to non-immersive VR
technologies. Though non-immersive VR or desktop VR
system are less expensive to implement, they are considered as
one of the least immersive implementations of VR techniques
since the virtual environment is viewed only through a flat
screen monitor. On the other hand, immersive VR systems
provide a more immersive and realistic experience by placing
the user in a virtual environment that looks and feels closer to
the real world. Other notable advantages of immersive VR
system are its ability to enhance the perception of depth, sense
of space and interaction with three dimensional objects. There
is a need to explore the effectiveness of VR technology in the
educational setting and measure the users’ performance when
engaging in the VR environment [55]. Many studies have
explored the use of VR systems for learning activities, but to
the best knowledge of the authors, this is the first study that has
been carried out in the direction of evaluating users’
performance in immersive VR systems versus non-immersive
VR systems for online engineering design activities.
3 METHODOLOGY
This section outlines a methodology to evaluate the
differences in performance outcomes between students using
immersive and non-immersive VR systems (Fig. 1). In this
methodology, performance is measured based on time to
complete a given engineering activity.
Figure 1. OUTLINE OF METHODOLOGY
The most important differentiating feature of immersive
VR systems is the immersion produced by stereo 3D visual
display using head/body tracking devices, which creates a
human centric rather than a computer-determined point of
view. Therefore, in immersive VR systems, the virtual
environment continually adjusts to retain the viewer’s
perspective. Immersive VR systems such as head-mounted
displays can track the users’ translational motions (x,y,z) as
well as the rotational motions (yaw, pitch, roll), thereby
allowing tracking in 6 degrees of freedom (Fig. 2).
Figure 2. SIX DEGREES OF FREEDOM ACHIEVED USING IMMERSIVE VR SYSTEMS
3.1 Research Hypothesis
Task completion time has been used as a performance
metric to evaluate the effectiveness of VR technology in
research. Jennett et al. showed that immersion can be measured
subjectively (through questionnaires) as well as objectively
(task completion time, eye movements) [56]. Lendvay et al.
used task completion time as one the metric to show that
preoperative robotic warm-up using VR systems enhanced
robotic task performance among surgeons [57]. Newmark et al.
used task completion time to study the correlation in the
assessment of laparoscopic surgical skills among medical
students using a VR laparoscopic trainer and a low-fidelity
video box trainer [58]. In an attempt to assess the feasibility of
VR with hand-held devices, Hwang et al. evaluated the
effectiveness of different VR platforms using task completion
time in one of their experiments [59]. Ni et al. found that large
display size (i.e. large field of view and resolution (number of
pixels) have a significant effect on the time taken to complete
a task in information rich virtual environments [60]. Since
immersive VR systems generally possess better field of view
and resolution, this research will explore whether they have a
significant impact on the task completion time in engineering
related design activities.
5 Copyright © 2015 by ASME
In this paper, a hypothesis based on task completion time
is tested initially and then a questionnaire is used to evaluate
the preference of students towards immersive and non-
immersive VR systems. The following is the hypothesis to be
tested:
Null Hypothesis H0: Time taken by students to complete the
product functional analysis using immersive VR system and
non-immersive VR system are identical
Alternative Hypothesis Ha: Time taken by students to
complete the product functional analysis using immersive VR
system is less than the time taken by the students to complete
the same activity using non-immersive VR system
If the null hypothesis is rejected, it can be argued that
immersive VR systems are better than the non-immersive VR
systems in terms of performance outcomes for a given task. A
case study involving 54 students is presented in the next section
to test this hypothesis.
4 CASE STUDY
Product functional analysis, whether it is product
dissection or assembly, is an integral part of understanding the
engineering design process that has been shown to enhance
students’ understanding of products, their components, and the
interactions that exist between them [61–64]. The Subtract and
Operate Procedure (SOP) is an integral part of the learning
process during product dissection, as participants get an
opportunity to disassemble a component, operate the system,
analyze the effect of the removed component and infer the
function of the removed component [64–66]. Bohm and Stone
have created advanced design repository systems for virtual
product representations [67]. Devendorf and Lewis have
explored the use of digital product repositories in enhancing
product functional analysis activities [62]. Vasudevan and
Tucker proposed an approach to digitally represent physical
design artifacts using low cost 3D scanning techniques [68].
The experiment presented in this paper, advances the
research on virtual product analysis in engineering design by
investigating whether immersive VR systems enhance
students’ performance in product functional analysis tasks,
beyond the traditional non-immersive approaches to virtual
prototyping and design. In this study, a product functional
analysis is performed by two different groups of students,
where one group uses the immersive VR system and another
group uses a non-immersive VR system. The non-immersive
VR group uses a game joystick to perform the activity in the
virtual environment by looking at the computer desktop and the
immersive VR group uses the same game joystick along with a
360 degree visual immersive tool to perform the same task in
the same virtual environment. The specific product functional
analysis task is to assemble the components of a coffee maker
in the correct manner, given a baseline coffee maker as a
reference model.
4.1 Research Environment
The entire study was carried out in the Design Analysis
Technology Advancement (D.A.T.A) Laboratory at Penn State
University. A total of six work stations were set up for the study
with three of them allocated to immersive VR systems and the
other three allocated to non-immersive VR systems. Figure 3
shows the laboratory setup of the immersive VR systems.
4.2 Virtual Reality System Hardware
Oculus Rift® is an immersive VR head-mounted display
that allows users to be immersed in a three-dimensional
environment. The set of lenses on the device creates a
stereoscopic 3D image and the external position tracking
accessory tracks the movement of an individual’s head
accurately which together, creates a 3D environment for the
user (Fig. 2). Recent researches have shown that Oculus Rift®
can be used as a suitable VR tool in education [69-70]. Oculus
Rift Development Kit 2 with a resolution of 960 X 1080 per
eye was used for the purpose of this study. Afterglow Wired
Controller for Xbox 360 (PL 3702) was used as the joystick to
interact with the objects in the virtual environment. The
workstation was configured with an Intel® Core ™ i5 – 4690
CPU @ 3.5 GHz processor and 8GB RAM.
Figure 3. LABORATORY SETUP FOR IMMERSIVE VR SYSTEMS
4.3 Virtual Reality Software
The virtual environment for the study was created by
researchers in the D.A.T.A lab using Unity 4 (Version 4.6.1f1).
Unity is a cross-platform game creation system which includes
a game engine and an integrated development environment
[71]. It is a flexible system that can be used to develop video
games for web sites, desktop platforms, consoles and mobile
devices, and supports over fifteen platforms. Additionally, it is
particularly popular in the independent game development and
research communities due to its low cost and ease of use.
4.4 Virtual Reality Environment
The virtual environment consisted of a classroom with a
work table, on which the students had to perform the product
functional analysis. The product used in this case study was a
coffeemaker. The students had to look at the model coffee
6 Copyright © 2015 by ASME
maker at the right and assemble a similar coffee maker using
components on the left. Figure 4 shows the virtual environment
used in the study and Figure 5 shows the left and right eye
images of the same virtual environment produced by Oculus
Rift®.
Figure 4. VIRTUAL REALITY ENVIRONMENT
Figure 5. LEFT AND RIGHT EYE IMAGES OF THE IMMERSIVE VR ENVIRONMENT IN OCULUS RIFT®
4.5 EXPERIMENTAL STUDY
This study was conducted in accordance with Penn State
University’s Institutional Review Board (IRB) under protocol
ID #1698 titled "The Digital Divide: Investigating Virtual
Reality Technologies That Help Bridge the Gap Between
Virtual and Brick & Mortar Learning”.
4.5.1 Study Sample. 54 engineering undergraduate
students were recruited by advertisement in the university
campus, and assigned randomly to one of two groups. Group 1
(29 students) participated in the activity using immersive VR
system – Oculus Rift® and Group 2 (25 students) participated
in the same activity using a non-immersive VR system. In
terms of gender, 47 were male (87%) and 7 were female (13%).
Ages ranged from 17-27 years, the mean age being 19.85 years
(Standard deviation = 1.62). 50 students (93%) who
participated in the study had at least 1-2 years of experience in
using joysticks and 24 students (83%) in group 1 were first
time Oculus Rift® users.
4.5.2 Study Design. 54 students were split into two groups
in a random manner. Before the start of the actual experiment,
both the groups were given a pre-experiment questionnaire
where they had to answer questions about expected learning
outcomes in VR systems. The students were provided with an
instruction manual on how to use the joystick. Ten minutes of
training time was then allotted to each student to get
accustomed with the joystick and the actual virtual
environment used in the experiment. The first group of students
performed the activity with the immersive VR system (Oculus
Rift® + Joystick as shown in Fig. 6) and the second group of
students used the non-immersive VR system (Desktop Monitor
+ Joystick as shown in Fig. 7) to perform the same activity.
Figure 6. STUDENT USING IMMERSIVE VR SYSTEM (OCULUS RIFT®)
Each student was asked to repeat the activity three times,
with an average used for their task completion time. The task
completion time of each student was measured using a digital
clock embedded within the virtual environment. The students
were asked to perform the activity at their own comfortable
pace. Upon completion of the activity, the students were given
a post-test questionnaire with the same set of questions in the
pre-experiment, which they now answered based on their VR
experience. In the pre-experiment and post-experiment
questionnaire, students were asked to circle a response ranging
from 1 to 5 that best characterizes how they feel about each
statement given in the questionnaire, where 1 = Strongly
Disagree, 2 = Disagree, 3 = Neither Agree Nor Disagree, 4 =
Agree, and 5 = Strongly Agree (5 Point Likert Scale).
7 Copyright © 2015 by ASME
Figure 7. STUDENT USING NON-IMMERSIVE VR SYSTEM
4.5.3 Study Results. The statistical analysis was carried
out using Minitab 17. From Table 2, it can be observed that
median task completion time of students using the immersive
VR system is lesser than that of students using non-immersive
VR system. The median difference in task completion time
between the immersive and non-immersive VR group was -
24.35 with 95% confidence intervals for the median difference
in completion time of -48.63 to -10.93. Figure 8 summarizes
the results of task completion time obtained from both the
groups. The center line of the box with the dark circle shows
the median completion time, the dark triangle is the mean
completion time, the box itself indicates the 25th percentile and
75th percentile and the ‘tails’ specify the 10th and 90th
percentile.
Table 2. MEDIAN TASK COMPLETION TIME OF TWO
GROUPS
Group
N
(Sample
Size)
Median
Completion
Time
(in Seconds)
Group 1: Immersive VR 29 23.21
Group 2: Non-Immersive VR 25 49.04
Since the completion time was not normally distributed,
the completion time of both groups were compared non-
parametrically using Mann-Whitney U Test. The p-value
obtained in the test was 0.0001 and since p-value is much lesser
than 0.05, the null hypothesis can be rejected. And it can be
concluded that there is strong evidence to prove that time taken
by students to complete the activity using the immersive VR
system is less than the time taken by the students to complete
the same activity using non-immersive VR system. Hence, the
performance outcomes of the students using immersive VR
systems are found to be significantly better than students using
non-immersive VR systems.
Figure 8. BOX PLOT COMPARING THE TASK COMPLETION TIME OF TWO GROUPS
For the purpose of this study, two statements from the
post-experiment questionnaire are used to investigate the
preferences of students towards using VR systems in
engineering design education. The students were asked to
circle a response on the 5 point Likert scale for the following
two statements:
S1: I find it useful to be able to virtually manipulate objects
when I am doing engineering design
S2: I find it easier learning when I am virtually manipulating
objects
Figure 9. STUDENT RESPONSE TO STATEMENT 1 IN POST-EXPERIEMENT QUESTIONNAIRE
Figure 9 shows that, after participating in the experiment,
a greater % of Group 1 (Immersive VR system) students
“Strongly Agreed” with statement 1 when compared to Group
2 (Non-immersive VR system). Also, none of the Group 1
Non-Immersive VR GroupImmersive VR Group
140
120
100
80
60
40
20
0
Task
Co
mp
leti
on
Tim
e (
in s
eco
nd
s)
23.21
49.04
31.02
64.20
Boxplot of Immersive VR Group, Non-Immersive VR Group
8 Copyright © 2015 by ASME
students “Disagreed” with the statement as opposed to 8% of
Group 2. But the pattern is reversed in the case of “Agreed”.
Figure 10. STUDENT RESPONSE TO STATEMENT 2 IN POST-EXPERIEMENT QUESTIONNAIRE
Figure 10 shows that, after participating in the experiment,
a greater % of Group 1 (Immersive VR system) students
“Strongly Agreed” or “Agreed” with the statement 2 when
compared to Group 2 (Non-immersive VR system) students.
The pattern seems to reverse in the rest of the cases. Overall,
Figures 9 and 10 demonstrate a positive feedback in terms of
students’ attitude towards learning using immersive VR
systems in comparison to non-immersive VR systems. At the
end of the questionnaire, all students were asked to briefly
describe their experience of interacting with objects in virtual
world versus the real world. Some of the feedback obtained
from Group 1 students who used the immersive VR system are
discussed. (1) Many students felt that the use of a joystick made
the experience less realistic and proposed that it would be more
realistic if they could use their hands to interact with the objects
in the immersive virtual environment. (2) Though most of the
students felt that the immersive VR experience was very
realistic, few of them reported to have trouble in estimating the
depth of objects. (3) About 7% of students who used the
immersive VR system reported motion sickness. The majority
of group 2 students who used non-immersive VR system
reported having difficulty using a joystick to interact with the
objects. Contrastingly, only very few students from group 1
reported such difficulties. This may be due to the fact that non-
immersive VR system does not continually adjusts to retain the
viewers’ perspective unlike immersive VR system. This could
be a contributing factor towards significant difference in task
completion time between the two groups.
5 CONCLUSIONS The aim of the research was to investigate whether there
exist any statistically significant differences in performance
outcomes between students engaged in non-immersive and
immersive VR systems. To the knowledge of the authors, this
is the first study evaluating users’ performance in immersive
VR systems versus non-immersive VR systems for online
engineering design activities. The median task completion time
of students using the immersive VR system (23.21 seconds)
was found to be lesser than that of students using non-
immersive VR system (49.04 seconds). The hypothesis that
time taken by students to complete the product functional
analysis activity using the immersive VR system is less than
the time taken by the students to complete the same activity
using non-immersive VR system was supported (p-value =
0.0001). This implies that, the performance outcomes of the
students using immersive VR systems are significantly better
than students using non-immersive VR systems. The ability of
the immersive VR system to continually adjust to retain the
viewers’ perspective in the immersive 3D virtual world is
considered as one of the major advantages. Apart from the
statistical test on the task completion time, the authors also
investigated the preferences of students towards using VR
systems in engineering design education. In general, a positive
feedback towards immersive VR systems was found among
users. Most of the Group 1 participants who used Oculus Rift®
reported that the experience of using immersive VR system
was very realistic and interesting, but pointed out that the usage
of the joystick in the immersive VR system made their
experience less realistic. This emphasizes on the necessity for
exploring the integration of 3D interactive technology such as
a haptic glove with immersive visual displays, in order to
achieve complete immersion. The other issues reported by the
student using immersive VR systems were related to difficulty
in perception of depth and motion sickness.
Immersion is considered as one of the most important
characteristic contributing towards the sense of presence
experienced by the user in the VR system. Further studies
involving integration of immersive visual displays with 3D
interactive devices are required. The 3D interactive devices
when integrated with immersive visual displays have the
potential to increase the level of presence among the users.
Apart from the technological characteristics like immersion,
individual characteristics such as age and gender can also affect
the level of presence experienced by users. Therefore, it would
be interesting to conduct large scale studies to investigate the
effectiveness of immersive VR systems among users of
different gender and age group. In future, the authors would
like to explore the reasons for the observed difference in
performance outcomes between users in immersive and non-
immersive VR systems. Immersive VR systems could provide
the potential platform for students to participate in learning
activities remotely from their home or anywhere else, thereby
facilitating distance education.
ACKNOWLEDGEMENT
The authors of this work would like to acknowledge the
Penn State’s Center for Online Innovation in Learning (COIL)
for the grant that funded this research. Any opinions, findings,
or conclusions found in this paper are those of the authors and
do not necessarily reflect the views of the sponsors. The
authors would also like to acknowledge Bryan Dickens, Steven
Sellers, Gabe Harms and Owen Shartle for their time and hard
9 Copyright © 2015 by ASME
work in developing the virtual reality environment for this
research.
REFERENCES
[1] J. M. Moshell and C. E. Hughes, “Virtual environments
as a tool for academic learning,” Handbook of virtual
environments: Design, implementation, and
applications, pp. 893–910, 2002.
[2] M. Slater and S. Wilbur, “A framework for immersive
virtual environments (FIVE): Speculations on the role of
presence in virtual environments,” Presence:
Teleoperators and virtual environments, vol. 6, no. 6, pp.
603–616, 1997.
[3] D. A. Bowman and R. P. McMahan, “Virtual reality:
how much immersion is enough?,” Computer, vol. 40,
no. 7, pp. 36–43, 2007.
[4] A. Van Dam, A. Forsberg, D. H. Laidlaw, J. J. LaViola,
and R. M. Simpson, “Immersive VR for scientific
visualization: A progress report,” Computer Graphics
and Applications, IEEE, vol. 20, no. 6, pp. 26–52, 2000.
[5] T. Mujber, T. Szecsi, and M. Hashmi, “Virtual reality
applications in manufacturing process simulation,”
Journal of materials processing technology, vol. 155,
pp. 1834–1838, 2004.
[6] K. Patel, J. N. Bailenson, S. Hack-Jung, R. Diankov, and
R. Bajcsy, “The effects of fully immersive virtual reality
on the learning of physical tasks,” in Proceedings of the
9th Annual International Workshop on Presence, Ohio,
USA, 2006, pp. 87–94.
[7] G. G. Robertson, S. K. Card, and J. D. Mackinlay,
“Nonimmersive virtual reality,” Computer, vol. 26, no.
2, pp. 81–83, 1993.
[8] C. Ware, K. Arthur, and K. S. Booth, “Fish tank virtual
reality,” in Proceedings of the INTERACT’93 and
CHI’93 conference on Human factors in computing
systems, 1993, pp. 37–42.
[9] W. Krueger and B. Froehlich, “The Responsive
Workbench,” IEEE Computer Graphics and
Applications, vol. 14, no. 3, pp. 12–15, 1994.
[10] T. Funkhouser and K. Li, “Guest Editors’ Introduction:
Large-Format Displays,” IEEE Computer Graphics and
Applications, vol. 20, no. 4, pp. 20–21, 2000.
[11] “Augmented Reality & Virtual Reality Market - 2018 |
MarketsandMarkets.” [Online]. Available:
http://www.marketsandmarkets.com/Market-
Reports/augmented-reality-virtual-reality-market-
1185.html. [Accessed: 03-Jan-2015].
[12] H. Kaufmann, D. Schmalstieg, and M. Wagner,
“Construct3D: a virtual reality application for
mathematics and geometry education,” Education and
information technologies, vol. 5, no. 4, pp. 263–276,
2000.
[13] N. I. Durlach, A. S. Mavor, and others, Virtual reality:
scientific and technological challenges. National
Academies Press, 1994.
[14] F. Mantovani, G. Castelnuovo, A. Gaggioli, and G. Riva,
“Virtual reality training for health-care professionals,”
CyberPsychology & Behavior, vol. 6, no. 4, pp. 389–
395, 2003.
[15] J. Sehell and J. Shochet, “Designing interactive theme
park rides,” Computer Graphics and Applications,
IEEE, vol. 21, no. 4, pp. 11–13, 2001.
[16] S. Ottosson, “Virtual reality in the product development
process,” Journal of Engineering Design, vol. 13, no. 2,
pp. 159–172, 2002.
[17] B. Bideau, R. Kulpa, N. Vignais, S. Brault, F. Multon,
and C. Craig, “Using virtual reality to analyze sports
performance,” Computer graphics and applications,
IEEE, vol. 30, no. 2, pp. 14–21, 2010.
[18] M. Reis, “PTCVoice: Could Virtual Reality Be The Next
Big Thing In Education?,” Forbes. [Online]. Available:
http://www.forbes.com/sites/ptc/2014/08/27/could-
virtual-reality-be-the-next-big-thing-in-education/.
[Accessed: 05-Jan-2015].
[19] V. S. Pantelidis, “Reasons to use virtual reality in
education,” VR in the Schools, vol. 1, no. 1, p. 9, 1995.
[20] T. A. Mikropoulos and A. Natsis, “Educational virtual
environments: A ten-year review of empirical research
(1999–2009),” Computers & Education, vol. 56, no. 3,
pp. 769–780, 2011.
[21] O. Çaliskan, “Virtual field trips in education of earth and
environmental sciences,” Procedia-Social and
Behavioral Sciences, vol. 15, pp. 3239–3243, 2011.
[22] V. Ramasundaram, S. Grunwald, A. Mangeot, N.
Comerford, and C. Bliss, “Development of an
environmental virtual field laboratory,” Computers &
Education, vol. 45, no. 1, pp. 21–34, 2005.
[23] T. Mikropoulos and J. Bellou, “The Unique Features of
Educational Virtual Environments,” in e-society, 2006,
pp. 122–128.
[24] M. Roussou, “Learning by doing and learning through
play: an exploration of interactivity in virtual
environments for children,” Computers in Entertainment
(CIE), vol. 2, no. 1, pp. 10–10, 2004.
[25] R. Driver, “Theory into Practice: A Constructivist
Approach to Curriculum Development,” in Development
and Dilemmas in Science Education, London: Falmer
Press, 1998.
[26] J. Piaget, Biologie et connaissance (Biology and
knowledge). Paris: Gallimard., 1967.
[27] J. E. Ormrod, Educational Psychology: Principles &
Applications. Englewood Cliffs, NJ: Prentice Hall, 1995.
[28] B. Dalgarno, “The potential of 3D virtual learning
environments: A constructivist analysis,” Electronic
Journal of Instructional Science and Technology, vol. 5,
no. 2, pp. 3–6, 2002.
[29] C. Dede, “The evolution of constructivist learning
environments: Immersion in distributed, virtual worlds,”
Educational technology, vol. 35, no. 5, pp. 46–52, 1995.
10 Copyright © 2015 by ASME
[30] D. J. Skiba, “Disruption in Higher Education: Massively
Open Online Courses (MOOCs),” Nursing Education
Perspectives, vol. 33, no. 6, pp. 416–417, 2012.
[31] K. Veeramachaneni, F. Dernoncourt, C. Taylor, Z.
Pardos, and U.-M. O’Reilly, “Moocdb: Developing data
standards for mooc data science,” in AIED 2013
Workshops Proceedings Volume, 2013, p. 17.
[32] S. Cooper and M. Sahami, “Reflections on stanford’s
moocs,” Communications of the ACM, vol. 56, no. 2, pp.
28–30, 2013.
[33] C. S. Tucker, B. Dickens, and A. Divinsky, “Knowledge
Discovery of Student Sentiments in MOOCs and Their
Impact on Course Performance,” in ASME 2014
International Design Engineering Technical
Conferences and Computers and Information in
Engineering Conference, 2014, pp. V003T04A028–
V003T04A028.
[34] Q. H. Wang and J. R. Li, “A desktop VR prototype for
industrial training applications,” Virtual Reality, vol. 7,
no. 3–4, pp. 187–197, 2004.
[35] H. Aoki, C. M. Oman, D. A. Buckland, and A. Natapoff,
“Desktop-VR system for preflight 3D navigation
training,” Acta astronautica, vol. 63, no. 7, pp. 841–847,
2008.
[36] S. LeBlanc, K. Paquin, K. Carr, and S. Horton, “Non-
immersive virtual reality for fine motor rehabilitation of
functional activities in individuals with chronic stroke: a
review,” Aging Sci, vol. 1, no. 105, p. 2, 2013.
[37] J. Shan, “Visualizing 3-D geographical data with
VRML,” in Computer Graphics International, 1998.
Proceedings, 1998, pp. 108–110.
[38] C.-H. Chen, J.-C. Yang, S. Shen, and M.-C. Jeng, “A
desktop virtual reality earth motion system in astronomy
education,” Educational Technology & Society, vol. 10,
no. 3, pp. 289–304, 2007.
[39] M. Krijn, P. M. Emmelkamp, R. Biemond, C. de W. de
Ligny, M. J. Schuemie, and C. A. van der Mast,
“Treatment of acrophobia in virtual reality: The role of
immersion and presence,” Behaviour research and
therapy, vol. 42, no. 2, pp. 229–239, 2004.
[40] M. Limniou, D. Roberts, and N. Papadopoulos, “Full
immersive virtual environment CAVETM in chemistry
education,” Computers & Education, vol. 51, no. 2, pp.
584 – 593, 2008.
[41] C. Tucker, K. Jackson, G. Kremer, and L. Schmidt, “The
Evolution of Tactile and Virtual Learning Preferences in
Undergraduate Engineering Education,” in Proceedings
of the 2014 American Society of Engineering Education,
2014.
[42] G. Kremer, C. Tucker, and K. Jackson, “Student
Preferences on Tactile Versus Digital Learning:
implications for conceptual design,” in 15th
International Conference on Engineering Product
Design Education, Dublin, Ireland, 2013, pp. 735–739.
[43] J. S. Tashiro and D. Dunlap, “The impact of realism on
learning engagement in educational games,” in
Proceedings of the 2007 conference on Future Play,
2007, pp. 113–120.
[44] T. M. Li, M. Chau, P. W. Wong, E. S. Lai, and P. S. Yip,
“Evaluation of a web-based social network electronic
game in enhancing mental health literacy for young
people,” Journal of medical Internet research, vol. 15,
no. 5, 2013.
[45] M. Asgari and D. Kaufman, “Relationships Among
Computer Games, Fantasy, and Learning,” in
Proceedings of the 2nd International Conference on
Imagination and Education, Vancouver, British
Columbia, Canada, 2004.
[46] D. Allison, B. Wills, D. Bowman, J. Wineman, and L. F.
Hodges, “The virtual reality gorilla exhibit,” Computer
Graphics and Applications, IEEE, vol. 17, no. 6, pp. 30–
38, 1997.
[47] K. S. Song and W. Y. Lee, “A virtual reality application
for geometry classes,” Journal of Computer Assisted
Learning, vol. 18, no. 2, pp. 149–156, 2002.
[48] A. J. Yeh and R. A. Nason, “VRMath: A 3D Microworld
for Learning 3D Geometry,” in Faculty of Education,
Lugano, Switzerland, 2004.
[49] J.-H. Kim, S. Park, H. Lee, K.-C. Yuk, and H. Lee,
“Virtual reality simulations in physics education,”
Interactive Multimedia Electronic Journal of Computer-
Enhanced Learning, vol. 3, no. 2, 2001.
[50] D. Demaree, S. Stonebraker, W. Zhao, and L. Bao,
“Virtual reality in introductory physics laboratories,” in
Proceedings of AIP Conference, 2005, vol. 790, pp. 93–
96.
[51] S. A. Barab, K. E. Hay, M. Barnett, and T. Keating,
“Virtual solar system project: Building understanding
through model building,” Journal of Research in Science
Teaching, vol. 37, no. 7, pp. 719–756, 2000.
[52] J. S. B. and R. P. Adler, “Minds on Fire: Open
Education, the Long Tail, and Learning 2.0,” Educause
Review, vol. 43, no. 1, pp. 16–32, 2008.
[53] G. Ahlberg, L. Enochsson, A. G. Gallagher, L. Hedman,
C. Hogman, D. A. McClusky, S. Ramel, C. D. Smith,
and D. Arvidsson, “Proficiency-based virtual reality
training significantly reduces the error rate for residents
during their first 10 laparoscopic cholecystectomies,”
The American journal of surgery, vol. 193, no. 6, pp.
797–804, 2007.
[54] N. E. Seymour, A. G. Gallagher, S. A. Roman, M. K.
O’Brien, V. K. Bansal, D. K. Andersen, and R. M.
Satava, “Virtual reality training improves operating
room performance: results of a randomized, double-
blinded study,” Annals of surgery, vol. 236, no. 4, p. 458,
2002.
[55] H.-M. Huang, U. Rauch, and S.-S. Liaw, “Investigating
learners’ attitudes toward virtual reality learning
environments: Based on a constructivist approach,”
Computers & Education, vol. 55, no. 3, pp. 1171–1182,
2010.
11 Copyright © 2015 by ASME
[56] C. Jennett, A. L. Cox, P. Cairns, S. Dhoparee, A. Epps,
T. Tijs, and A. Walton, “Measuring and defining the
experience of immersion in games,” International
Journal of Human-Computer Studies, vol. 66, no. 9, pp.
641–661, Sep. 2008.
[57] T. S. Lendvay, T. C. Brand, L. White, T. Kowalewski,
S. Jonnadula, L. D. Mercer, D. Khorsand, J. Andros, B.
Hannaford, and R. M. Satava, “Virtual reality robotic
surgery warm-up improves task performance in a dry
laboratory environment: a prospective randomized
controlled study,” Journal of the American College of
Surgeons, vol. 216, no. 6, pp. 1181–1192, 2013.
[58] J. Newmark, V. Dandolu, R. Milner, H. Grewal, S.
Harbison, and E. Hernandez, “Correlating virtual reality
and box trainer tasks in the assessment of laparoscopic
surgical skills,” American journal of obstetrics and
gynecology, vol. 197, no. 5, pp. 546–e1, 2007.
[59] J. Hwang, J. Jung, and G. J. Kim, “Hand-held virtual
reality: a feasibility study,” in Proceedings of the ACM
symposium on Virtual reality software and technology,
2006, pp. 356–363.
[60] T. Ni, D. A. Bowman, and J. Chen, “Increased display
size and resolution improve task performance in
information-rich virtual environments,” in Proceedings
of Graphics Interface 2006, 2006, pp. 139–146.
[61] M. Devendorf, K. Lewis, T. W. Simpson, R. B. Stone,
and W. C. Regli, “Evaluating the use of
cyberinfrastructure in the classroom to enhance product
dissection,” in ASME 2007 International Design
Engineering Technical Conferences and Computers and
Information in Engineering Conference, 2007, pp. 25–
34.
[62] M. Devendorf, K. Lewis, T. W. Simpson, R. B. Stone,
and W. C. Regli, “Evaluating the use of digital product
repositories to enhance product dissection activities in
the classroom,” Journal of Computing and Information
Science in Engineering, vol. 9, no. 4, p. 041008, 2009.
[63] J. S. Lamancusa, J. E. Jorgensen, and J. L. Fridley,
“Product dissection-a tool for benchmarking in the
process of teaching design,” in Frontiers in Education
Conference, 1996. FIE’96. 26th Annual Conference.,
Proceedings of, 1996, vol. 3, pp. 1317–1321.
[64] H. J. Thevenot and T. W. Simpson, “Guidelines to
minimize variation when estimating product line
commonality through product family dissection,”
Design Studies, vol. 28, no. 2, pp. 175–194, 2007.
[65] D. Lefever and K. Wood, “Design for assembly
techniques in reverse engineering and redesign,” in
ASME Design Theory and Methodology Conference,
1996, pp. 78712–1063.
[66] K. L. Wood, D. Jensen, J. Bezdek, and K. N. Otto,
“Reverse engineering and redesign: Courses to
incrementally and systematically teach design,” Journal
of Engineering Education, vol. 90, no. 3, pp. 363–374,
2001.
[67] M. R. Bohm and R. B. Stone, “Product design support:
exploring a design repository system,” in ASME 2004
International Mechanical Engineering Congress and
Exposition, 2004, pp. 55–65.
[68] N. Vasudevan and C. S. Tucker, “Digital Representation
of Physical Artifacts: The Effect of Low Cost, High
Accuracy 3D Scanning Technologies on Engineering
Education, Student Learning and Design Evaluation,” in
ASME 2013 International Design Engineering
Technical Conferences and Computers and Information
in Engineering Conference, 2013, pp. V001T04A013–
V001T04A013.
[69] T. Scoville, M. Pham, and J. Lartigue, “Promoting K-8
Learning using Oculus Rift: Employing Virtual Reality
to Increase Learning Outcomes in Elementary Biology,”
in World Conference on E-Learning in Corporate,
Government, Healthcare, and Higher Education, 2014,
vol. 2014, pp. 1100–1105.
[70] J. Lartigue, J. Cathcart, T. M. Kelleher, A. Pfundstein,
and M. D. Williams, “Leveraging Oculus Rift for an
Immersive Distance-Learning Experience: A High
Definition, Panoramic Lecture Recording/Playback
System using Commercial Virtual Reality Tools,” in
World Conference on E-Learning in Corporate,
Government, Healthcare, and Higher Education, 2014,
vol. 2014, pp. 1090–1099.
[71] S. Wang, Z. Mao, C. Zeng, H. Gong, S. Li, and B. Chen,
“A new method of virtual reality based on Unity3D,” in
2010 18th International Conference on Geoinformatics,
2010, pp. 1–5.
