Arm Pointing Movements in a Three Dimensional Virtual Environment: Effect of Two Different Viewing Media

Sandeep Subramanian, Christian Beaudoin and Mindy F Levin

Abstract - Virtual Reality (VR) is being used increasingly in many fields of medicine, including rehabilitation. Both 2D and 3D virtual environments (VEs) can be viewed either through a head mounted display (HMD) or on a screen (computer monitor and rear projection system, SPS). However, the question of whether the medium through which the environment is viewed affects motor performance has not been addressed. The objective of our study was to determine whether movement patterns were different when movements were performed in a 3D fully immersive VE viewed via an HMD or SPS. Two groups of subjects were recruited (stroke, healthy). They performed pointing movements to targets placed in the ipsilateral, central and contralateral arm workspaces in a VE. The VE, designed to resemble the interior of an elevator, was viewed via an HMD or a SPS. Arm motor impairment and spasticity were evaluated in both groups of subjects. The kinematics of the pointing movements were recorded using an optical tracking system (Optotrak Certus, 100 Hz, 6 markers). Arm motor performance (speed, precision and trajectory straightness) and movement quality outcomes (elbow and shoulder ranges of motion and trunk forward displacement) were analyzed using 2 way ANOVAs. Preliminary results suggest that the control group had straighter movements and used more shoulder flexion as compared to the stroke group. When the VE was viewed via both media, there were no differences in terms of endpoint precision and speed, elbow and shoulder ranges of motion and trunk forward displacement in both groups. Both groups reported that they completely enjoyed performing the movements when viewing them via both media. All subjects in the control group and 80% of subjects in the stroke group reported that VE was engaging, that it felt real and that the movements performed were similar to those made in the physical world. The results of this study have implications for the design of rehabilitation applications using VR aimed at improving arm motor activity and function.

Manuscript submitted April 15, 2008. This work was supported by the Canadian Institutes of Health Research (CIHR) and the Canadian Foundation for Innovation (CFI). S. Subramanian is a graduate student at School of Physical and Occupational Therapy, McGill University, 3654 Promenade Sir-William-Osler Montreal, QC, H3G 1Y5, Canada, and Jewish Rehabilitation Hospital (JRH) site of Center for Interdisciplinary Research in Rehabilitation (CRIR). Address: 3205 Place Alton Goldbloom, Laval, QC H7V 1R2. tel: 450-688-9550, ext 4824, fax: 450-688-3116. email: sandeep.subramanian@mail.mcgill.ca C. Beaudoin is with the JRH site of CRIR. Address as above. tel: 450 688 9550, extn 623, fax 450 688 3673. email: christ_beaudoin@hotmail.com M. F. Levin is Associate Professor, at the School of Physical and Occupational Therapy, McGill University and a researcher at the JRH of CRIR. Addresses as above. tel: 450-688-9550, ext. 3834, fax: 450-688-3116. email: mindy.levin@mcgill.ca

I. INTRODUCTION

IRTUAL Reality (VR) is a technical assembly that allows human subjects to interact in a virtual environment (VE) with objects or events created by

a computer. VR provides an illusory sense of being immersed in a world where objects viewed by the user via different media can be visualized and interacted with [1]. Initially created for video games and flight simulators, the use of VR has been explored as a delivery system for education, diagnosis and treatments in many fields of medicine including rehabilitation of physical and cognitive impairments [2]. The added value of using VR for treatment delivery is the possibility of increasing the salience of the activity in a more motivating environment than the traditional rehabilitation setting [3].

For rehabilitation applications, VR equipment may consist of a visual display (HMD or screen) to view VEs having different degrees of immersion. Immersion is a psychological state in which the person feels that he is surrounded by, is a part of and communicates with an environment that provides a constant flow of experiences and stimuli [4]. It includes the level to which the VR environment is extensive, with respect to sensory systems and inclusive, with regards to the amount to which external sensory data (from physical reality) is shut out from the participant’s experience. For immersion to occur there needs to be a consistency between the participant's proprioceptive feedback about body movements and information supplied to the participant on the visual display. This may be achieved through real-time tracking of head and body motion, such that, a turn of the head results in a corresponding change in the visual display [5]. Immersion is intended to encourage the belief that one has left the real world and is now "present" in the VE. The sense of presence has been defined as the feeling of being in an environment i.e. being immersed in it. The consequential behavior is consistent with the subject’s behavior in the same situation in a similar environment in the physical world [6]. Immersive VR systems promote a greater sense of presence in the environment through the wearing of a HMD which effectively blocks out the physical surroundings, leaving the user looking only at the VE [1]. VR is being increasingly used as a methodology for upper and lower limb rehabilitation in patients with stroke-related brain damage. Both 2D [7, 8] and 3D [9, 10, 11] fully immersive environments have been described. However, in order to determine the validity of VR as a treatment delivery system, it is essential to know

V

978-1-4244-2701-7/08/$25.00 ©2008 IEEE 181

whether the movements performed in a VE are similar to those performed in an equivalent physical environment (PE) since sensory experiences in each environment may differ potentially affecting how movements are made. Viau et al., [12] compared movements made by subjects with or to virtual objects in a 2D VE to those made with or to real objects in a PE. In both environments, subjects used similar movement patterns while reaching, grasping, transporting and placing a ball. However, they used more elbow extension and less wrist extension when they moved in the VE as compared to the physical world. The authors suggested that this might have been due to the absence of depth perception and tactile feedback at the end of the reach in the VE. Pointing movements made by subjects with chronic post-stroke hemiparesis in a 3D fully immersive VE viewed through an HMD were compared to movements made in a similarly designed PE [13]. The movements had similar characteristics in both environments in terms of ranges of elbow and shoulder motion and interjoint coordination. However, subjects used less forward trunk displacement (compensation) and made more curved and less precise movements in the VE compared to the PE (movement quality).

There are certain limitations with the use of HMDs for viewing 3D fully immersive environments. Users may experience ‘cybersickness’ or Virtual Reality Induced Symptoms and Effects (VRISE) with an HMD. Cobb et al., [14] found that almost 60% of the subjects who wore an HMD experienced some symptoms of cybersickness like nausea, dizziness and vomiting and had visual problems such as reduced binocular acuity and eyestrain. In a recent study, healthy subjects viewed a 3D VE through four different display media - HMD, computer monitor, rear screen projection system (SPS) and a reality theatre (curved large screen display) [15]. Symptoms of nausea and disorientation were higher when the VE was viewed through an HMD as compared to the computer monitor, SPS and the reality theatre.

The older generations of HMDs weighed about 28 - 35 ounces and had a field of view (FOV) of only about 50° diagonal, 30° vertical x 40° horizontal (Kaiser XL -35 and XL -50). Normal adult FOV spans approximately 200° horizontally, taking both eyes into account, and 135° vertically [16]. Knight and Baber [17] found that wearing HMDs caused healthy users to modify their neck posture and concluded that the musculoskeletal system of the head and the neck may be placed under a greater amount of stress while wearing this device. In a recent study conducted in our lab [18], 12 subjects with stroke-related brain damage were randomized into two groups (n = 6) and practiced repetitive arm pointing movements in either a PE or a similarly designed VE. The subjects practiced the movements for 10 consecutive days and their movement kinematics were recorded before and after practice (Optotrak 3010, 6 markers, 100 Hz). Feedback was provided about movement speed and precision as well as excessive trunk use during practice in both environments. Subjects who practiced movements in

the VE improved arm movement quality by increasing shoulder flexion (p<0.05) and tending to increase shoulder horizontal adduction (p<0.06) ranges of motion. Patients practicing arm movements in the VE also tended to use less trunk forward displacement after training as compared to those practicing in the PE, who had slightly increased trunk displacement. This was unexpected, since both groups had the same amount of practice (10 days) and had received the same type and amount of feedback. Some of the difference may be attributed to the fact that those training in the VE wore an HMD while those training in the PE did not. Wearing of the HMD may have limited the amount of trunk movement because of its weight or because of the need to keep the head straighter due to the reduced FOV (FOV = ~50° compared to 160° in the physical world) since too much head motion may have resulted in the loss of view of the environment. Whether viewing the VE through different display media makes a difference in the movement patterns (quality of movement) during reaching or pointing tasks has not been previously addressed. The specific objective of our study was to determine whether the quality of movement differed when the same movements were performed in 3D fully immersive VE viewed through an HMD or via an SPS. On the basis of our previous work, we hypothesized that the subjects viewing the VE via an SPS would have greater trunk forward displacement than those viewing it through an HMD.

II. METHODS

A. Subjects

The study, approved of by local ethics committees, was conducted at the Motor Control Laboratory of the Jewish Rehabilitation Hospital, in Laval, Quebec. Two groups of subjects between the ages of 35 and 70 years were recruited: individuals with chronic post-stroke hemiparesis and healthy, non-disabled subjects. The subjects in the stroke group had sustained a single unilateral stroke of non-traumatic origin, had a recovery stage score of 3-6/7 in the arm and hand section of the Chedoke McMaster (CM) Scale [19], had no hemispatial neglect or apraxia and could understand simple instructions. Those with shoulder pain, brainstem or cerebellar stroke, or other neurological or orthopedic conditions affecting the arm or trunk were excluded. The control group consisted of age- and sex-matched subjects who were included if they had no orthopedic and/or neurological problems. Subjects with claustrophobia were excluded as they would have difficulty in using the HMD. For the stroke subjects, upper limb impairment was evaluated using the Fugl-Meyer Stroke Assessment [20, 21] and the Composite Spasticity Index (CSI) [22].

B. VR System The VR system consisted of different peripherals

182

connected together on the same computer running a CAREN VR simulation system (Computer Assisted Rehabilitation Environment; Motek BV). The peripherals incorporated into the system were an HMD with a FOV of 50° (Kaiser XL 50, resolution 1024 x 768, frequency 60 Hz), SPS (2m x 1.5m) and an Optotrak Motion Capture System (Northern Digital Corp., Type Certus). The computer was an IBM compatible PC (Dual Xeon 3.06 GHz, 2GB RAM, 160 GB hard drive) running Windows XP. An important component of the system was the graphics card (a dual head Nvidia Quatro FX 3000) that provided a stereoscopic visual representation of the environment with high frame rates (around 70 Hz).

The VE was a representation of an interior elevator scene, consisting of six numbers (1-6) placed in the midline, ipsilateral and contralateral arm workspaces. Numbers 1-3 were positioned in the top row and numbers 4-6 in the bottom row. The numbers were written on squares measuring 36 cm2 (6cm x 6cm) and were placed at a centre-to-centre distance of 26cms. The distance to the middle target was equal to the subject’s arm’s length, measured from the edge of the acromion process of the shoulder to the tip of the index. The middle target was aligned with the xiphoid process of the subject’s sternum. Subjects were seated on a comfortable chair, with the hip and knee flexed to 90° and the shoulder in slight abduction (20°) and internal rotation. The elbow was flexed to 90° with complete pronation of the forearm. The wrist was in the neutral position with the fingers semi-flexed and held at the level of the xiphoid process of the sternum. Subjects performed 72 pointing movements (12 randomized trials to each of 6 targets), divided into three blocks of 24 trials each, with rest periods between blocks. Presence in the environment was evaluated with a validated sense of presence questionnaire [4]. Block randomization was used to assign subjects to groups. In both groups, half of the subjects performed the pointing movement viewing the VE through the HMD first and the other half performed them viewing the VE through the SPS first. The subjects wore 3D glasses while performing the movements viewing the VE via the SPS. Peripheral vision of the physical environment was blocked by black felt cloth attached to the glasses to make the environments more comparable.

C. Data recording and analysis Markers (infrared emitting diodes) were placed on anatomical landmarks of the hand, arm and trunk to record kinematic data: tip of the index finger, dorsomedial border of the wrist crease, lateral epicondyle, ipsilateral and contralateral acromion processes and junction of upper and lower third of the sternum. Data were recorded with an Optotrak Certus system for 5-6 s at a sampling rate of 100 Hz. Kinematic outcome measures were: endpoint trajectory straightness and smoothness, movement precision, range of elbow extension, shoulder flexion and shoulder horizontal adduction as well as trunk

anterior displacement. For details on kinematic analysis, please refer to [18]. Levene’s test of homogeneity of variances was conducted and kinematic outcomes were compared across groups (healthy, stroke) and environments (SPS, HMD) using 2 way ANOVAs.

III. RESULTS

Preliminary data of pointing movements made to the lower central target by four healthy subjects (control group) and five subjects with chronic post stroke hemiparesis (stroke group) are shown in Figs. 1-3. Fig. 1 shows the mean endpoint and trunk trajectories for one healthy subject (left) and one stroke subject (right) reaching to the three lower targets (Target 4, 5, 6) while viewing the VE in each of the two media. Movements were more curved when subjects viewed the VE via the SPS compared to the HMD. Overall, endpoint precision (Fig. 2, top row left) was similar for both groups. Control subjects tended to move faster (Fig. 1, top row right) and had straighter movements [F2,6 = 15.86, p < 0.05; Fig. 2,bottom row) than stroke subjects. Control subjects also used a greater range of shoulder flexion [F2,6 = 7.58, p < 0.01; Fig. 3, top row left) and tended to use greater ranges of shoulder horizontal adduction (Fig. 3, top row right), elbow extension (Fig. 3, bottom row left) and trunk forward displacement (Fig. 3, bottom row right) as compared to the stroke group but the latter three ranges were not significantly different. There were no differences in the movement outcomes when the VE was viewed through the HMD or via the SPS for both groups of subjects. The questions from the sense of presence questionnaire were grouped together into five domains. These were: 1) familiarity, 2) engagement in the environment, 3) how real the environment felt, 4) movement similarity in each environment and 5) interest/enjoyment. Control subjects indicated 100% agreement across both viewing media for domains 2 to 5. For domain 1, 75% and 50% reported that they did not feel familiar with the VE when viewed via the SPS or HMD respectively. When the stroke group viewed the environment through the SPS, all subjects reported that they enjoyed performing the movements in the VE (domain 5). Eighty percent of the subjects indicated that the VE was engaging, felt real, that movements performed were similar to the real world and that they were familiar with the environment. Similar results were obtained when the VE was viewed through the HMD.

IV. DISCUSSION

Preliminary results indicate that the movements made by subjects in the stroke group were more curved and that stroke subjects used less shoulder flexion compared to the controls. Stroke subjects also tended to move slower, to use a smaller range of elbow extension and shoulder horizontal adduction and to use more trunk displacement compared to non-disabled controls. These results are

183

consistent with our earlier findings comparing movements in similar groups of subjects [13] with respect to movement speed and ranges of movement. The movement

Fig. 1. Example of endpoint and trunk trajectories of the pointing movements made to the lower central target (T5) for one healthy (left) and one stroke (right) subject performed while viewing the virtual environment through head-mounted display (grey lines) and screen projection system (black lines).

Fig. 3. Range of shoulder flexion, shoulder horizontal adduction, elbow extension and trunk forward displacement for the pointing movements performed in the VE viewed via the HMD (grey bars) or screen (black bars) by healthy subjects and subjects with stroke.

Fig.2. Precision, speed and trajectory straightness for the pointing movements performed in the VE viewed via the HMD (grey bars) or screen (black bars) by healthy subjects and subjects with stroke. outcomes did not differ when the VE was viewed through either of the viewing media. Thus our hypothesis that subjects would use more trunk displacement when the VE was viewed via the SPS as compared to the HMD was not supported by our current findings. Our hypothesis was formulated on the basis of earlier work in our lab. We observed that stroke subjects tended to use their head and trunk as one unit, an en-bloc strategy, while they viewed the VE through the HMD and that head and trunk motion would be restricted by the limited field of view of the HMD. However, our preliminary findings of the

similarity in performance of movements in both environments and the results of the presence questionnaire suggest that the VE was perceived in the same way regardless of the medium used to view it.

The equivalence of performance in the two VEs suggests that the SPS can be used as a medium to view 3D VEs for upper limb stroke rehabilitation. It may be a more cost effective alternative to the use of HMDs. The application of these results, however, is limited due to a small sample size.

V. CONCLUSION

VR is a recently emerged methodology that is being

increasingly used in rehabilitation. However, it still needs to be completely determined which is the best medium through which to view a VE and whether the medium through which the VE is viewed may affect movement performance. The current results take us a step forward towards the use of SPS. The results of the full study will help clarify these points. More research is needed before the use of the SPS can be recommended. It also needs to be estimated whether training in 3D fully immersive VE viewed via SPS is similar or better to conventional training. This will have implications for the design of cost effective rehabilitation applications using VR aimed at enhancing arm motor activity and function.

ACKNOWLEDGMENTS We wish to acknowledge Ruth Dannenbaum for help with patient recruitment and evaluation and Jeniffer Ranallo for help with data collection.

184

REFERENCES

[1] M. D. J. McNeill, L. Pokluda, S. M. Mcdonough, J. Crosbie, “Immersive virtual reality for upper limb rehabilitation following stroke,” IEEE International Conference on Systems, Man and Cybernetics, 2004.

[2] K. L. Simone, M. T. Schultheis, J. B. S. Remimbas, S. R. Millis, “Head-Mounted Displays for clinical virtual reality applications: pitfalls in understanding user behavior while using technology,” Cyberpsychol Behav, vol. 9, (no.5), pp. 591-603, 2006.

[3] M. T. Schultheis, A. A. Rizzo, (2001) “The application of virtual reality technology in rehabilitation”, Rehab Psychology, vol. 46, (no. 3), pp. 296–311, August 2001.

[4] B. G. Witmer, M. J. Singer, “Measuring presence in a virtual environment: a presence questionnaire”, Presence, vol. 7, (no. 3), pp. 225-240, June 1998.

[5] M. A. Slater, “Note on presence terminology”, Presence – Connect Online, available http://presence.cs.ucl.ac.uk/presenceconnect/articles/ Jan 2003/ melslaterJan27200391557.html, accessed on April 15, 2008.

[6] M. A. Slater, V. Linakis, M. Usoh, R. Kooper, “Immersion, presence and performance in virtual environments: an experiment using tri-dimensional chess”, presented at the Virtual reality and Software and Technology Conference, Hong Kong, July 1996, available http://www.cs.ucl.ac.uk/staff/m.slater/Papers/ Chess/Immersion_2.html, accessed on April 15, 2008.

[7] D. Rand, N. Katz, and P.L. Weiss, “Evaluation of virtual shopping in the VMall: comparison of post-stroke participants to healthy control groups,” Disabil Rehabil, vol. 29, (no. 22), pp. 1710-9, November 30 2007.

[8] S.H. You, S.H. Jang, Y.H. Kim, M. Hallett, S.H. Ahn, Y.H. Kwon, J.H. Kim, and M.Y. Lee, “Virtual reality-induced cortical reorganization and associated locomotor recovery in chronic stroke: an experimenter-blind randomized study,” Stroke, vol. 36, (no. 6), pp. 1166-71, June 2005.

[9] J. Fung, C.L. Richards, F. Malouin, B.J. McFadyen, and A. Lamontagne, “A treadmill and motion coupled virtual reality system for gait training post-stroke,” Cyberpsychol Behav, vol. 9, (no. 2), pp. 157-62, April 2006.

[10] A. Lamontagne, J. Fung, B.J. McFadyen, and J. Faubert, “Modulation of walking speed by changing optic flow in persons with stroke,” J Neuroeng Rehabil, vol. 4, pp. 22, June 2007.

[11] S. Subramanian, L.A. Knaut, C. Beaudoin, B.J. McFadyen, A.G. Feldman, and M.F. Levin, “Virtual reality environments for post-

stroke arm rehabilitation,” J Neuroeng Rehabil, vol. 4, pp. 20, June 2007.

[12] A. Viau, A. G. Feldman, B. McFadyen, M. F. Levin, “Reaching in reality and in VR: a comparison of movement kinematics,” J Neuroengineering Rehabil., vol. 1, number 1,

pp. 11, December 2004. [13] L. A. Knaut, S. Subramanian, A. K. Henderson, C. Beaudoin, S. J.

DeSerres, D. Bourbonnais, M. F. Levin, “Comparison of kinematics of pointing movements made in a virtual and a physical environment in patients with chronic stroke,” Soc for Neurosci Abstr., Atlanta 2006, number 451.18, available www.sfn.org/am2006.

[14] S. V. G. Cobb, S. Nichols, A. Ramsey, J. R. Wilson, “Virtual reality induced symptoms and effects (VRISE)”, Presence, vol. 8, (no. 2), pp. 169-186, April 1999.

[15] S. Sharples, S. Cobb, A. Moody, and J.R. Wilson, “Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems,” Displays, vol. 29, (no.20), pp. 58-69, March 2008.

[16] E. B. Werner, Manual of visual fields, New York 1991, Churchill Livingstone.

[17] J.F. Knight and C. Baber, “Effect of head-mounted displays on posture,” Hum Factors, vol. 49, (no. 5), pp. 797-807, Oct 2007.

[18] S. Subramanian, L. A. Knaut, C. Beaudoin, M. F. Levin, “Enhanced feedback during training in virtual versus real world environments”, Proceedings of Virtual Rehabilitation, pp. 8-13, Venice, September 2007.

[19] C. Gowland, P. Stratford, M. Ward, J. Moreland, W. Torresin, S. Van Hullenaar, J. Sanford, S. Barreca, B. Vanspall, and N. Plews, “Measuring physical impairment and disability with the Chedoke-McMaster Stroke Assessment,” Stroke, vol. 24, (no. 1), pp. 58-63, January 1993.

[20] A.R. Fugl-Meyer, L. Jaasko, I. Leyman, S. Olsson, and S. Steglind, “The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance,” Scand J Rehabil Med, vol. 7, (no. 1), pp. 13-31, 1975.

[21] P.W. Duncan, M. Propst, and S.G. Nelson, “Reliability of the Fugl-Meyer assessment of sensorimotor recovery following cerebrovascular accident,” Phys Ther, vol. 63, (no. 10), pp. 1606-1610, October 1983.

[22] M.F. Levin and C. Hui-Chan, “Are H and stretch reflexes in hemiparesis reproducible and correlated with spasticity?,” J Neurol, vol. 240, (no. 2), pp. 63-71, February 1993.

185