Enhanced Depth DiscriminationUsing Dynamic Stereoscopic 3DParameters

Arun KulshreshthDepartment of EECSUniversity of Central FloridaOrlando FL 32816arunkul@knights.ucf.edu

Joseph J. LaViola Jr.Department of EECSUniversity of Central FloridaOrlando FL 32816jjl@eecs.ucf.edu

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bearthis notice and the full citation on the first page. Copyrights for componentsof this work owned by others than ACM must be honored. Abstracting withcredit is permitted. To copy otherwise, or republish, to post on servers or toredistribute to lists, requires prior specific permission and/or a fee. Requestpermissions from permissions@acm.org.CHI’15 Extended Abstracts, April 18 - 23, 2015, Seoul, Republic of KoreaACM 978-1-4503-3146-3/15/04.http://dx.doi.org/10.1145/2702613.2732714

AbstractMost modern stereoscopic 3D applications (e.g. videogames) use optimal (but fixed) stereoscopic 3Dparameters (separation and convergence) to render thescene on a 3D display. However, keeping these parametersfixed does not provide the best possible experience since itcan reduce depth discrimination. We present twoscenarios where the depth discrimination could beenhanced using dynamic adjustments to the separationand the convergence parameters based on the user’s lookdirection obtained from head tracking data.

Author KeywordsStereoscopic 3D; Dynamic Stereo Parameters; HeadTracking; Visual Fatigue; User Study; User Experience.

ACM Classification KeywordsH.5.m [Information Interfaces and Presentation (e.g.HCI)]: Miscellaneous.; K.8.0 [Personal Computing]:Games

IntroductionStereoscopic 3D displays present two images offset to theleft and right eye of the user and these images are thenfused by the brain to give the perception of 3D depth.The generation of these two images uses two stereoparameters: separation and convergence. Separation isdefined as the interaxial distance between the centers of

the two virtual eye camera lenses in the scene and theconvergence is defined as the distance of the plane whereleft and right eye camera frustums intersect (see Figure1). Currently, most stereoscopic 3D applications fixconvergence and separation values for optimal viewingduring usage time. However, this approach reduces stereodepth in certain scenarios. Two examples are when thedepth range has a large variability between differentscenes (e.g. transition from inside a room to an outdoorscene) and when a large object (e.g. a gun in FPS games,the cockpit in air-combat games, etc.) is present in frontof the camera. The fact that these parameters areoptimized to minimize visual discomfort uniformly duringusage usually limits the convergence and separationvalues. Depth discrimination (the ability to judge relativedepths of objects in the scene) in a stereo 3D applicationcould potentially be improved if the stereo parameters aredynamically adjusted based on the scene.

Figure 1: Off-Axis stereoprojection.

We designed two scenarios: a scene with large depthrange variability across different directions and a scenewith a large object in front of the camera. Each of thesescenarios had situations that could be augmented by usingdynamic stereo parameters to enhance depthdiscrimination. We conducted a within subjectsexperiment to evaluate the effectiveness of dynamic stereoparameters (separation and convergence). We examinedqualitative data on a user’s perception of depth,immersion, presence, and visual discomfort.

Related WorkRecent work on stereoscopic 3D found it useful for gamesdepending upon the task involved [6, 7]. Whilestereoscopic 3D has shown some positive benefitsdepending on the task, it also has shown to causenegative symptoms as well, such as eyestrain, headache,dizziness, and nausea [5]. Stereo comfort could be

increased by either changing stereo parameters or usingdepth of field (DOF) blurring.

Several researchers [1, 3] have explored gaze-based depthof field (DOF) effects to minimize visual fatigue.However, people generally disliked the DOF effect withtemporal lag of the gaze-contingent effect being a possiblereason. Ware [8] proposed dynamic adjustment of thestereo separation parameter to minimize visual discomfortand optimize stereo depth. Our work adjusts bothseparation and convergence parameters for a better visualexperience with enhanced depth discrimination.Furthermore, their results revealed that the separationmust be changed gradually over a few seconds to allowusers to adjust without noticing any visual distortion ofthe scene. Bernhard et al. [1] explored dynamicadjustment of stereo parameters using gaze data andfound that it reduces stereo fusion time and provides amore comfortable viewing experience.

The past work on dynamic stereo mentioned above usedsimple static scenes (e.g. random-dot stereograms, apicture, etc.) to evaluate their work. None of the workexplored the benefits of dynamic stereo in complex sceneslike in modern video games. To the best of ourknowledge, our work is the first to systematically exploredynamic stereo for more complex dynamic scenes.

Dynamic Stereoscopic 3DStereo parameters (separation and convergence) could beoptimized based on the type of scene. Ideal applicationcandidates for these optimizations could be classified intwo broad categories. The first category is an applicationwhere there is a large variation in depth range acrossscenes and the second category is an application whichalways has a large object in front of the camera.

Type 1: Large depth range variationThe separation value is dependent on the depth range ofthe scene. For better depth discrimination, the separationis directly proportional to the maximum depth in thescene. Similarly, the convergence distance is also limitedby the depth in the scene for a comfortable viewingexperience. When there is a large depth variation acrossscenes, the separation and convergence values have to beset based on the scene with least depth range. If theseparation and the convergence values are set based on ascene with large depth then they will make another scenewith less depth uncomfortable to look at. Therefore, theseparameters must be changed dynamically from scene toscene for enhanced depth discrimination in all the scenes.

(a) Depth is limited by

wall in this direction

(b) Unlimited depth in

this direction

Figure 2: Scene 1: A scene withvariable depth range acrossdifferent directions.

Figure 3: Scene 2: A scene witha large object in front of thecamera.

We implemented a scene which has a limited depth in onedirection and a large depth range in the opposite direction(see Figure 2). Head tracking is used to control the headof a first person controller (FPC) and a mouse is used torotate the body of the FPC. The convergence value isdynamically changed based on the object being looked atand the separation is changed based on the depth range ofthe scene in front of the camera (see Algorithm 1 fordetails). The convergence and the separation values arechanged gradually, as proposed by Ware [8], to allowenough time for the user’s eyes to adjust.

Type 2: Large object in front of cameraWhen a large object (e.g. a gun in FPS games, thecockpit in air-combat, etc.) is present in front of thecamera, the stereo parameters have to be optimized tokeep that large object always in focus thereby limiting thedepth discrimination ability. However, when the player’shead is rotated/translated, that nearby object may not bein the player’s view and stereo depth could be increasedleading to enhanced depth discrimination.

We implemented an air combat game scene (see Figure 3)as a representative of this category of applications. In the

game, the player has to control an aircraft, using ajoystick, in a first person view controlled using headtracking. In addition, the user can move his/her headcloser to the screen to zoom into the scene foriron-sighting distant enemies. We optimized stereoparameters under two conditions. First, when the user islooking sideways (left/right) and second, when the user iszoomed into the scene (see Algorithm 2 for details). Inboth of these cases, the user is not looking at the cockpit.When the player’s head is rotated sideways (left/right),the separation is increased with linear scaling proportionalto the head’s rotation and the convergence is notchanged. When a user zooms in the scene, the separationis increased with linear scaling proportional to the head’sdisplacement. At the same time, the convergence islinearly decreased with the head’s displacement to keepboth the crosshair and background in focus. Thesedynamic parameters ensured a comfortable stereoscopic3D experience and provided better depth discriminationfor this air-combat game.

Implementation Details

We used Nvidia’s 3D vision for our implementation andthus used the NVAPI library to change the convergenceand the separation. According to the NVAPI library, thenormalized eye separation is defined as the ratio of theinterocular distance (between the eyes) and the displayscreen width. The separation value used in the driver is apercentage of this normalized eye separation and hence isa value between 1 and 100. Convergence is defined as thedistance (in meters) of the plane of intersection of the leftand right eye camera frustums with off-axis (or parallel)projection (see Figure 1). Projection matrices werecalculated automatically by the driver.

Scene 1. For static stereo, the convergence was set to1.0 and the separation was set to 20.0. In the case ofdynamic stereo, the algorithm is described in Algorithm 1.

Algorithm 1 Calculate stereo parameter for scene 1

1: S1 ← separation for lower depth range2: S2 ← separation for higher depth range3: C1 ← convergence for higher depth range4: SF← smothing factor5: threshold← depth threshold6: ∆t← time between frames rendered on screen7: t← SF×∆t8: C← 1.09: S← S1

10: Use raycast to find object Obj in front of camera11: d ← distance of Obj12: if d < threshold then13: C← C + (d− C)× t14: S← S + (S1 − S)× t15: else16: C← C + (C1 − C)× t.17: S← S + (S2 − S)× t.

18: convergence← C19: separation← S

We set SF = 3, threshold = 50, C1 = 30, S1 = 20 andS2 = 50 in the implementation. These values wereobtained based on several pilot studies for scene 1.

Figure 4: The experiment setupconsisted of a 27” BenQXL2720Z 3D monitor, Nvidia 3DVision kit, a TrackIR 5 with ProClip (mounted on a headphone),a Logitech Extreme 3D Projoystick, and a PC (Core i74770K CPU, GTX 780 graphicscard, 8 GB RAM).

Scene 2. For static stereo , the convergence was set to4.0 and the separation was set to 5.0. The dynamic stereoalgorithm is described in Algorithm 2. We setC0 = 4.0, C1 = 0.001, S0 = 5.0, S1 = 60.0, roty1 = 10and roty2 = 60.0 in our implementation. These valueswere obtained based on several pilot studies for scene 2.

User EvaluationsWe conducted an experiment to evaluate the effectivenessof dynamic stereo parameters. We recruited 12participants (10 males and 2 females ranging in age from18 to 33 with a mean age 27.83) from the universitypopulation. The experiment duration ranged from 20 to

Algorithm 2 Calculate stereo parameter for scene 2

1: ihp← initial head position2: mhp← head position when completely zoomed in3: chp← current head position4: roty← current head rotation along y-axis5: roty1 ← min head rotation along y-axis6: roty2 ← max head rotation along y-axis7: C0 ← initial convergence8: C1 ← final convergence after zooming9: S0 ← initial separation

10: S1 ← maximum separation11: C← C0

12: S← S0

13: nnzoom in case14: if |ihp− chp|> 0 then15: C← C1 + (C0 − C1)× (mhp− chp)/(mhp− ihp)16: S← S1 + (S0 − S1)× (mhp− chp)/(mhp− ihp)

17: nnlook left/right case18: if roty > roty1 and roty < roty2 then19: C← C0

20: S← S1 + (S0 − S1)× (roty2 − roty)/(roty2 − roty1)21: else if roty > roty2 then22: C← C0

23: S← S1

24: convergence← C25: separation← S

30 minutes. The experiment setup is shown in Figure 4.We used the Unity3D game engine for implementing thescenes. The TrackIR 5 camera and the Nvidia IR emitterwere mounted on the top of monitor. Participants wereseated about 2 feet away from the display. To make surethat all our participants were able to see stereoscopic 3D,we used the Nvidia medical test image to test stereoabilities of participants and all our participants passed thetest. Note that Nvidia 3D glasses are designed such thatthey can be easily used over prescription glasses withoutany interference to the user.

We chose a within-subjects design for our experiments.Each scene was presented to the participants with bothstatic and dynamic stereo parameters. The users wereasked to judge the relative depth of objects in both thescenes (like the cubes in the first scene and other objectsin the second scene) and based on that they answeredquestions about depth discrimination. While performingthis judgment task, they did not know if the scene useddynamic stereo or static stereo. In addition, they wereasked to rotate their head and not their eyes to lookaround in both scenes. Each condition was presented tothe participants in pre-selected counterbalanced orderbased on a Latin square design. After the experiment, theparticipant filled out a post-questionnaire about eachscene with questions about depth discrimination, userpreference, and visual discomfort.

Qu

esti

on

Sce

ne1

Sce

ne2

DD

Z=

−3.084,p<

0.005

Z=

−3.078,p<

0.005

JD

Z=

−3.086,p<

0.005

Z=

−2.971,p<

0.005

PF

Z=

−2.810,p<

0.05

Z=

−2.638,p<

0.05

Table 1: Results of Wilcoxonsigned rank test for qualitativequestions. DD: DepthDiscrimination, JD: Judgment ofDepth and PF: Preference

ResultsTo analyze the Likert scale data, we used Wilcoxon signedrank test with α = 0.05. The results for the qualitativequestions are summarized in Table 1 and mean values areplotted in Figure 5. Compared to static stereo:

• depth discrimination was significantly improved withpresence of dynamic stereo.

• significantly more people felt that they were able tocorrectly judge the relative depths of objects inscenes when dynamic stereo was present.

• significantly more people preferred using dynamicstereo.

Except for one participant, no one felt any significantlynegative symptoms by watching the scenes in stereoscopic3D (static as well as dynamic). One participant was verysensitive to stereoscopic 3D. He experienced moderate eyestrain and discomfort with both static as well as dynamicstereo.

DiscussionOur scenes were designed keeping stereoscopic viewing inmind and used design guidelines from the literature [6, 7].We chose the separation and the convergence values foreach scenario such that the visual discomfort wasminimized. During our pilot testing, these values wereoptimized based on user feedback to ensure that they arecomfortable for most users. Most of our user studyparticipants did not experience any visual discomfort witheither static or dynamic stereo.

Our study also had some limitations. We used headtracking data to approximate the user’s look direction.But, a user may not always be looking straight aheadsince the eyes could look in a different direction. Weasked our users to rotate their head and not their eyes tolook around in the scene. However, this was not naturaland could have a minor effect on our results. We expectthat using an eye tracker would even further improve ourresults. We did not consider the variation in interoculardistance between the users in our experiments. However,we expect that the results would be similar since ouralgorithms use (see implementation details) the ratio ofinterocular distance (between 58mm and 70mm [2]) anddisplay width (27 inch in our experiment) which isminimally affected by this variation in interocular distance.In addition, our small sample size (12 participants) couldhave a minor affect on our results.

We would like to mention that the use of dynamic stereowould change the geometry of the scene (e.g. an increasein separation makes the world seem smaller and/or theobserver feel larger) and may not be a good idea insituations where scale is of critical importance such as incase of industrial design applications. Regardless, ourresults indicate that dynamic stereo has potential toimprove depth discrimination in stereo 3D applications.Future application designers should use dynamic stereoadjustments to provide a better experience to the user.

However, these parameters should be chosen wisely, basedon the scene, to minimize visual discomfort.

1234567

DD

JDPF

DD

JDPF

Scene1

Scene2

Likert Scale (95% CI)

Questionnaire M

ean

 Ratings

Static Stereoscopic3D

Dynam

ic Stereoscopic 3D

DD

Depth

Discrim

ination

JD    Judgement of Depth

PF    Preference

Figure 5: Mean qualitativeratings for both scenes based ontype of stereoscopic 3D

Conclusion and Future WorkWe presented two scenarios where optimizing the stereoparameters (separation and convergence) could enhancethe depth discrimination of the user. Our preliminaryresults indicate that participants preferred to use dynamicstereo over static stereo since it significantly improved thedepth discrimination in the scene. Our study is apreliminary step towards exploring the effectiveness ofdynamic stereo in stereoscopic 3D applications and furtherresearch with more scenarios is required.

In our experiment, the values of the stereo parameterswere determined based on our pilot studies. However, webelieve that these values could be expressed in terms ofdisplay size, distance of the user from the display, anddistance of the object being looked at in the scene. Weplan to explore this direction in future work. Furthermore,we did not consider any quantitative measures as part ofthis work. We plan to include depth judgment tasks (e.g.Howard-Dolman test [4]) in our future experiments toquantify the differences between dynamic and static stereoscenes. In addition, we used head tracking data toapproximate the user’s look direction. In the future, wewould like to use eye tracking to get more accurate gazedirection and it should further improve our dynamic stereoalgorithms.

AcknowledgmentsThis work is supported in part by NSF CAREER awardIIS-0845921 and NSF awards IIS-0856045 andCCF-1012056.

References[1] Bernhard, M., Dell’mour, C., Hecher, M., Stavrakis,

E., and Wimmer, M. The effects of fast disparity

adjustment in gaze-controlled stereoscopicapplications. In Proceedings of the Symposium on EyeTracking Research and Applications, ETRA ’14, ACM(New York, NY, USA, 2014), 111–118.

[2] Dodgson, N. A. Variation and extrema of humaninterpupillary distance, 2004.

[3] Duchowski, A. T., House, D. H., Gestring, J., Wang,R. I., Krejtz, K., Krejtz, I., Mantiuk, R., and Bazyluk,B. Reducing visual discomfort of 3d stereoscopicdisplays with gaze-contingent depth-of-field. InProceedings of the ACM Symposium on AppliedPerception, SAP ’14, ACM (New York, NY, USA,2014), 39–46.

[4] Howard, H. J. A test for the judgment of distance.Transactions of the American OphthalmologicalSociety 17, 195-235 (1919).

[5] Howarth, P. A. Potential hazards of viewing 3Dstereoscopic television, cinema and computer games:a review. Ophthalmic & Physiological Optics : TheJournal of the British College of Ophthalmic Opticians(Optometrists) 31, 2 (2011), 111–122.

[6] Kulshreshth, A., Schild, J., and LaViola Jr., J. J.Evaluating user performance in 3D stereo and motionenabled video games. In Proceedings of theInternational Conference on the Foundations of DigitalGames, ACM (New York, NY, 2012), 33–40.

[7] Schild, J., and Masuch, M. Fundamentals ofstereoscopic 3D game design. In ICEC, InternationalFederation For Information Processing (2011),155–160.

[8] Ware, C. Dynamic stereo displays. In Proceedings ofthe SIGCHI Conference on Human Factors inComputing Systems, CHI ’95, ACMPress/Addison-Wesley Publishing Co. (New York, NY,USA, 1995), 310–316.