Verifying Sensorimotoric Coordination of Augmented Reality Selection under Hyper- and Microgravity

8/13/2019 Verifying Sensorimotoric Coordination of Augmented Reality Selection under Hyper- and Microgravity

1/10

International Journal of Advanced Computer Science, Vol. 3, No. 5, Pp. 217-226, May, 2013.

ManuscriptReceived:

25,Feb.,2013

Revised:

17,Mar.,2013

Accepted:

2,Apr.,2013

Published:

15,Apr.,2013

KeywordsHuman

Computer

I nteraction,

Augmented

Reality,

Aerospace,

L if e Science

AbstractFu ture user interf ace technologies

will shift away from conventional displays,

mice and keyboards and will claim the joint

use of our physical wor ld. Augmented Reali ty

bri dges this gap by enhancing the real world

with vir tual in formation that can lead to an

improved perception of our dail y work l if e or

complex tasks. At altered gravitational

conditi ons, working in space station denotes an

increased workload of astronauts

perf ormances of on-board activi ties. The use of

Augmented Reali ty wil l support the space crew

at handling intra-vehicular displays and

control items in a natural manner. We present

an experiment that has investigated the impact

of hyper- and microgravity whil e perf orming

selection tasks in a head-mounted Augmented

Reali ty environment. We were interested in the

correlati on between the human body frame of

reference and haptic feedback by precise

pointing movements towards a target. To

evaluate sensorimotoric coordination and

workload we performed a comparative user

study under parabolic f li ght conditions. In awithin-subject design we evaluated different

placement configurations of a virtual

keyboard. The objective measures showed a

signif icant requir ement of haptic feedback.

1. IntroductionToday working with desktop or laptop computers still

implies traditional WIMP (Windows, Icons, Menus, Pointer)

style using mouse, keyboard and monitor devices. More andmore handheld devices are becoming available in the form

of PDAs, smartphones or tablet computers that are operatedby 2D touch user interfaces. Less common are immersiveVirtual Reality systems (e.g., powerwall, CAVE,head-mounted displays) that require different user interfacetechniques, like 3D mice, joysticks or data gloves, andsix-degree-of-freedom tracking systems. But none of theminvolves the physical reality for a direct interface enhancedwith 3D registered virtual information that can increase ourperception in our daily life, but also in facilitatedperforming of complex tasks. As future interface technology,Augmented Reality (AR) closes this gap by combining

German Aerospace Center (DLR), Germany; University of Rostock,

Germany, Julius Khn Institute, Germany ([email protected],

eckard..mmoll@jki. .bbund.de, [email protected])

physical-world and computer-generated data, which

augment the real world in real-time [1].On the International Space Station (ISS), the crew has to

perform intra-vehicular operations that include the handlingof display and control items, or on-board activities, like the

performance of maintenance and experimentation tasks atcomplex research payloads. Here the astronaut is guided bya standardized procedure that is displayed on a laptop

computer, the Crew Commanding Station (CCS), and is anessential factor for success or failure. Overall, our researchaims at the development and evaluation of a mobile ARassistance system for space operations to improve thesupport for the operational ground team and the space crewin performing service and maintenance tasks on-board theISS. A key aspect of our research is related to theexploration of innovative 3D interaction techniques in ARenvironments to support interaction with virtual content,focusing on the evaluation of selection techniques thatrequire precise pointing movements. Working in spaceunder changed gravitational conditions implies an increasedworkload of the astronauts performance. Previous studies

related to the impact of different acceleration conditionswhile performing of AR selection tasks ought to beconsidered at our current design stage.

In this paper we present a study conducted to quantifyand qualify the effects of hyper- and microgravity on humansensorimotoric coordination while pointing towards virtual

targets attached and displayed in the real world by anoptical see-through (OST) head-mounted display (HMD). In

response to visual stimuli we verified the interactionbetween the spatial orientation and the support of hapticfeedback for precise object selection by pointing towards atarget using a virtual keyboard (see Fig. 1).

Fig. 1 The Augmented Reality interface for the selection task to investigate

the sensorimotoric coordination under changed gravitational conditions

We investigated human hand-eye coordination andworkload while applying different placement configurationsof the virtual keyboard. Bowman et al. [2] suggest that by

Verifying Sensorimotoric Coordination of Augmented

Reality Selection under Hyper- and MicrogravityDaniela Markov-Vetter, Eckard Moll, & Oliver G. Staadt


2/10


International Journal Publishers Group (IJPG)

21

using a virtual keyboard haptic feedback is an importantcomponent. When using virtual keyboards, the virtual keysshould be placed on a physical surface. In contrast to that,studies in microgravity [3], [4] have shown that movementsexecuted toward a target in external space (e.g., pointing

movements) are deteriorated in microgravity. Based onthese findings our study aimed at identifying of commonbasic principles regarding the configuration of interactiveAR interfaces under hyper- and microgravity conditions incorrelation with the spatial orientation and the support ofhaptic feedback. That resulted in three placementconfigurations (see Table 1), whereby we distinguished thespatial orientation between inside and outside of the humanbody frame of reference associated with and without hapticfeedback. Two modalities are placed inside the human bodyframe of reference (HAHMD view configuration, BAbody configuration) and one in external space (PA physical surface configuration). Additionally, the body and

the physical surface configuration support passive hapticfeedback.

TABLE1THE PLACEMENT CONFIGURATIONS OF THE VIRTUAL KEYBOARD

The goals of this study were: (1) to verify the feasibility

of using AR interfaces under parabolic flight conditions; (2)

to compare performance, efficiency of pointing tasks in allconfigurations (HA, BA, PA) under hypergravity andmicrogravity conditions; and (3) to determine the subjective

cognitive workload, impression and satisfaction. Wehypothesized that the attachment of virtual 3D AR

interfaces at humans body (BA, see Fig. 2) increases thestability and accuracy of controlling a performed pointingtask under hyper- and microgravity. To compare the results,a ground study under 1-g condition was performed. Theresults should identify special requirements and earlyconsideration of the influence of different accelerationconditions for our ongoing developing of an AR supportedassistance system for space operations. This paper

represents an extended version of [5] presenting additionalresults.

Fig. 2 The BA configuration is inside the human body frame of referenceand with haptic feedback

2. MethodologyTo investigate human adaption of handling and

controlling pointing tasks using different keyboardconfigurations impacted by hyper- and microgravity, we

performed a comparative user study under parabolic flightconditions. The experimentation was performed during the56

th ESA (European Space Agency) Parabolic Flight

Campaign in May 2012 on board the Zero-G Airbus A300in Bordeaux, France, performed by NOVESPACE andfunded by the ESA (see Fig. 3). A parabolic flight [6] is anaircraft maneuver that provides one period up to 20 secondsreduced gravity or weightlessness and two periods ofincreased gravity (~1.5-gto 1.8-g).

Fig. 3 The participants of the 56thESA Parabolic Flight Campaign in May

2012 in Bordeaux, France, performed by NOVESPACE (Photo ESA)

We conducted the study during three flight days with 31parabolas per flight day, whereas the first parabola was fortest. Fig. 4 shows the parabolic flight profile that wasprovided on all flight days. Each parabola takes around 3minutes and includes the up- and downwards hypergravityphases (1.8-g) and the microgravity phase (0-g).

For the experimentation in flight we used the upwardshypergravity phase (22 sec), the microgravity phase (22 sec)and 22 sec of the 1-g phase (see Fig. 5). The comparableground study (1-g PRE, 1-g POST) was performed in theaircraft one day before and directly after the flight. To avoidmotion sickness during the flight, each subject was

administrated with Scopolamine. Each test was performedin a lying posture. In microgravity the subject came up in a

Mode Description Reference

system

Haptic

feedback

HA The keyboard is fixed tothe HMDs field of view,like a head up display. The

location is not becontrolled by the subject.

inside no

BA The keyboard is displayed

on a two dimensionalsurface that is held in thesubjects hand. Thelocation is controlled by

subjects hand movements.

inside yes

PA The keyboard is displayedon a real fixed surface in

front of the subject. Thelocation is controlled by

subjects body and headmovements.

outside yes


3/10

Author I et al.:the full paper title comes here.


219

horizontal floating posture around 5 cm above the floor.

Fig. 4 The parabolic flight profile for 31 parabolas of the 56 thParabolic

Flight Campaign of one flight day provided for two subjects

For providing haptic feedback with the PA keyboard

configuration we designed a swinging arm (see Fig. 6)equipped with a board that could be moved and rotatedvertically. During flights, two subjects and one operatorperformed the experiment. While the subjects completed theexperiment task, the operator was responsible for thecoordination and the data collection unit. Each subjectperformed the experiment within 15 parabolas, while hewas assisted by the other subject (see Fig. 4).

Fig. 5 The experimentation schedule and the used phases of one parabola

sequence

A. SubjectsThe experiment was performed by 6 participants (1

female and 5 male) with an average age of 39.6 years. Four

subjects had experienced parabolic flights previously andtwo subjects were novices. Four subjects did wear glasses

and all subjects were right-handed. Four subjects had adominant right eye, while the remaining two subjects had a

dominant left eye. All subjects received Scopolamine byinjection as anti-motion-sickness medication 0.5mg (1female, 1 male subject), 0.7mg (1 male subject) and 1.0mg(3 male subjects). The subjects were precisely informedabout the experiment protocol and signed an informedconsent form.

Fig. 6 The swinging arm used for the PA configuration outside the human

body frame of reference and with haptic feedback

B.ApparatusThe experiment setup (see Fig. 7) consisted of a head

mounted display (HMD), an optical sensor and tangible ARinterfaces [7] as input devices. The HMD was a monocularoptical see-through (OST) HMD (dataGlass2/a from

Shimadzu), which has a semi-transparent LCD display witha resolution of 800x600 pixels, a diagonal field of view

(FOV) of 30 degrees, and the support of human eyeshorizontal peripheral FOV of almost 180 degree. Thedisplay was adjusted to the right eye. For tracking purposeswe equipped the HMD with an optical sensor (MicrosoftHD 5000 webcam with 66 degree diagonal FOV). TheHMD was mounted at a bicycle helmet, that allowed aquick change of the HMD setup that was connected to thedata processing unit (Lenovo Thinkpad T420s, 2.8 GHz

CPU, NVIDIA Quadro NVS 4200M).To obtain good registration of the tracked objects with its

rendered display position while using an OST HMD, a usercentered calibration method [8] was required. Thereby theexact position of the subjects eye relative to the opticalsensor was measured. Each subject has performed theself-calibration once before experimentation in aircraft. Toobtain the position and orientation of the subjectsviewpoint, we used the mounted webcam for inside-outoptical tracking. Thereby the optical sensor detects physical

markers in real time and subsequently the real camera poseis calculated. We used two different physical markers: a

keyboard marker for displaying the virtual keyboard and a

pointing marker for subjects right forefinger to pointtowards the keyboard. For the BA keyboard configurationthe keyboard marker was fit on subjects left hand, and for

the PA configuration the keyboard marker was attached tothe board of the swinging arm. Such interfaces are known astangible AR interfaces and enable effective spatialcomposition and arrangement of virtual objects in physicalspace.

For cases of unexplainable effects and to reconstruct theperformance in flight later, we have recorded the completeexperimentation with a video camera (Sony HD Handycam)that was mounted in aircraft above the experiment setup.The operational software system was written in C++ using

Qt 4.6.2 for operators 2D user interface, ARToolkitProfessional [9] library for marker tracking and


4/10



22

OpenSceneGraph 2.8.3 [10] for rendering the virtualcontent.

Fig. 7 The apparatus: (left) the HMD-camera setup, mounted onto a bicycle

helmet, (right) the experimentation area in the aircraft

C. Task and ProcedureWearing an optical see-through HMD, the subject entered

random letters onto the virtual keyboard (see Fig. 1), which

were signaled in green. For the virtual keyboard layout weuse the common QWERTZ keyboard layout. By hitting acorrect key, it was signaled in red. Before the experiment,the subject attached the pointing marker on the rightforefinger and adjusted a virtual 3D hand model to thefingertip. In case of the BA keyboard configuration, thesubject also attached the keyboard marker to the left hand.By entering the hypergravity and the microgravity levels,the operator started the experiment session timer and thefirst key was signaled in green. Not before the subject hitthe first signaled key, the subjects session timer was started.Only if the subject hit a key in the right way, the nextrandom key was signaled. After 22 seconds or by operatorsmanual command, the session stopped. Under 1-g

conditions the operator started and stopped the system byhimself. During the session the operation system recordedall inputs of the subject and the operator.

For scheduling the experiment session in flight (see Fig.5) we timed our procedure strictly by the pilots audioannouncements of trajectory: Pull Up (increased Gz-loadup to 1.8-g) and Injection (rapid fell Gz-load to ~ 0-g).The 1-g IN-FLIGHT session was started by the operatorafter the downwards hypergravity phase and after the airconditioning switched on. The experiment procedure wasconducted on ground in the aircraft and in flight. Table 2

shows and describes all acceleration levels. The medicationwas applied in all levels, except in the 1-gPRE level. The

subjects were lying on the floor and in 0-g IN FLIGHThorizontal floating.

For subjective measurements, the subject completedquestionnaires after performing the experiment tasks of one

keyboard configuration. During the flight, that was done inthe breaks (see Fig. 4). An overall questionnaire wascompleted after complete the experimentation. To overcomeproblems with handling of the physical marker (keyboardand pointing device) and the parabola schedule, allparticipated subjects practiced the experimental procedurein advance on the ground. Because subjects are not used tohandle virtual keyboards in AR environments and to avoidthat subjects feel rushed, the operation software supports an

artificial delay of 20 image frames after each entered key.

D. Study DesignThe user study consists of three independent variables

(keyboard configurations: HA, BA, PA) on five levels(acceleration conditions: 1-g PRE, 1.8-g IN-FLIGHT, 0-g

IN-FLIGHT, 1-g IN-FLIGHT, 1-g POST). In a

within-subject design, each participant performed theexperiment procedure for all independent variables in all

levels that resulted in a factorial design of 5 x 3 = 15. Toensure the functional capability under the strict

experimentation conditions we could not use variations ofthe presentation order for counterbalancing. The taskrepeating rate for each variable amounted to five times ateach level. That resulted in 15 x 5 = 75, (i.e., each subjectperformed 75 tasks). Overall, 450 (75 tests x 6 subjects)data samples were expected.

We have identified several confounding factors (e.g.,HMD calibration error, unstable lighting conditions, etc.)and corresponding counteractions that we did not study, but

could still affect the experiment. For reconstruct the subjectperformance and to identify further confounding factors, weused the video sequences that we have recorded in flight.

TABLE2THE EXPERIMENTATION CONDITIONS (GRAVITATION VS.MEDICATION)

E.MeasuresFor evaluating the efficiency of the virtual keyboard

configurations, three different types of measures were used:(1) pointing behaviour and performance measures; (2)physiological measures; (3) subjective measures. Thepointing behaviour and performance were measured bysubjects response time of hitting a key, the frequency ofcorrect pointing and the pointing error rate by the frequencyof incorrect pointing. Subjective measurements werecollected by a rating procedure and questionnaires. Thesubjective workload was measured with the NASA RTLX(Raw Task Load Index) and contains the same items as theNASA TLX [11], but without the paired comparison stage[12], which can be often be a lengthy process, and difficultto carry out under parabolic flight time conditions. Thesubjects had to rate their mental, physical and temporaldemand, performance, effort and frustration level after

performing the tests in one keyboard configuration on acontinuous rating scale from very low to very high (1-100).

The subjects rated their workload for all three keyboardconfigurations (HA, BA, PA) after performing PRE,

IN-FLIGHT and POST tests. Because of time limitations,additional differentiation between the gravitation levels in

flight was not possible. Thus the IN-FLIGHT tests include

Level Description Medication

1-g PRE one day before flight, inaircraft

no

1.8g IN FLIGHT upward hypergravity phase yes

0-g IN FLIGHT weightlessness phase yes

1-g IN FLIGHT between the parabolas yes

1-g POST after the flight on ground, inaircraft

yes


5/10



221

1.8-g, 0-g, 1-ggravitation level. PRE (without medical) andPOST (with medical) test were performed under 1-gcondition on ground in aircraft.

Furthermore we designed several post-flightquestionnaires to obtain information about the overall

impression, the quality of usability, subjects satisfactionand personal preferences. After performing the PRE,IN-FLIGHT and POST tests the subjects completed aquestionnaire that was measured the usability, satisfactionand coordination. The subjects were asked how well theycould select the keys (usability), how satisfied they werewhile selecting (satisfaction) and their ability to coordinatethe finger thereby (coordination). The answers were givenon a 5-Likert scale formed by an ordinal sequence (veryeasy, easy, partly, difficult, very difficult). For gettingpreferences the subjects were asked for their preferences:for PRE and POST tests the subjects should make a decisionwhich keyboard configuration (HA, BA, PA) was the best

for performing the test; for IN-FLIGHT test the subjectsshould decide which gravitation level (1.8-g, 0-g) theywould prefer to handle the performed keyboardconfiguration. Finally, for overall impression, aftercompleting all levels of the experiment the subjects

completed a final questionnaire with eight questions: (1) Iunderstand my mistakes quickly and easily. (2) I use the

keyboard successfully at any time. (3) The use of thekeyboard is effortless. (4) The use of the keyboard is easy to

learn. (5) I was trained quickly with the keyboard. (6)Overall I was satisfied with the handling of the keyboard. (7)The key selection works like I wanted it. (8) The keyboard iscomfortable to use. The answers should rate for all three

keyboard configurations (HA, BA, PA) separately on a5-Likert scale formed by an ordinal sequence (full agree,agree, undecided, disagree, full disagree).

3. Data AnalysisThe expected number of data sets was 450 data sets (270

in flight, 180 on ground in aircraft). Because one subject didnot performed the PRE test, we evaluated 435 (270 flight,165 ground in aircraft) data sets. In flight we evaluated alldata sets of 90 parabolas, not using the first parabola in eachflight. In general one subject had problems with adjusting

the HMD at first flight day. For reproducibility acomparison of ground tests (PRE, POST) and IN-FLIGHT(1.8-g, 0-g, 1-g) tests was performed. Based on the subjects completion time we present mean value, standard deviation(SD) and the percentage error rate of key hits. The meanvalue was determined by the median. We evaluated therelative frequencies of the number of correct and false keyhits in consideration of the Logit transformation as Linkfunction and the binomial distribution as probabilitydistribution depends on the gravitation level (level), the

keyboard configuration (config) and their interaction(level*config). Based on an overall interaction, we

compared level*config with level and accordingly with

config. The statistical test was the Chi-square -homogeneitytest (Proc GENMOD, SAS 9.3).

Additionally we grouped the data in different levels

haptic feedback and inside/outside of subjects bodyreference system. Because the signaled keys onto the virtualkeyboard were computed randomized online, we checkedup the frequency of signaled keys by level*config for allsubjects together and each subject separately. Regarding the

ratings and questionnaires, one subject was not able tocomplete the questionnaires. Therefore we evaluatedquestionnaires of five subjects. For evaluation the NASARTLX, we averaged the individual scales to obtain a totalworkload score of all subjects. The ordinal scaledpost-questionnaires were analyzed by theCochran-Mantel-Haenszel Chi-square test. For statisticalanalysis the significance level was set at p < .05. Becausethe results were widely scattered and the number of subjectswas limited to six, we also considered results with p > .05and < .10 as approaching significance to identify as possibleupcoming trends.

4. ResultsThe experimentation system worked very well at an

average rate of 41 frame per second (SD=11 fps). Becausethe Zero-G aircraft has stable environment conditionsregarding lighting, the optical marker-based tracking systemwas very stable and did not require additional adjustment ofthe video lighting threshold value during operation.

All three keyboard configurations (HA, BA, PA) wereapplicable under all gravitation level. Fig. 8 shows the BAand PA configurations in operation under 1.8-g and 0-g.The task of typing random letters was performed by each

subject under all three keyboard configurations successfully.We carried out analysis to establish differences between theacceleration conditions for all independent variables of the

keyboard configuration.

Fig. 8 IN-FLIGHT experimentation: (left) the BA configuration under the

hypergravity condition; (right) the PA configuration under the microgravitycondition

A.Pointing Behaviour and PerformanceA total of 3524 keys were signaled for the IN-FLIGHT

levels (1.8-g, 0-g, 1-g). Table 3 shows the numbers ofsignaled keys for each keyboard configuration per eachgravitational level. Because pointing movements performedunder microgravity are slowed [13], overall the lowestnumber of keys was signaled under 0-g. Under 1.8-g themost keys were signaled for the PA keyboard and the fewestnumber of keys were signaled for the HA keyboard. Under

microgravity the most number of keys were signaled for theBA keyboard and the fewest number also for HA.


6/10



22

Overall the subjects achieved a median number of correct

key hits of 13 (SD=4) and of false key hits of 2 (SD=3) thatresults in a 15.4% error rate. The averaged completion timeof 20188 ms (SD=1727) with averaged 13 correct keysresults in 0.64 keystrokes per second.

TABLE3NUMBER OF OVERALL SIGNALED KEYS

*only five subjects

Key hits by acceleration levels: Independent of the

keyboard configurations and distinguished between thegravitation level 1-g PRE, 1-g POST and the IN-FLIGHTlevels 1.8-g, 0-g, 1-g and 1-g the subjects achieved thefewest number of correct key hits under microgravity of

averaged 11 and the highest average number of false keyhits of 3 that results in an error rate of 27.3% at a mean time

of 20250 ms (0.54 keystroke per second). Table 4 shows allmean key hits with its standard deviation and the error rate

for all IN-FLIGHT levels. In PRE-tests and 1.8-glevel theaverage number of correct key hits was 13 (PRE: SD=3,1.8-g: SD=4) with an error rate of 15.4% completed inaveraged 19766 ms (0.65 keystroke per second). In 1-gIN-FLIGHT level and POST-tests the subjects hit averaged

14 correct keys (1-gIN: SD=3, POST: SD=4) with thelowest error rate of 14.3% at an averaged time of 20297 ms(0.68 keystroke per second).

TABLE4MEAN KEY HITS,STANDARD DEVIATION AND ERROR RATE BY

GRAVITATIONAL LEVEL OF THE IN FLIGHT PROCEDURE

Key hits by keyboard configurations: If we consider

only the keyboard configurations HA, BA, and PA, Table 5shows the mean key hits, standard deviation and error ratepresented by the configurations. The performance of thepointing task with the HA configuration lead to the lowestcorrect key hits at a mean of 10 (SD=4) with the highestnumber of false key hits of 3 (SD=3) that result in error rateof 30%. In contrast, the configurations BA and PA led tonearly same results with correct key hits with a mean of 15(SD=3) and 14 (SD=3) and error rates of 13.3% and 14.3%.

Grouped the configurations by haptic feedback, BA_PA(with haptic feedback) resulted to averaged 14 (SD=3)correct key hits and averaged 3 false key hits (SD=2) withan error rate of 21.4%. Grouped by body reference system,

HA_BA (inside) led to a median of 12 correct key hits(SD=4) and 2 false key hits (SD=3) that resulted in an error

rate of 25%.(SD=3) and 14 (SD=3) and error rates of 13.3% and

14.3%.

TABLE5MEAN KEY HITS,STANDARD DEVIATION AND ERROR RATE BY KEYBOARD

CONFIGURATION

Key hits by keyboard configurations grouped by

acceleration level:Fig. 9 shows the number of entered keysof the keyboard configurations at each acceleration level ofall subjects over all data sets (270) of the IN-FLIGHTprocedures. The median of the number of correct keys hits

using the BA configurations under 1.8-gand 0-gconditionswas greater than that of those with HA and PA. The lowestmedian number of false key hits was achieved with the BAconfiguration under 1.8-gcondition and the highest medianalso in BA but under 0-gcondition. The HA configurationachieved under 1.8-gand 0-gconditions the worst results ofcorrect key hits. Under 0-g condition the PA keyboardachieved the lowermost median number of false key hits.

Fig. 9 Comparison of the number of entered keys with all keyboardconfigurations at each level of the IN FLIGHT procedure

Analysing the relative frequencies of the number of

correct and false key hits depends on the gravitation level

(level), the keyboard configuration (config) and theirinteraction (level*config), we compared level*config onsame level stage and same config stage. Applying the

Chi-square test there are interesting results on same configstage. A significant difference on the same levelstage (see

Table 6) was revealed on the config stages HA and PAcompared to all level stages with the 0-g level: p=0.0305< .05. On same configstage significant differences could berevealed on level stages 0-g_1.8-g compared to all configstages with BA and HA (see Table 7).

If we consider only the subjects with experiences inparabolic flight (four subjects) there are significantdifferences on same level stage and config stages. On the

level stage 1.8-g the HA configurations differedsignificantly from BA and PA with pBA=0.0198 < .05and

HA BA PA

number of correct keys 10 (SD=4) 15 (SD=3) 14 (SD=3)

number of false keys 3 (SD=3) 2 (SD=3) 2 (SD=2)

error rate (in %) 30.0 13.3 14.3

1.8-gIN 0-gIN 1-gIN

number of correct keys 13 (SD=4) 11 (SD=4) 14 (SD=3)

number of false keys 2 (SD=3) 23(SD=3) 2 (SD=2)

error rate (in %) 15.4 27.3 14.3

Level HA BA PA

1-g PRE 285* 378

* 369

*

1.8-g IN FLIGHT 335 435 440

0-g IN FLIGHT 234 414 374

1-g IN FLIGHT 368 452 472

1-g POST 257 460 485


7/10



223

pPA=0.0867 < .10. Also in the levelstage1-gPOST was aapproaching significance of p=0.0771 < .10 of HAcompared to PA. On same config stage significantdifferences were resulted from all keyboard configurationson all levels compared 0-g and 1.8-g. Because we

distinguish between two different categories of keyboardconfigurations body reference system and hapticfeedback we performed the Chi-square test for both.Grouped by the body reference system HA_BA (inside) andPA (outside), there is an approaching significant differenceof p=0.0672 < .10 at 0-g on same level stage. On sameconfigstage the group HA_BA is significant by comparisonof 0-gand 1.8-g(p=0.0021 < .05) and by comparison of 0-gand 1-g IN-FLIGHT (p=0.0017 < .05). By using a greatersample size, at same or smaller variability a statisticaldifference between HA_BA and PA can be assumed.

TABLE6INTERACTION LEVEL*CONFIG ON SAME LEVEL STAGE

TABLE7INTERACTION LEVEL*CONFIG ON SAME CONFIG STAGE

Grouped by the haptic feedback BA_PA (with hapticfeedback) and HA (without haptic feedback), there wasachieved an approaching significant difference of p=0.0899< .10on same levelstage at 0-g. On same configstage the

group BA_PA differed significantly by comparison of 0-gand 1.8-g(p=0.0208 < .05) and by comparison of 0-g and

1-gIN-FLIGHT (p=0.0052 < .05). In contrast the group HAdiffered by comparison of 0-gand 1.8-g (p=0.0557 < .10)

and by comparison of 0-g and 1-g IN-FLIGHT (p=0.0293< .05) on same configstage. Related to the virtual keyboard,the frequency of randomized signaled keys measured by theinteraction level*config was very well distributed for allsubjects and each subject separately.

B. Subjective MeasurementsSubjects workload (NASA RTLX): Using the means

of 5 subjects, Fig. 10 shows the ratings of the IN-FLIGHT

procedure including 1.8-g, 0-gand 1-g, whereby the loadingfactors were in general rated lower for the PA configuration.

Altogether, interesting are the variations between the BAand HA mode. While in PRE tests in nearly all loading

factors (except performance and effort) the BAconfiguration was rated higher, in the other conditions

(IN-FLIGHT and POST) the subjects rated the HAconfiguration clearly higher.

Fig. 10 Subjective workload for the in-flight procedures (1.8-g, 0-g, 1-g)

measured by rating the NASA RTLX

Impression, satisfaction in PRE and POST tests:Subjects ratings of the keyboard configurations in terms of

usability, satisfaction and coordination in PRE tests(without medical) and POST (with medical) have shown

that the subjects could easy select the keys (usability) withthe PA keyboard. While in PRE tests with the BA keyboardsimplicity was not clear, but in POST test the subjects wereable to select the keys easily. Asking for satisfaction, theanswers in PRE tests were well distributed in the better halfof the scale. However, in POST tests the subjects were clearsatisfied with PA and BA in the same way. For the HA

keyboard the subjects were more satisfied in POST test as inPRE test. Regarding the fingertip while selecting, in PREtest the subjects could coordinate their finger with the HAand PA keyboard easier as with the BA keyboard. Incontrast, the subjects could coordinate easily with all threekeyboards in POST tests. Using statistical analysis therewere no significant differences in answering the questions.Asking after personal preferences, under 1-g conditionduring PRE tests without medical, 1 subject preferred HAkeyboard and 1 subject preferred the BA. In POST testswith medical after the flight 2 subjects preferred the BA

keyboard. In PRE and POST tests 3 subjects preferred thePA keyboard.

Impression, satisfaction of IN-FLIGHT procedures:While performing the IN-FLIGHT tests, the subjects

performed one keyboard configuration within 5 parabolas,whereby, one parabola includes tests in 1.8-g, 0-gand 1-g.

We have identified clear differences in ratings the keyboardconfiguration between the levels 1.8-g, 0-gand 1-gin termsof usability, satisfaction and coordination (see Fig. 11).While under 1.8-gcondition the answers were partly in thebetter half, under 0-gthe answers were distributed over thecomplete scale. Averaged the subjects were only partlysatisfied with the key selection. Asking after usability andcoordination of the HA keyboard revealed in a distributionfrom easy to difficult. In contrast, under 1-g condition the

answers were clear in the better part of the scale for allkeyboard configurations. All factors were ranked as veryeasy or easy. Using statistical analysis there were no

level level config p-value

0-g 1.8-g HA 0.0481< .05

0-g 1.8-g BA 0.0063< .05

0-g 1.8-g PA 0.5937> .05

config config level p-value

BA HA 1.8-g 0.1371> .05

BA PA 1.8-g 0.4043> .05

HA PA 1.8-g 0.4707> .05

BA HA 0-g 0.2898> .05

BA PA 0-g 0.1755> .05

HA PA 0-g 0.0305< .05


8/10



22

significant differences. Choosing which acceleration levelwould be the best for the PA and the BA keyboard, twosubjects preferred to handle the keyboards under 1.8-gand 3subjects preferred the handling under 0-g. In contrast to thatall 5 subjects preferred the 1.8-g level to use the HA

keyboard.

Fig. 11 Subjective measurements for usability, satisfaction andcoordination rated for the IN-FLIGHT levels by the post-questionnaire

Final overall questionnaire: All subjects could

understand their mistakes in a quickly and easy with the PAand the BA keyboard. Using the HA keyboard only threesubjects could understand it. Successful using of thekeyboard was agreed of all subjects for the BA keyboard,mostly for the PA and nearly all was undecided for the HAkeyboard. Nearly all subjects used the PA keyboard andmore than the half of the subjects the BA keyboardeffortless. All subjects were the opinion that the PA and BAkeyboard is easy to learn and that they were trained quickly.With HA three subjects agreed with this, one was undecidedand one subject disagreed with this. All subjects weresatisfied with handling the PA keyboard, and nearly allsubjects with the BA keyboard. Three subjects were not

satisfied while using the HA keyboard. Nearly all subjectswere the opinion that the key selection with PA works like

their wanted it. The BA keyboard works for three subjectslike their wanted, and with the HA keyboard three subjects

were undecided and two subjects did not like the operatingmode. Comfortable using was agreed for the PA keyboard

from all subjects. For four subjects the BA keyboard wasalso comfortable in handling, but for more than the half of

the subjects the HA keyboard was uncomfortable.

5. DiscussionWe established that pointing towards a virtual keyboard

in AR environments is operable under parabolic flightconditions using keyboards with haptic feedback (BA, PA)

and without haptic feedback (HA), and keyboards that areattached to humans body frame inside (HA, BA) andoutside (PA). Quantitative measurements of correct andfalse key hits revealed that using the BA keyboard achievedthe highest number of correct key hits under hyper- and

microgravity, and the lowest number of false key hits underhypergravity, but the highest number of false key hits undermicrogravity. Using BA keyboard under 1.8-gcompared toall other conditions (0-g, 1-g) differs significantly (p

Documents

Verifying Sensorimotoric Coordination of Augmented Reality Selection under Hyper- and Microgravity