Embed Size (px)
Citation preview
AFRL-HE-WP-TP-2006-0074
Improving Human Interfaces in MilitarySimulation Applications
Steve RoweJoshua Band
Charles CohenCybernet Systems Corporation
Michael HaasHuman Effectiveness Directorate
September 2006
Air Force Research LaboratoryHuman Effectiveness Directorate
Approved for public Warfighter Interface Divisionrelease; distribution isunlimited. Collaborative Interfaces Branch
Wright-Patterson AFB, OH 45433-7022
20070103051
Publi repoftinig buidmi for fta conledon of hifonatton Is estnialetd to averaeg 1 hour per reaponue. Inudln tol lime for rweviein haebuctina seaching date inoubcea.gatw*Vm rWngmid rbtafta fme data treaded, mid conipletb mid reveWing Mwoedlo of infonallon. Send coninienta regardinig ft burdwen eralba or any Over aspect of Oft Collectionofi kInfomielon. i sugigmiestions $or reduckig ft leaden ID Weahngbnx Headqurers~ Service, Directouate for Informalion Operations mid Reports,1215 Jeferson Do*i Higway, Wuho 1204, Aulinglon,. VA 220243=2 mid to the Offie of biengemet and Budget,Paperwoti Reduction Piplec (0704-0188) Wasiftgton. DC 20503.PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.1. REPORT DATE (DD-MM-YYYY) 2. REPORT TYPE3.DTSCVRD(rl-TOSeptember 2006 Conference ProceedingsI4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER
Improving Human Interfaces in Military Simulation Applications FA8650-05-M-567
5b. GRANT NUMBER
19. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) 5d. PROJECT NUMBERSteve Rowe *3005
Joshua BandCCharles Cohen *5.. TASK NUMBERMichael Haas CCHC
5f. WORK UNIT NUMBER5R
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) S. PERFORMING ORGANIZATION
*Cybemet Systems Corporation REPORT NUMBER
727 Airport BlvdAnn Arbor Ml 48108CC Human Effectiveness Directorate, WPAFB, OH
9. SPONSORINGIMONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORIMONITOR'S ACRONYM(S)Air Force Materiel Command CC AFRL/HECPAir Force Research Laboratory________________Human Effectiveness Directorate 11. SPONSORINGIMONITORINGWarfighter Interface Division AGENCY REPORT NUMBERCollaborative Interfaces Branch AFRL-HE-WP-TP-2006-0074Wright Patterson AFB OH-145433-7022
12. DISTRIBUTION AVAILABILITY STATEMENTApproved for public release. Distribution is unlimited. Cleared by AFRLIPA as AFRL/WS-06-0958 on 10 April 2006.
13. SUPPLEMENTARY NOTESThis will be published in the Proceedings of Spring 2006; Simulation Interoperability Workshop.
14. ABSTRACTThis paper highlights some of the key human factors issues that we found in OneSAF, and how we chose to remedy them in our ownsystem design. The goal is to provide guidelines for application human interface design standards so that future tools developed forthe Department of Defense allow the operator to perform more effectively.
15. SUBJECT TERMS
16. SECURITY CLASSIFICATION OF: 117. LIMITATION4 OF 18. NUMBER '19. NAME OF RESPONSIBLE PERSON
Unclassified ABSTRACT OF PAGES Michael Haasa. REPORT lb. ABSTRACT I C. THIS PAGE SAR 8 11gb. TELEPONE NUMBER (Includs area code)u U U A
STANDARD FORM 298 Back (Rev. 8/98)
Improving Human Interfaces in Military Simulation Applications
Steve RoweJoshua Band
Charles J. Cohen, Ph.D.Cybernet Systems Corporation
727 Airport Blvd.Ann Arbor, MI 48108
[email protected], [email protected]
Michael W. Haas, Ph.D.Air Force Research Laboratory
2255 H StreetWPAFB, OH 45433
Keywords:Human-Computer Interaction, User Interface, Simulation, OneSAF, GOMS
ABSTRACT: Military simulation applications such as OneSAF OTB provide a flexible environment for building andsimulating exercises. We recently undertook a human factors analysis of OneSAF to identify the key weaknesses in itsinterface, and identified several systematic problems. Among these were: excessive mouse movements, precise mousemovements, excessive visual search, finding objects in multiple data views at the same time, and repetitive, successivemenu selections. Other problems exist, but these general design flaws are important in that they highlight areas ofmajor performance loss and cognitive loading.
These design flaws can threaten situation awareness by improperly controlling the operator's attention, making itdifficult to find, interpret, and assimilate information and placing excessive cognitive demands on the operator. Carefuldesign and analysis of information management tools using a combination of perceptual, motor and cognitive modelingtechniques [1], usability engineering methods [21 and user studies can help us avoid these problems by optimizing theability for expert users to execute tasks quickly, easily locate information, avoid confusion and recover quickly fromerrors. This approach to design, grounded in user and task models, leads not only to interfaces that are moreaesthetically pleasing, but to tools that support better performance and are less cognitively demanding, freeing upintellectual and human resources for decision making and action.
Based on this analysis, we prototyped a number of web-based tools for improved human-system interaction that weredesigned to minimize these areas of performance loss. These tools were used to develop a system prototype, which weused to benchmark potential performance gains. We found these gains to be dramatic; In the case of changing aparameter of the map view, for instance, we were able to reduce the required mental operators (M in GOMS-KLM) byhalf and reduce mouse movement substantially as well.
This paper highlights some of the key human factors issues that we found in OneSAF, and how we chose to remedy themin our own system design. The goal is to provide guidelines for application human interface design standards so thatfuture tools developed for the Department of Defense allow the operator to perform more effectively.
1. Introduction recently undertook the creation of an interoperabilitytoolkit for the U.S. Air Force; we developed a tactical
The mandate from the Department of Defense is to enable simulation display as a demonstration application. Since
joint, network-centric military operations. As we in the this work was being done for the Human Engineering
information technology industry develop systems to directorate, our focus was not just on the underlying
support this vision, we must mitigate the problems that interoperability technology, but also on the design of the
arise from the over-availability of data. The authors Human-Computer Interface (HCI). In order to
demonstrate that our system was an improvement, we interface. CMN-GOMS (Card, Moran, Newell) is theneeded a basis of comparison. We chose OneSAF (One original version of GOMS, and does not defineSemi-Autonomous Forces), a popular constructive appropriate operators useful for our study. Finally, CPM-simulation application. We discovered many areas for GOMS emphasizes user learning and cognitive load.potential improvement in the design of the user interface Since we assume that all military users of OneSAF are, byof tactical display applications. This paper discusses our definition, expert users, learning rate is beyond the scopemethods, results, and recommendations. of this particular project. Cognitive load is certainly an
important factor, but such as study was beyond the scope2. Methods and Assumptions of this initial research effort.
The findings presented here are the result of GOMS 3. OneSAF OTB GUI(Goals, Operators, Methods, Selection rules) analysis [5].This method has its roots in cognitive psychology, and The OneSAF Objective Test Bed is a powerfulhas become extremely popular for characterizing human- application with many features. One of the drawbacks ofcomputer interaction. A GOMS model is composed of having so many features is that it is very difficult to createmethods that are used to achieve specific goals. The an efficient, uncluttered user interface.methods are then composed of operators at the lowestlevel. The operators are specific steps that a user performs The graphical user interface for OneSAF is shown inand are assigned a specific execution time. If a goal can Figure 1. The application was developed using the Xbe achieved by more than one method, then selection Window system with the Motif widget set. The work arearules are used to determine the proper Method. is divided into 4 sections: the top "menu and button bar"
area, the left panel "high level selection" area, the bottomWe assume that the users of military simulation tools are panel "detailed information" area, and the central "tacticalexperts, since GOMS analysis is only appropriate for map" area.expert-level users.
2.1 The Keystroke Level Model (KLM)
We use the KLM as the set of GOMS operators in ouranalysis. The KLM lends itself to the usual interaction ofuser with computer, as it defines the following operators:Keystroke, Button press, Point the mouse, Hand fromkeyboard to mouse or vice-versa, Mental preparation fortask (including visual search), Response time ofcomputer. For any method to be performed, that methodis broken into a sequence of these operators. Summingthe operator times yields the time needed to perform theoperation. The times that we use are given in Table 1.
Table 1 - Operator Times from Kieras[1]K Keystroke 280 msP Point mouse 1100 ms Figure 1: The OneSAF Graphical User Interface has 4B Press or release mouse button 100 ms active areas on screen at all times.
H Home hand to device 400 msM Mental preparation 1200 msR Computer Response (tool tip) 100 ms
2.2 Alternative GOMS methods
There are three other widely used variants of GOMS thatwe chose not to use in our work. NGOMSL (NaturalGOMS Language) is a specification of the goals into aregular language. This level of formalism was notappropriate for the non-natural language OneSAF
Figure 3: Middle-clicking on the unit icon (the veryFigure 2: The hot spot of the OneSAF cursor is not the tiny block circled on the map) in OneSAF displays acenter of the crosshairs. pop-up dialog of information about the unit Notice
The overall workflow of the application is modal; that is, the large visual distance between the icon and the
each of the buttons on the button bar selects a mode (such information box.
as creating new units, selecting units, modifying the map,etc.). Modal interfaces tend to decrease the operational 4.1 Determine Unit Position Informationcomplexity of tasks specifically in that mode. Thisbenefit is offset by an increase in the mental operation In this case study, we examine the goal of determining theduration because one must remember what mode the position (latitude and longitude) of a unit on the tacticalapplication is in, and the need to change the mode of the display. There are two methods to do this in OneSAF:application in order to perform a goal from a differentmode. 1) Select the "selection" mode with the mouse
(MHPBB). Middle-click on the unit (PB) toAnother quirk of the OneSAF GUI design is that the display the pop-up information window (M) asselection cursor is displayed as a crosshair reticule, but seen in Figure 3,the "hot spot" (selection point) of the cursor is actuallythe upper left corner of the (invisible!) square that 2) Select the unit in the high level selection areacircumscribes the crosshair (see Figure 2). This has the (MHPBB), shift focus to the information areaeffect of increasing the time for all mouse-pointing (M), and scroll the information pane so that theoperations performed by novices due to misplaced button location is visible (MPBPBMPBPBM).presses. However, we find that expert users have learnedto largely disregard the shape of the cursor when making Substituting times from Table 1 for each operation yieldsselections. that method 1 predicts a time of 5.3 seconds. Method 2
predicts 12.4 seconds. The majority of the additional time4 Case Studies in method 2 is taken by adjusting scroll bars in a sub-
window (see Figure 4). Although this display allows for aThis section describes our GOMS-KLM analysis of substantial amount of data to be displayed, that advantageseveral tasks that can be performed in OneSAF and also comes that the cost of a great deal of required visualin our prototype application, search coupled with mechanical scrollbar adjustment.
Figure 4. A close-up of the information detail panel ofOneSAF. Note the scroll box nested inside anotherscroll box.
Inftuenewr centric warfare applications, rapid
access to knowledge is key. How can we reduce the timerequired to access this fairly basic information?
12
First, often-used information should be obtainable using Figure 5 Our prototype uses tool-tips to displaythe "quick" method (method 1 in this example), while less frequently used data, thereby avoiding the need for
frequently needed information can be relegated to a more mouse clicks or a shift in user focus.obscure location. Focusing then, on method 1, we seek toimprove performance. Consider changing the applicationfrom modal to non-modal. This alone would remove theleading MHPBB (2.9 seconds) from the operation. 4.2 Unit Selection
Second, time can be reduced by changing the pop-up For the purposes of selecting a unit, both OneSAF and ourinformation box from one that requires a button press (0.1 prototype application are very similar. Both support theseconds) to a tool-tip (configurable response time). This direct selection of the unit's icon in the tactical displayalso has the added benefit of reducing cognitive with point-and-click (MPBB), as well as selection
workload, within an auxiliary window with point-and-click (alsoMHPBB). The differences in performance arise because
Finally, the pop-up box should appear near the cursor to of the required precision of the mouse pointing. As notedminimize visual search. This would haveme effect of previously, the selection cursor in OneSAF is a cross hairreducing the actual time for the mental refocus associated with a misplaced hot spot. This increases the time it takeswith the new pop-up, the user to position the mouse over the icon for selection.
Moreover, if the mouse is pressed when not precisely overIn our prototype application (see Figure 5), position the icon, the effect is that the map's zoom level isinformation is displayed by pointing the mouse cursor at changed. Changing the zoom level can move the icon ofthe unit on the tactical display (HP) resulting in a tool tip interest completely out of the user's field of view,(RM). Thus our system has a predicted time of 2.8 requiring remedial action to restore the workflow.
seconds. Using the static unit selection window is easier, but itrequires the user to know which unit in the windowcorresponds with the unit of interest in the tacticaldisplay. In OneSAF, this is accomplished solely throughunit designation. In order to find the unit in the unitselection window that corresponds to a given icon on thetactical map, the user identifies the unit name (M), and
then performs a visual search through the unit selectionwindow (M, and if scrolling is required, HPBPMB).
Although our results were based on a simulation developing massive multi-user networking and simulationapplication interface, it should be clear that future joint environment technologies. He acted as lead interfaceforces command and control applications will have a engineer on the prototype testing application mentioned insimilar set of functionality, and therefore a similar user this paper.interface. In order to enhance the effectiveness of ourcommanders using such software, we must create DR. CHARLES COHEN has been working in the fieldsstandards with the provision for engineering user of image processing, robotics, human-computerinterfaces to minimize user cognitive load and eliminate interaction, and artificial intelligence for over a decade.wasted physical motion. At Cybernet, he has been the project manager for many
projects for the United States Armed Forces (Air Force,7. References Navy, and Army), National Aeronautics and Space
Administration, and other government agencies. His[1] John, B. E., & Kieras, D. E. (1996). "Using GOMS projects include work on simulation, training, real-time
for user interface design and evaluation: Which optical pose determination, robotics, virtual reality, objecttechnique?," ACM Transactions on Computer- identification, feature and body tracking, and humanHuman Interaction, 3, 287-319. performance evaluation. Dr. Cohen's main areas of
interest are gesture recognition and massive number of[2] Nielsen, J. & Molich, R. (1990). Heuristic Evaluation agent network architectures and systems, which he is
of User Interfaces. Computer Human Interface (CHI) developing and integrating for the Army and Air Force.90 Proceedings, 249-255.
[3] Wickens, C., & Hollands, J.: Engineering Psychologyand Human Performance, Prentice Hall, New Jersey2000.
[4] Shneiderman, Ben: Designing the User Interface,Addison Wesley, Reading, Massachusetts 1998.
[5] Card, S., & Moran, T, & Newell, A.: The Psychologyof Human-Computer Interaction,- Lawrence ErlbaumAssociates, Inc., New Jersey 1983.
[6] Helander, M., & Landauer, T., & Prabhu, P.:Handbook of Human-Computer Interaction, NorthHolland, Amsterdam 1997.
[7] Brinck, T., & Gergle, D., & Wood, S.: DesigningWeb Sites that Work - Usability for the Web,Morgan Kaufmann Publishers, San Francisco 2002.
[8] Ware, Colin: Information Visualization, MorganKaufmann Publishers, San Francisco 2004.
[9] Beyer, H., & Holtzblatt, K.: Contextual Design,Morgan Kaufmann Publishers, San Francisco 1998.
[10] John, B., & Prevas, K., & Salvucci, D., & Koedinger,K.: "Predictive Human Performance Modeling MadeEasy," CHI 2004, April 24-29, 2004, Vienna,Austria.
Author Biographies
STEVE ROWE is a senior software engineer at CybernetSystems Corporation. Besides being well versed insimulation environment, intelligent agents, and scenariodevelopment work, he has taught in the classroom andalso worked in the development of automated teachingmaterials at KnowledgeNet Corporation.
JOSHUA BAND is a research engineer at CybernetSystems Corporation. He has significant experience
Figure 6: Highlighting a unit's icon in our prototypeapplication highlights the corresponding entry on theright in bright yellow.
In our system, a similar visual search is required. Figure 7: Use of tabbed pane to localize the changes ofHowever, we reduce the time required for the visual a modal interface.search by reducing the level of cognition required. We dothis by simultaneously highlighting a selected icon withthe corresponding display in the unit selection window(see Figure 6). This reduces the time it takes the user to The time it takes for a user to find the next interfacerecognize the unit of interest, because it is visually set element with which to interact is also part of the mentalapart from the other units in a very dramatic fashion. operation equation. If an element is always displayed in
Another way to augment this operation would be to the same place, an experienced user will very quickly
automatically scroll the unit selection window such that move their eyes to that place; in this case, there really is
the selected unit is visible, no "search". Similarly, if an element appears in a placethat is very near the users current focus, then the eyes donot need to move, and again no search needs to take
5. Decreasing the Mental Operation Times place.
In this section, we depart from formal GOMS analysis OneSAF's interface requires the user to frequently moveand discuss some general improvements that can be made between the four control areas of the interface. Each suchto the OneSAF user interface based on qualitative shift requires a visual search (sometimes accompanied byobservation. the use of scroll bars) in order to continue the operation.
5.1 Modality In our application, we chose to use context-sensitive pop-up menus (that appear next to the mouse icon) for most of
As noted in section 3, OneSAF uses a modal operation our control interfaces. This has the effect of keeping theparadigm. Without debating the wisdom of the choice of controls where the user's attention is, rather than forcinga modal interface, we believe that the efficiency can be them to go find the controls.improved by using the tabbed pane user interface widget.This has the effect of changing the current look of the 6. Conclusioninterface sufficiently to constantly remind the user whichmode they are in, thus reducing cognitive workload. We have provided some case studies of the performance
We demonstrate a tabbed interface (Figure 7) for different improvements possible with OneSAF. We have
modes of operation in our prototype application. The tabs introduced the GOMS-KLM method for analyzing userinterface performance trade-offs. We have shown thatprovide access to controls for reviewing operational improvements can be made by both reducing the number
orders, playing back simulation data, and scanning of operations required for a method, and by reducing the
factor that an operation (e.g. Mental) actually costs.
