Framework for Selecting Appropriate Interface Technologies

Framework for Selecting Appropriate Interface Technologies for the Future Ops Room

Brad Cain CAE Inc.

Chris Nicholson CogSim Technologies Inc.

Prepared By:

CAE Inc. 1135 Innovation Drive Ottawa, Ontario, K2K 3G7

Contractor's Document Number: 5878-001 Version 02

PWGSC Contract Number: W7707-145734/001/HAL

Contract Technical Authority: Tania Randall, Defence Scientist.

Contract Report DRDC-RDDC-2016-C066 February 2016

This S&T document is provided for convenience of reference only. Her Majesty the Queen in right of Canada, as represented by the Minister of National Defence ("Canada"), makes no representations or warranties, express or implied, of any kind whatsoever, and assumes no liability for the accuracy, reliability, completeness, currency or usefulness of any information, product, process or material included in this document. Nothing in this document should be interpreted as an endorsement for the specific use of any tool, technique or process examined in it. Any reliance on, or use of, any information, product, process or material included in this document is at the sole risk of the person so using it or relying on it. Canada does not assume any liability in respect of any damages or losses arising out of or in connection with the use of, or reliance on, any information, product, process or material included in this document.

© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence, 2016

© Sa Majesté la Reine (en droit du Canada), telle que représentée par le ministre de la Défense nationale, 2016

CAE Inc. 1135 Innovation Drive

Ottawa, Ont., K2K 3G7 Canada Tel: 613-247-0342 Fax: 613-271-0963

FRAMEWORK FOR SELECTING APPROPRIATE INTERFACE TECHNOLOGIES FOR THE FUTURE OPS ROOM

MARITIME INFORMATION WARFARE (MIW) TASK 6

Brad Cain, CAE Canada, Defence and Security

Chris Nicholson, CogSim Technologies Inc.

CONTRACT NO. W7707-145734/001/HAL W7707-145734

Order No. W7707-4501260384

FOR

DEFENCE RESEARCH AND DEVELOPMENT CANADA (DRDC)

9 Grove St Dartmouth, NS

Ms. Tania Randall

+1 (902) 426-3100 x283

Contract Report 1 0 F e b r u a r y 2 0 1 6

CAE Document No. 5878-001-02

Contract Report DRDC-RDDC-2016-C066 February 2016

This S&T document is This S&T document is provided for convenience of reference only. Her Majesty the Queen in right of Canada, as represented by the Minister of National Defence ("Canada"), makes no representations or warranties, express or implied, of any kind whatsoever, and assumes no liability for the accuracy, reliability, completeness, currency or usefulness of any information, product, process or material included in this document. Nothing in this document should be interpreted as an endorsement for the specific use of any tool, technique or process examined in it. Any reliance on, or use of, any information, product, process or material included in this document is at the sole risk of the person so using it or relying on it. Canada does not assume any liability in respect of any damages or losses arising out of or in connection with the use of, or reliance on, any information, product, process or material included in this document.

© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence, 2016 © Sa Majesté la Reine (en droit du Canada), telle que représentée par le ministre de la Défense, 2016

Framework for Selecting Appropriate

Interface Technologies for the Future Ops Room Maritime Information Warfare (MIW) Task 6

A P P R O V A L S H E E T

Document No. 5878-001 Version 02

Document Name: Framework for Selecting Appropriate Interface Technologies for the Future Ops Room Maritime Information Warfare (MIW) Task 6

Primary Author

16 Feb 2016 Name Brad Cain Date

Position Senior Consultant Secondary Author

17 Feb 2016 Name Chris Nicholson Date

Position Human Factors Specialist

Reviewer

Name Shelley Kelsey Date

Position Human Factors Group Lead

Approval

Name Tab Lamoureux Date

Position Account Manager

5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001

5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001 5878-001

10 February 2016 – ii – 5878-001 Version 02

© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence, 2016 © Sa Majesté la Reine (en droit du Canada), telle que représentée par le ministre de la

Défense nationale, 2016



A B S T R A C T

Defence Research and Development Canada (DRDC) is conducting research through the Maritime Information Warfare (MIW) program to provide guidance on the selection of technology and techniques for managing Maritime information for the Royal Canadian Navy (RCN). The rate of development of commercial information technology is out-pacing the ability of users to determine what is useful or the limits of how emerging technology can be applied suitably. The current document describes work done to develop a technology assessment framework that may assist technology advisors in evaluating the suitability of information display technologies for specific naval tasks.

The framework was prepared in Microsoft Excel, using task description categories typical of Human Factors task analysis approaches and identifying technology characteristics relevant to the task description categories. The framework currently comprises three worksheets:

• Worksheet 1: describes tasks in a standardized manner, defines the requirements and lists the constraints of the user performing the task

• Worksheet 2: defines the rules that were derived from the scientific literature and industry best practice documents, relating the task requirements and constraints to technology characteristics

• Worksheet 3: identifies and describes the technological characteristics of existing (and emerging) information display device

The framework is not yet complete and includes two additional worksheets: one as a placeholder for the results of a future analysis calculus that assesses technologies and another as a working example that illustrates the early development of such an analysis calculus. Guidance on relating task constraints and technology characteristics in the framework are proposed based on published scientific and applied Human Factors usability research. The current version of the technology assessment framework should form a solid base from which to continue development through iterative, spiral developments.

10 February 2016 – iii – 5878-001 Version 02





R E V I S I O N H I S T O R Y

Revision Reason for Change Origin Date Version 01 Initial DRAFT release for comment 22 January 2016 Version 02 Revised report addressing client comments 10 February 2016

10 February 2016 – iv – 5878-001 Version 02





T A B L E O F C O N T E N T S

1 INTRODUCTION ..................................................................................................... 1 1.1 Background .......................................................................................................... 1 1.2 Objective .............................................................................................................. 1 1.3 Scope ................................................................................................................... 1

2 METHOD ................................................................................................................. 3

3 RESULTS ............................................................................................................... 6 3.1 Background Investigation ..................................................................................... 6 3.1.1 Lessons Learned from the Nuclear Industry ...................................................... 6 3.1.2 Human Factors in Naval Tasks .......................................................................... 8 3.2 Framework Development ..................................................................................... 9 3.2.1 Task Description .............................................................................................. 10 3.2.2 Technology Characteristics .............................................................................. 12 3.2.3 Rule Development ........................................................................................... 16 3.2.4 Example Calculus ............................................................................................ 17 3.3 Instructions for Using the Framework ................................................................. 19 3.3.1 Users of the Framework ................................................................................... 20 3.3.2 Framework Administrators ............................................................................... 20 3.4 Limitations of the Framework.............................................................................. 21

4 RECOMMENDATIONS ......................................................................................... 22

5 CONCLUSION ...................................................................................................... 23

6 REFERENCES ...................................................................................................... 24

7 ABBREVIATIONS ................................................................................................ 28

APPENDIX A ANALYSIS OF A SAMPLE OF NAVY COMMAND STAFF TASKS A-1

APPENDIX B TASK DESCRIPTION FIELDS ........................................................ B-1

APPENDIX C TECHNOLOGY CHARACTERISTIC FIELDS .................................. C-1

APPENDIX D TECHNOLOGIES CURRENTLY REPRESENTED IN THE FRAMEWORK TECHNOLOGY DATABASE .................................. D-1

D.1 Wearable ....................................................................................................... D-1 D.2 Totable ........................................................................................................... D-2 D.3 Fixed .............................................................................................................. D-2 D.4 Input devices .................................................................................................. D-4 D.5 Miscellaneous ................................................................................................ D-5

10 February 2016 – v – 5878-001 Version 02





APPENDIX E RULE DESCRIPTIONS .....................................................................E-1 E.1 Communication and Collaboration ..................................................................E-1 E.2 Information and Data Description ...................................................................E-2 E.3 Actions and Information Manipulation .............................................................E-4 E.4 Decision Making and SA .................................................................................E-6 E.5 Task Environment and Operator Clothing .......................................................E-7

10 February 2016 – vi – 5878-001 Version 02





L I S T O F F I G U R E S

Figure 3-1: Screen shot of part of the Task Description sheet of the framework showing the hierarchical structure, row elements and descriptive comments (in yellow) as well as a sample-task data set ............................................................................... 11

Figure 3-2: Screen shot of part of the Technology Characteristics tab of the framework, showing the organization and descriptive comments (in yellow) for a selection of technologies ........................................................................................................... 13

Figure 3-3: Screen shot of part of the Rules tab of the framework showing the structure, row information that corresponds to rules and descriptive comments (in yellow) ... 16

Figure 3-4: Example of Satisfied (Green) and Unsatisfied Rules .................................. 18 Figure 3-5: Example of the Colour Gradient Technology Suitability Assessment Scheme

............................................................................................................................... 19

10 February 2016 – vii – 5878-001 Version 02





L I S T O F T A B L E S

Table 3-1: Preliminary Set of Human Capabilities used to Structure Task Description Data ......................................................................................................................... 8

Table 3-2: Technologies and high level organization based on portability suggested by DRDC for consideration ......................................................................................... 14

10 February 2016 – viii – 5878-001 Version 02





1 INTRODUCTION

This document describes the work done in response to the Defence Research and Development Canada (DRDC) Atlantic Statement Of Work (SOW) entitled “Task 006, Maritime Information Warfare (MIW): Framework for Selecting Appropriate Interface Technologies for the Future Ops Room” (Standing Offer Contract W7707-145734, Call-up 6.) The work includes a companion document in Microsoft Excel that presents a draft implementation of the work described in the current document.

1.1 Background

The SOW provides a brief description of the objectives of DRDC’s MIW program. Part of the MIW program is concerned with exploring options for enhancing command team effectiveness through the introduction of emerging technologies and techniques that support information management and promote improved situation awareness activities within command teams.

Commercial information technology is developing rapidly, introducing new hardware, software and user-interface techniques that outpace the scientific community’s ability to study their effectiveness. Thus, many emerging technologies purport or have the potential to do tasks easier, faster or cheaper than ever before. Unfortunately, many of the expectations do not have reliable evidence to support decisions for adopting emerging technology; this creates a risk for both acquisition and operation in the naval domain.

In many cases, however, the technology has a common constraint that does not change in a fundamental way very quickly: the technology should support the operator. While operator knowledge may evolve quickly with experience, the underlying cognitive and physical capabilities remain constant over the life cycle of the technology. Knowledge of these effectively immutable operator capabilities provides a basis for assessing technologies under generalized work domains for their suitability at supporting the operator during the performance of a specified task.

1.2 Objective

The objective of the work was to create a tool to help analysts assess user interface (UI) technologies by comparing task descriptions, human capabilities, environmental constraints and technology characteristics. The tool is intended to guide the analyst toward appropriate UI technology solutions that will support the operators performing the task.

1.3 Scope

The statement of work focussed on naval command teams and while this focus was respected in the work performed, the results should be more broadly applicable as the task descriptions developed were largely general, although some task features involve Navy-specific constraints (e.g. SHINCOM compatibility).

10 February 2016 – 1 – 5878-001 Version 02





The statement of work identified five specific tasks with sub-tasks to be undertaken:

1. Develop a framework for determining appropriate interface technologies for individual and team-based tasks of a future naval ops room.

a. Identify the benefits and limitations of each technology for specific tasks or goals.

b. Identify or develop a set or categorization of naval ops room tasks or goals for which the value of various interface types can be assessed.

c. Determine the interface characteristics and operating constraints that are relevant to interface selection for identified tasks or goals.

d. Determine the set of interfaces technologies that should be included in the framework.

e. Develop a framework concept for rating the suitability of each technology for specified tasks.

2. Populate the framework with data.

3. Apply the framework to select interface technology for a Course of Action (COA) test bed.

4. Identify framework limitations.

5. Write a contractor report.






2 METHOD

The following approach to the call-up work was presented to DRDC and accepted as a suitable method to meet the objectives.

The first step was to hold a kick-off teleconference amongst the immediate stakeholders to ensure that CAE understood DRDC’s intent and that DRDC understood the risks associated with the tasking.

The proposed approach to develop a UI requirements assessment framework was based on common Human Factors task description and analysis approaches (Kirwan & Ainsworth, 1992), organized around identifying generic characteristics in the following categories:

1. task/goal

2. operator

3. team

4. environment

5. technology

CAE proposed that an analysis calculus would be developed and each task requirement would be weighted according to the characteristics of the particular task or goal under consideration. The rules and weights used in the calculus would be determined by reviewing the Human Factors literature for guidance on usability and best practice recommendations. Each weighting would have an associated justification or citation that would permit later review of the rationale for the weighting. A scoring scheme would be developed within the framework to produce a rank ordering of technology suitability. These calculations would normally be hidden from the user, but can easily be accessed for editing by an administrator allowing the framework to evolve.

To address the need to populate the UI requirements assessment framework with data for each task or goal, the key characteristics of the interface technologies would be ranked, rated or scored so that a summary ranking statistic could be produced. It was proposed to demonstrate the framework by applying it to the COA test bed.

A description of the work performed, the resulting framework and its limitations would be provided in a report (this document) while the framework itself would be provided in an Excel workbook.






The process of assessing technologies involves developing a suitable task description and generalization of experimental studies of technology effectiveness involving characteristics associated with those task descriptions (Fleishman, 1967). Reports that document Navy command team activities (Lamoureux & Banbury, 2011; Matthews, Webb, & Bryant, 1999a, 1999b; Matthews, Rehak, & McFadden, 2007) were used to select a representative set of specific task descriptions of the command team composition, naval operations room tasks and equipment involved for developing and maintaining situation awareness of the Recognized Maritime Picture (RMP). This task set is provided in Appendix A along with more abstract task descriptions that were generated to form the basis of the task descriptions following approaches from applied task analysis methods (Kirwan & Ainsworth, 1992; MA&D, 2015).

The current work deviated from Fleishman’s recommended process in the search phase for suitable experimental studies of the included technologies. The development of user interface technologies moves much more rapidly today than in 1967 and the experimental literature cannot keep up with the growth – there is a lack of research on the effects of new technologies in industry (Burks, Harper, & Bartha, 2014). Much of what is known about the emerging technologies of the past decade is contained in individual study reports rather than collated in summary guidance documents.

However, the process followed in the framework is progressive in the sense that it first considers whether a device or technology has a certain class of capabilities (e.g. input) to determine whether it is minimally compatible with the task. This is followed by an elaboration of that class to determine additional details that would benefit from usability knowledge to rank the compatible options, if the usability knowledge is available.

It should be noted that the goal of the current work was to develop a framework to assess the applicability of a wide range of current and future technologies for broad use in a future naval operations room. The key distinction here is a broad range of tasks rather than specific tasks to analyze. The proposed framework was meant to be general enough to assess any technology across all possible activities conducted in the operations room, as well as any possible future workflows. However, the framework was still expected to assess specific tasks and workflows as well. To accomplish this, tasks needed to be broken down in to their fundamental components (for example, view information, pressing a button, listening to audio, etc.). In this way, all tasks, regardless of their differences, have overlapping components. By deconstructing a selection of naval operation room tasks (Lamoureux & Banbury, 2011; Matthews, Webb & Bryant, 1999a, 1999b; Matthews, Rehak & McFadden, 2007) as well as some nuclear power station operation room tasks (Kisner, 1992), it was possible to derive a database of general task descriptions. The logic was that by using the correct combination of these 100 plus general task descriptions, any possible operations room task or workflow can be represented in the framework.

Development of the framework was loosely based on the example provided in the SOW. The framework was elaborated and reorganized based on discussions with DRDC, information obtained from technology standards (e.g. DoD, 2012) and some usability evidence found during






a short Internet search for publicly available journal articles (there was insufficient time for an extensive usability literature review).






3 RESULTS

3.1 Background Investigation

3.1.1 Lessons Learned from the Nuclear Industry

The application of a technology selection framework to a naval environment is extremely specific which makes a review of similar prior work quite difficult. However, similar technology assessment approaches have been attempted for nuclear power plant control rooms (Hugo & Gertman, 2015; Hugo, Gertman & Tawfik, 2013; Kisner, 1991). Nuclear power plant control rooms have some fundamental similarities with naval operation rooms including:

• operators interacting with complex systems through a variety of user interface technologies

• teamwork amongst operators

• time critical operations

• the need for avoidance of operator error

Hugo and Gertman (2015) report that “there is currently no generally accepted guidance for human systems interaction technology selection” and although these authors were referring to the nuclear power generation industry, the comment is indicative of the state of the art. Typically, the method for selecting technologies is to start with a need (e.g. display system data for operator) and find a technology that satisfies the need. In the past, the number of different technologies was rather limited, although the number of devices within a class of technology could be large, with differences in device capabilities occurring both within and across vendors. Today, the number of different classes of technologies is growing more rapidly and the number of devices within a class is changing frequently, driven by market forces and, seemingly, a quest to incorporate more features into ever smaller packages.

The first benefit of developing a framework is that it not only satisfies one need, it also develops a system to evaluate a number of different tasks with many combinations of needs. Another consideration is the speed with which technology is currently progressing. Not long ago if someone wanted a means for presenting system information to an operator, that person had one or two technologies capable of doing so. Now, however, there are numerous technologies to select from and many devices within each technology category being developed regularly. This highlights the second benefit of developing a selection framework: it identifies the relevant basic technology characteristics that influence the human system interaction regardless of the actual technology. To accomplish this, Hugo and Gertman (2015) developed a list of technologies, categorized that list for various operator domains and summarized the list of technologies that comply with identified usability criteria.






Hugo and Gertman (2015) suggest that designers should have a strong understanding of the actual conditions under which a given technology is likely to be used in the expected environments. They recommended extensive use of simulation, test beds and prototypes be used to validate the use of any technology under consideration. Finally, it is important to understand future trends and the potential development of technologies over the next 10 to15 years and the implications for modular design or maintenance of a technological component in the solution.

Hugo and Gertman identify six criteria that technology selection should accomplish, although these are inter-independent:

1. reduce task complexity;

2. reduce operator error to improve system reliability;

3. improve systems usability;

4. reduce operator workload;

5. support variability among operator; and

6. improve operator situation awareness.

In developing a framework for selecting suitable control technologies for nuclear power plants, Kisner (1991) describes the following requirements to consider:

• Performance requirements

• Reliability requirements

• Maintenance requirements

• Operational environment

• Development environment

• Equipment limitations

• Need for future upgrades






To summarize the works of Hugo, Gertman and Kisner, the most useful framework for selecting and assessing new technologies is one that is generic enough to enable comparison of many interface technologies. The framework should be flexible enough to deal with a level of future assumptions, but specific enough to be usefully applied when information about current technologies is available. A selection framework should not be expected to produce final decisions about technologies’ implementation; rather, it should be used as guidance for narrowing the options and highlighting strong points or shortcomings of a given technology.

The current framework only considered some of Kisner’s factors as there was no data available to perform an assessment. However, the framework should be readily amenable to extension to include factors such as reliability or maintenance when those data are available.

3.1.2 Human Factors in Naval Tasks

The domain focus of the present work is Maritime defence and security, predominantly naval. However, while naval tasks are domain dependant, the basic actions performed in naval tasks are often quite general. So while the context is an important factor to consider to identify procedures and constraints (Hugo & Gertman, 2015), usability is a Human-System Integration (HSI) factor that is driven more by the operator’s physical or mental capabilities and limitations.

Initially, Task Descriptions were structured around traditional Human Factors task analysis methods (Kirwan & Ainsworth, 1992; MA&D, 2015) that were anticipated to lead to a method for predicting the suitability of each technology to Navy command staff tasks.

Table 3-1: Preliminary Set of Human Capabilities used to Structure Task Description Data

Human Capability Modality Detail Categories

Perception Auditory • simple/complex signals

• simple/complex verbal message

• simple/complex numeric message

Visual • text message/information

• spatial/graphical display

• monochrome and colour

• contrast

• brightness

• viewing angle

Tactile • alert vibrations

• touch screen feedback

Cognition Text/verbal • simple/complex message






Human Capability Modality Detail Categories

Spatial/graphical

• simple/complex

• 2D/3D graphics

Workload • processing speed

• working memory capacity

Attention • Alert salience

Behaviour

Motor • fine/coarse continuous adjustment/input

• fine/coarse discrete adjustment/input

• gross motor

Verbal

• text/speech

• simple/complex

Movement • single station task/requires movement to other locations

Communication mode

• Direct speech

• Remote/Local

• Gesture

• Face-Face

• SHINCOM

Environment Noise

Lighting

Vibration

PPE

This list was subsequently refined using task descriptions of the Halifax Class Operations Room command staff (Lamoureux & Banbury, 2011) that were analyzed and coded (Appendix A), but this produced descriptions that were too abstract. The task descriptors were aggregated into higher level descriptions that used more common expressions instead of Human Factors/Psychology jargon to be more broadly interpretable. The final list of task descriptors that was implemented in the framework is reproduced in Appendix B.

3.2 Framework Development

The preliminary framework is contained within one Microsoft Excel workbook. The workbook comprises of three principal worksheets: Task Descriptions, Rules and Technology






Characteristics. Each of the framework elements is described in following sections. A fourth worksheet in the workbook, Recommendations, is a placeholder for a summary present of the results of an assessment; however, it has not yet been completed. In its place, a fifth, temporary worksheet has been included in the framework to illustrate how the logic of an assessment may occur when the results worksheet is completed.

The breadth and depth of the worksheets makes presentation in a report awkward. Screenshots of the various worksheets are provided in the corresponding sections to provide a visual key for the reader, but it is recommended that the reader refer to the framework workbook directly to explore the details described below. Comments are embedded in the Excel workbook to provide additional information about the purpose and organization of the framework structure.

3.2.1 Task Description

Task descriptions are only useful in the context of this framework if they can be linked to objective technology characteristics by rules from the scientific literature or empirically derived best practices. For example, an important part of operations room activities is “integration of information from multiple sensor readouts” (Lamoureux & Banbury, 2011). Alone, this task description would not provide any useful guidance for selection a specific technology – it is far too subjective. The only way to make this useful in the framework is to break it down into general task descriptions that have relevance to objective technology characteristics. For example, “integration of information” might require any of a number of attributes such as:

• display of data to the user

• user may effectively read presented data at 60 cm

• the user must be able to store or retrieve data

Using empirical data from technically sound usability studies has implications for the types of task descriptions that are suitable for this framework so that it focusses on evidence-based decision rules whenever feasible. This approach is important for avoiding biases and prejudices that can sway decisions about technology suitability.

The Task Description sheet (partially shown in Figure 3-1) is the main sheet that users of the framework will interact with when characterizing a specific task or workflow. The complete list of task description fields is presented in Appendix B Technology characteristics.

The task descriptions are intended to be sufficiently general that they can be combined to represent not only the tasks they were originally derived from, but also the anticipated range of naval command and control tasks.






Figure 3-1: Screen shot of part of the Task Description sheet of the framework showing the hierarchical structure, row elements and descriptive comments (in yellow) as well as

a sample-task data set

There is a hierarchal structure to the Task Descriptions, arranged in rows. The descriptors are organized under five major task attribute headings that are highlighted in the worksheet with a pale blue cell shading. The five task attribute headings are as follows:

1. Communication and Collaboration

2. Information and Data Description

3. Action and Information Manipulation

4. Decision Making and Situational Awareness

5. Task Environment and Operator Clothing

Further subcategories or individual descriptors of elements of the task are nested within these major headings starting in Column A and continuing in Columns B and C as needed for the decomposition. The level of detail can be hidden or exposed with the + and – buttons along the left-most side of the screen as required. Each descriptor has been given a temporary identification number in this hierarchy although the number is not currently used.






An example of each descriptor is provided in Column F. For example, on line 18 under “Information and Data Description”, task description 8.1 “Task involves audio information/hearing” would be set to NO if the task does not use audio. The six sub-descriptions below this descriptor are not applicable, so additional rows may be hidden to declutter the display. If description 8.1 is set to YES, then the task does use audio, and the user can expand the hierarchy (if it is collapsed) to enter additional data such as 8.1.1 “Task involves aural alerts,” etc.

Each task to be considered is described in its own column, currently shown in Columns G, H and I for three hypothetical tasks. Arbitrary selections for Task 1 have been made in Column G to demonstrate some of the envisioned framework capabilities. The default entry for each task description in each of these columns is an empty cell, which simply indicates that it has not been assessed. Each descriptor has a pull down list of valid values for the descriptor. The task descriptions are currently limited either binary (e.g., yes/no) or categorical (e.g., low/medium/high). There is currently no numerical weighting of the importance of task descriptions, although this could be accomplished as a logical extension of the categorical levels.

Column J provides a simple explanation of the rule associated with the descriptor. The rule example is kept short for convenience, but a complete version should provide detailed explanations of each of the descriptor value options. If these example rules clutter the display too much or are found to be seldom used, they could be inserted as comments associated with the descriptors.

The task descriptions were developed assuming that the user would have some concept of how the task would be performed, but an effort was made to avoid suggesting technological solutions or reference to a specific technology in the task descriptions. Nevertheless, some features will reflect a prescribed modality of use. The user may adjust the assessment of the task requirement to explore “what if” comparisons of using different modalities. For example, rather than reference technology characteristics directly, such as display size; e.g., Does the task require a small, medium, or large display?, the task descriptions were designed to focus on, in this case, the information required in the task, e.g., task requires minimal, moderate, or large amounts of graphical information. This method allowed for segregation of the task descriptions and technology solutions. In some other situations, specific compatibility with Navy hardware may be important (e.g., SHINCOM), so some provision was made to accommodate constraints based on existing technology capabilities. The user has the ability to relax or impose these constraints as desired.

3.2.2 Technology Characteristics

The Technology Characteristics sheet is used as a database of objective technical data about each technology or device that is to be considered when analyzing a task. The width and depth of this sheet make presentation in a report awkward so it is recommended that the Excel workbook be used to explore the details of the Technology Characteristics worksheet and use the image shown in Figure 3-2 as a visual key to the Excel workbook.






The technology database is organized under four categories (across Row 1 starting at Column G) according to a somewhat arbitrary portability classification that is used in the marketing literature:

1. Portable

2. Totable

3. Fixed

4. Miscellaneous

Each of these four major categories is subdivided into several technology classes that were identified in consultation with DRDC. Currently, there are 26 technology classes included in Row 2. Each of these technology classes has at least one type of device as an example (currently there is a total of 38 devices included in Row 3). A list of technologies and devices currently included in the database is also provided in Appendix D along with a brief description. This database is expected to grow as new technologies emerge and devices are improved. The lowest level may require regular editing to keep pace with the evolving technological products and their features.

Figure 3-2: Screen shot of part of the Technology Characteristics tab of the framework, showing the organization and descriptive comments (in yellow) for a selection of

technologies

Discussions with DRDC identified a number of traditional and emerging technologies that should be considered in the framework for future naval operations room command team. These were classified according to portability, as shown in Table 3-2, although this organization also reflects the trend for emerging technologies to be smaller.






Table 3-2: Technologies and high level organization based on portability suggested by DRDC for consideration

Wearable Totable Fixed Miscellaneous

HMD (e.g., AR and VR displays, google glass) Smartwatch Wristband Smart clothing Smart earphones

Smartphones Tablets Laptops E-readers

2D Display 3D Display Multi-view Display Transparent Display Kiosk Speakers Large Wall Display

3D Control Device Bio Monitors Eye tracking Physical Morphing Displays Flexible Displays Holographs E-ink Display Implanted IRFID Tag Retinal Projection

The current version of the framework includes descriptions of tangible technologies only. Initial discussions with DRDC identified devices with functionality that was associated more with user interface software design (asymmetrical interaction, voice command or gestures) rather than the underlying, supporting technologies. These technologies were excluded from the application of the framework because these are often features enabled by some higher-level technology (i.e. gestures on a smart phone). Appendix D provides a description of the technologies that were considered for inclusion in this version of the framework and the technologies that were suggested but not included are denoted with an asterisk (*) in Appendix D along with the reason for their exclusion.

Devices that only provide manual input capability (e.g., mouse, trackball, keyboard, etc.) were also omitted from the technology database. The majority of the technologies under consideration do not require additional input devices as they have an integrated input capability. Technologies that do require external input devices generally include some form of input device and the usability of these traditional input devices has been documented in a number of studies of computer hardware. Such devices are also excluded from the framework database and denoted in Appendix D with an asterisk (*).

The framework database contains a diverse selection of user interface technologies. To allow all of these technologies to be analyzed across all of the task descriptions, a broad list of Technologies Characteristics had to be compiled. These technologies characteristics are an amalgamation of technology specifications, features and capabilities. Because the framework is meant to be very general, not all technology characteristics will be relevant for every included technology. However, some technology characteristics are relevant for every device (e.g., the device’s size).






The first four technology characteristics under the Physical Specification heading (labeled 1 – 4 in the framework) are used to identify which capabilities a device incorporates and includes:

1. display technologies;

2. input technologies;

3. communication technologies; and

4. performance monitoring technologies.

Individual devices have characteristics that have been organized into at least one of these characteristics, sometimes all four. This is the first step in identifying which technology may have the required capabilities for any given task description.

Specific characteristics of technologies are listed by row in Column A, organized into five broad categories:

1. Physical specifications

2. Display

3. Audio

4. Input

5. Connectivity

These categories were developed after reviewing scientific literature (Hinckly & Wigdor, 2012; Hugo & Gertman, 2015; Salvendy, 2012; Spencer & Johnston, 2003) as well as conducting a survey of the current technology market advertising. Using specific technologies from a wide range of technology types as example cases, individual characteristics relevant to each technology type were condensed. The final list of technology characteristics attempts to capture all objective characteristics (e.g., display dimensions) a given technology could possess while avoiding subjective characteristics (e.g., technology is best used when).

Each technology characteristic required a metric to represent whether or not a given technology possessed that characteristic and occasionally, how it compares to other technologies. These metrics may be binary yes/no (e.g. technology has or does not have a characteristic), categorical (e.g., technology has a display, such as LCD and LED) or continuous/quantitative (e.g., technology weighs 300g). The full list of technology characteristics is presented in Appendix C. Most of the attributes are associated with a rule that relates them to some aspect of the task; however, there are some attributes that were included that do not have an explicit rule association but are expected to be referenced as the rules are developed further.






When a new technology is added to the framework database, data entries for relevant Technology Characteristics in Column A will be required. The attribute values for each Technology Characteristic are listed under the heading Metric. Some of these characteristics require a simple yes/no value to indicate whether or not the technology possesses a given characteristic while others are quantitative values such as screen size, which is an absolute value commonly reported in inches. Some of the technology characteristics included in the framework do not, as yet, have rules that relate the quantitative values to explicit, evidence-based guidance on usability. It is expected that relevant usability information exists in the scientific literature; and so these characteristics are included in anticipation that the usability rules will be added in a future version of the framework.

3.2.3 Rule Development

The Rules worksheet is intended to document scientific and best practice guidance to align the task description with the technology characteristics based on Human Factors knowledge as with the other worksheets, the bread and depth make presentation in a report awkward and viewing details is easier in the Excel workbook. A screenshot of a portion the worksheet is shown in Figure 3-3 as a visual key to the framework workbook.

Figure 3-3: Screen shot of part of the Rules tab of the framework showing the structure, row information that corresponds to rules and descriptive comments (in yellow)

The Rules sheet lists each of the descriptions from the Task Descriptions sheet in the left most columns (Columns A through D.) In Column F, a set of rules is outlined for assessing each Task Description and the corresponding Technology Characteristics. In Column G, a summary of Technology Characteristics relevant for a given Task Description are noted.






Rule development was the last step in finalizing the selection framework. Each of the Task Description factors was analyzed to identify the corresponding technology characteristics relevant for each task description. Information from the scientific literature or Military Standards (e.g., MIL-STD- 1472G) was used whenever feasible and the associated references cited with the rule. Details about the rules created are shown in Appendix E. The rules outline which technology characteristics are important for each task description, as well as what values of technology characteristics are better suited for that particular task description.

The rules component of the framework links the task descriptions with the relevant technology characteristics. For example, if the task description is “the task involves the presentation of text” the rules identify which technology characteristics identify technology capable of presenting text to the user (i.e. the technology has a display.) In many cases, the rules are simple binary choices; either a device has a capability that supports the task or it does not. Some task descriptions may have a more detailed rule associated with their assessment; others may have multiple rules indicating multiple technology characteristics are associated with that task description. Some rules form complex “if/then” structures that connect the task descriptions of the technologies that minimally satisfy all the task descriptions.

The rules also serve to validate the current framework by identifying task descriptions that may have no correspondence to technology characteristics or, vice versa, technology characteristics that may have no relevance to practical task descriptions. In cases where multiple technologies satisfy or are equally suitable for the needs of a given task, an in-depth look at the continuous or categorical technology characteristics is warranted.

Ideally, the rules will provide direct guidance for any future automation that is added to the framework. This screened list should allow the user to make a final, educated decision on which technologies are best suited to the task under study.

3.2.4 Example Calculus

The Example Calculus worksheet is an interim development step that was developed to explore options for how to apply the Rules to the Task Descriptions and the Technology Characteristics. The Example Calculus sheet dynamically highlights (using Excel’s conditional cell formatting) those technologies that satisfy the rules used to link the Task Descriptions with Technology characteristics.

The Example Calculus sheet duplicates information by referencing cells in the Rules and Technology Characteristics sheets to clarify how the rules are being applied and to facilitate development. Task descriptions are reproduced in Columns A-F with their corresponding rules listed in Column G. Every rule relates to a specific technology characteristic, which has a reference number in the adjacent Column H. Values of the test case Task 1 (Column G on the Task Descriptions sheet) are referenced in Column F – these are dynamic and will change to reflect the entries on the Task Description sheet. The value required for a given technology to satisfy a given rule is presented in Column I based on the Task 1 description.






This produces an n Task description by n Technologies comparison matrix. At present, the comparisons are done visually through colour coding using conditional cell formatting.

For binary choices (i.e., whether a technology supports a task or it does not), cells are coloured green to represent a given technology satisfying a given task description rule. Uncoloured cells represent an unsatisfied rule. Darker cell shading indicates increasing suitability of the corresponding technology for the task being assessed.

For example, Figure 3-4 shows a screen shot from the Example Calculus worksheet and the following list explains how the conditional formatting has been applied.

• “Task involves aural alerts”:

o The Task Description attribute 8.1.1 has been set to Yes – meaning the assessed task requires presentation of aural alerts.

• The first associated rule for Task Characteristic 8.1.1 is “Must be ‘Yes’ to TC 41”:

o To satisfy this rule the technology must have built in speakers (Technology Characteristic 41).

• In this example in the framework, the task description value is Yes:

o The Example Calculus sheet returns a Yes in Column I.

• Only technologies that have a Yes in technology characteristic 41 (technology has built-in speakers) will be highlighted in green.

• If the user selected No (rather than Yes) to task description 8.1.1:

o The associated rule becomes irrelevant so no value (yes or no) is highlighted for this attribute.

Figure 3-4: Example of Satisfied (Green) and Unsatisfied Rules. To be used to explain the concept only.






For technology attributes that may have differing attribute levels (e.g., viewing angle, size, etc.), another conditional formatting approach applies different colours to indicate the level of technology suitability to a task. In this example, shown in Figure 3-5, is explained in the following list:

• “Task involves multiple operators viewing”:

o Task description 5.3 has been set to Yes.

o Meaning: the task being assessed requires multiple operators to simultaneously view information.

• The second associated rule for Task Characteristic 5.3 is “Larger TC is better”:

o Meaning: technologies with larger viewing angles (Technology Characteristic 26) are better suited for this task description.

• For this technology description, each technology has a viewing angle in degrees:

o The cell on this row corresponding to the technology with the largest viewing angle will be displayed in green.

o The cell corresponding to the technology with the smallest viewing angle will be displayed in red.

o The colour of the other cells of compatible technologies will fall somewhere in between the red and green depending on magnitude of their viewing angle.

Figure 3-5: Example of the Colour Gradient Technology Suitability Assessment Scheme

3.3 Instructions for Using the Framework

The following two sections describe the steps for using the framework. The user section is intended for people who wish to determine which technologies in the framework database would support a task that they are assessing. The administrator section deals with maintaining the framework database technologies and the rules that link the task descriptions to the technology characteristics.






3.3.1 Users of the Framework

The point of entry for a user of the framework is the task description sheet (see Figure 3-1 for a partial screen shot of this worksheet.) Much of the description of the framework sheets has been provided in 3.2 and people wishing to use the framework should refer to the section outlining the task description (3.2.1) for details about the structure of this sheet. Some task descriptions have sub-components that add more detail. These details may be expanded or collapsed by clicking on the “+” or “-” icons respectively on the extreme left of the worksheet.

The user describes the task to be assessed in Column G by selecting an option for the pull-down list in each cell that corresponds to the task descriptions towards the left of the worksheet. Column G is the only task description that is evaluated in this version of the framework. Examples of the task descriptors may be found in Column F. All task description cells in Column G have a default value and the user should verify that the default value is appropriate for the task in question.

When the task description has been completed, the user may view the results of the assessment on the Example Calculus worksheet. The Example Calculus worksheet is an interim development tool that shows which technologies are appropriate for the described task, indicated by colour coding corresponding cells. Technology characteristics that support the task requirements are shaded green in the corresponding device attribute cell while those that are red do not support the task requirements. In this preliminary implementation, the darker cell shading corresponds to greater membership in the evaluation of appropriateness or inappropriateness for supporting the task.

It is anticipated that the results of the assessment will be summarized in the Recommendations worksheet; however, this feature has not been developed.

3.3.2 Framework Administrators

Framework administrator tasks involve updating the task descriptors for additional information, the technology characteristic data and the rules that bind these together in the analysis. People wishing to modify the framework should refer to Section 3.2 for a more detailed description of the layout of each worksheet.

Each task descriptor should have at least one rule that corresponds to a data element in the technology characteristics database. Several rules may be required to capture the interaction between these worksheets and these rules may involve separate cells or be a compound rule in a single cell as appropriate. New task descriptors should only be added to the framework if the administrator has both a rule and a technology characteristic that can be used to discriminate amongst devices. Similarly, it is unnecessary to add technology characteristics that have no corresponding rule that links them to some task requirement.

The rule sheet is not used directly in the assessment, but it contains the logic behind the assessment. This provides a reference to assist verifying and validating the application of rules.






The technology characteristic worksheet should be edited by administrators when new technologies arrive in the market, devices change their capabilities or different brands of devices within a technology are added for consideration. Each new entry should be accompanied by data relevant to the type of functionality if offers (e.g., display, input, communication, monitoring).

3.4 Limitations of the Framework

This project is an initial attempt at creating a framework for assessing the suitability of technologies for tasks and it has a number of limitations compared with the envisioned final product. Several of the risks identified in the kick-off meeting were realized but the most prominent were the broad scope compared with the resource availability and the lack of readily available guidance on technology usability.

The Task Description sheet has a number of entries derived from analysis of a selection of naval command staff tasks. Whether the list of descriptive entries is sufficient or excessive is unknown at present. However, the validation of the list of task descriptors should occur as a natural consequence of applying the framework to real or contrived problems. A detailed task analysis may also be required to uncover additional features, but a spiral-type of development may be more effective at uncovering limitations through application using the current set.

The Technology Characteristics sheet is largely driven by the task requirements and the assessment rules. At the moment, the Technology Characteristics are sufficient for the state of development of the framework.

The Rules sheet that compares the task description to the technology characteristics is currently the major limitation of the framework. Despite a broad search of the literature, only a few documents were found that provided evidence based guidelines. In some cases, the research may not have been conducted yet so suitable, quantitative guidance may not exist beyond binary decisions. Even where guidance has been found in the scientific literature about the suitability of technologies for certain tasks, extrapolation of this knowledge to a measure of utility has not been attempted. Further work is required through additional literature reviews to delve deeper into the technology effectiveness research literature, then render that information into a scalar quantity before a measure of suitability can be derived that discriminates amongst acceptable technologies in a graduated manner.

Finally, the Recommendations sheet has not been attempted as it was not deemed appropriate until a sufficiently comprehensive set of rules had been developed. Instead, an Example Calculus sheet was developed and applied using some of the rules currently in the framework. The current rule set largely comprises binary choices that are not ideally suited for assessing a degree of utility as much as assessing a simple pass/fail criterion. This will be a necessary element of a final framework design, but it falls short of the desired ability to discriminate amongst a set of minimally acceptable technologies while looking for the best candidate or conducting cost-benefit comparisons.






4 RECOMMENDATIONS

The following recommendations are proposed to address the current limitations of the framework and advance it towards the final, desired product.

1. Conduct a more extensive literature review of the scientific literature and industrial best practices to expand and detail the rules.

a. Search for general guidance that can be quantified or has been proposed as design standards.

b. Search for scientific data that can be incorporated into a measure of utility.

c. Elaborate the Rules sheet to incorporate the information found in the literature review.

2. Using Visual Basic, apply the necessary rules to allow automated technology selections based on the selections on the Task Descriptions Sheet.

a. Develop a calculus that integrates the product of applying the rules to the task descriptions and technology characteristics and implement that calculus in the Recommendations sheet.

3. Apply the framework to a set of sample tasks to validate the framework and modify the framework in a series of spiral development cycles that involves the following framework design changes:

a. Refine the task description categories:

i. Elaborate on the task descriptors.

ii. Elaborate on the task descriptor data elements.

b. Refine the categories of technology characteristics:

i. Elaborate on the technology characteristic data elements.

c. Refine the rules:

i. Quantify rules as appropriate.

d. Refine the results display worksheet.

4. Populate the Technology Characteristics sheet with technologies of specific interest to DRDC and the RCN.






5 CONCLUSION

New technologies seldom perform at the promised level and have sometimes introduced additional problems, largely because their application failed to adequately consider the operators’ needs or constraints within the work-domain. In other words, designers designed for the technology capability rather than adopting a user-centred, domain-dependant, design philosophy (Wilson, Pavlas, Sharit & Salas, 2010.) The current work is an initial attempt to develop a tool to screen potential technologies for suitability to help guide user centred application of promising new technologies. The current framework was designed to assess suitability of existing and emerging interface technologies within the context of naval C2. In the future, it may be beneficial to expand the scope of both the Task Descriptions and Technology Characteristics sections of the framework to include (qualitative or conceptual) interaction types (e.g., asymmetric virtual reality interaction with multiple users).

The framework development focussed on establishing a set of task descriptions based on the task analysis literature and investigations of naval command staff activities. The resulting list should provide a firm basis for describing a large number of Navy command and control tasks. The technology description was largely a straightforward taxonomy and characterization of existing and emerging information interface technologies.

The rules relating the task description to the technology characteristics are the critical component of the framework. Unfortunately, there is no single source of information for all of the factors that should be considered. There are some military and industrial standards for older technologies, but many newer technologies have not been tested to the point that firm guidance has been established. Elaboration of the rule set currently within the framework is required to advance the framework towards its envisioned end state.

The task of developing a technology selection framework that is both general and sufficiently comprehensive to cover naval command staff activities proved to be quite challenging and the current framework is incomplete with a number of limitations. However, it does provide a starting point for subsequent development through application in a spiral-development approach to application design. The type of holistic display of included in the Example Calculus worksheet may add more value to decision makers than a simple figure of merit for the suitability of a technological solution to a task requirement. The final decision on technology selection should be based on informed decision and the Example Calculus worksheet demonstrates how the framework might be implemented as a tool to guide users to an educated decision.






6 REFERENCES

Ahlstrom, V. & Longo, K. (2003) Human Factors Design Standard (HF-STD-001). Atlantic City International Airport, NJ: Federal Aviation Administration William J. Hughes Technical Center.

American National Standards Institute. (1988) American national standard for human factors engineering of visual display terminal workstations (ANSI/HFS Standard No. 100-1988). Santa Monica, CA: The Human Factors Society, Inc.

Board of Standards Review/Human Factors and Ergonomics Society. (2002) Human factors engineering of computer workstations standard for trial use (BSR/HFES 100). Santa Monica, CA: The Human Factors Society, Inc.

Cardosi K., M. & Murphy E. D. (1995) Human factors checklist for the design and evaluation of ATC systems (DOT/FAA/RD-95/3.1, DOT-VNTSC-FAA-95-3.1). Washington, DC: U.S. Department of Transportation, Office of Aviation Research.

Cherry, E. C. (1953) Some experiments on the recognition of speech, with one and with 2 ears. Journal of the Acoustical Society of America, 25, 975 – 979.

Cockburn, A., Ahlstrom, D. & Gutwin, C. (2012) Understanding performance in touch selections: Tap, drag and radial pointing drag with finger, stylus and mouse. International Journal of Human-Computer Studies, 3, 218 – 233. doi:10.1016/j.ijhcs.2011.11.002

Department of Defense (1989) Human engineering design criteria for military systems, equipment and facilities (MIL-STD-1472D). Philadelphia, PA: Navy Publishing and Printing Office.

Department of Defense (1989) Human engineering guidelines for management information systems (MIL-HDBK-761A). Philadelphia, PA: Navy Publishing and Printing Office.

Department of Defense (1999. Design criteria standard – Human engineering (MIL-STD- 1472F). Philadelphia, PA: Navy Publishing and Printing Office.

Department of Energy (1992) Human factors engineering design criteria (DOE-HFAC1). Washington, DC: United States Department of Energy.

Farhadi-Niaki, F., Etemad, S. A. & Arya. A. (2013) Design and Usability Analysis of Gesture-Based Control for Common Desktop Tasks. In Human-Computer Interaction. Interaction Modalities and Techniques, Masaaki Kurosu (Ed.) Lecture Notes in Computer Science, Vol. 8007. Springer Berlin Heidelberg, 215–224. doi: http://dx.doi.org/10.1007/978-3-642-39330-3 23






Fleishman, E. A. (1967) Performance assessment based on an empirically derived task taxonomy. Human Factors, 9(4), 349–366.

Gong, R., Xu, H., Wang, Q., Wang, Z. & Li, H. (2013) Investigation of perceptual attributes for mobile display image quality. Optical Engineering. 52, 1 - 10. doi:10.1117/1.OE.52.8.083104

Hinckley, K. & Wigdor, D. (2012) Input Technologies and Techniques. In J. A. Jacko (Eds.), The Human-Computer Interaction Handbook (96 - 123). Boca Raton, FL: Taylor & Francis Group.

Hugo, J. V., & Gertman, D. I. (2015) A method to select Human-System Interfaces for nulcear power plants. Nuclear Engineering and Technology, November, 1–11.

Hugo, J. V., Gertman, D. I., & Tawfik, M. S. (2013) Development of Human Factors Guidance for Human-System Interface Technology Selection and Implementation for Advanced NPP Control Rooms and Fuel Cycle Installations (Contract Report No. INL/EXT-13-30118 (Contract DE-AC07-05ID14517)) (p. 96). Idaho Falls, Idaho 83415: Idaho National Laboratory Human Factors, Controls and Statistics Department. Retrieved from https://inldigitallibrary.inl.gov/sti/5813242.pdf

International Organization for Standardization. (1992) Ergonomic requirements for office work with visual display terminals (VDTs). Part 3: Visual display requirements (ISO 9241-3). Geneve, Switzerland: Author.

International Telecommunications Union. (2003). Series G: Transmission systems and media, digital systems and networks. International Telephone and Connections and Circuits – General Recommendations on the Transmission Quality for an Entire International Telephone Connections. Retrieved from: http://handle.itu.int/11.1002/1000/6254

Kaber, D. B., Alexander, A. L., Stelzer, E. M., Kim, S., Kaufmann, K. & Hsiang, S. (2008) Perceived clutter in advanced cockpit displays: Measurement and modeling with experience pilots. Aviation, Space, and Environmental Medicine, 79, 1007 – 1018.

Kirwan, B., & Ainsworth, L. K. (1992) A guide to task analysis. Boca Raton, Florida: CRC Press, Taylor and Francis Group.

Kisner, R. A. (1991) A framework for selecting suitable control technologies for nuclear power plant systems. Nuclear Safety, 32(4), 511–520.

Kisner, R. A. (1992) A Framework for Selecting Suitable Control Technologies for Nuclear Power Plant Systems. Oak Ridge National Laboratory, Instrumentation and Control Division.






Lamoureux, T., & Banbury, S. (2011) Halifax class modernization program human engineering task analysis report (Contract Report No. SDRL-SE-049 Version 09 (CAE PS Document # 1006-045 Version 02)) (p. 549). Montreal, Quebec: Lockheed Martin Canada.

Lin, C. J., Liu, C. N., Chao, C. J. & Chen, H. J. (2010) The performance of computer input devices in a vibrating environment. Ergonomics, 53, 478 – 409. doi: 10.1080/00140130903528186

MA&D (2015) Integrated Performance Modelling Environment (IPME) User Guide, Version 4.6. Alion Science and Technology.

MacKenzie, S. & Soukoreff, R. W. (2002) Text entry for mobile computing: Models and methods, theory and practice. Human-Computer Interaction, 17, 147 – 198.

Mantiuk, R., Rempel, A. G. & Heidrich, W. (2009) Display considerations for night and low-illumination viewing. In Proceedings of the 6th Symposium on Applied Perception in Graphics and Visualization (pp. 53-58). ACM.

Matthews, M. L., Webb, R. D. G., & Bryant, D. J. (1999a) Annexes to: Cognitive task analysis of the HALIFAX-class operations room officer (Contract Report No. DCIEM No. CR 1999-029) (p. 76). Toronto: Defence and Civil Institute of Environmental Medicine.

Matthews, M. L., Webb, R. D. G., & Bryant, D. J. (1999b) Cognitive task analysis of the HALIFAX-class operations room officer (Contract Report No. DCIEM No. CR 1999-028) (p. 49). Toronto: Defence and Civil Institute of Environmental Medicine.

Matthews, M., Rehak, L., & McFadden, S. (2007) Final Report on Human Factors Research Conducted under the COMDAT Technology Demonstrator Project (Contract Report No. DRDC Toronto No. CR-2007-149) (p. 113). Toronto: DRDC Toronto.

Moacdieh, N. M., Prinet, J. C. & Sarter, N. B. (2013) Effects of modern primary flight display clutter: Evidence from performance and eye tracking data. In Proceedings of the Human Factors and Ergonomics Society annual meeting (pp. 11–15). Thousand Oaks, CA: Sage. doi:10.1177/1541931213571005

National Aeronautics and Space Administration (1989) Man-systems integration standards (NASA-STD-3000A). Houston, TX: National Aeronautics and Space Administration.

National Air Traffic Services (NATS) (1999) Human Factors Guidelines Database. Christchurch, UK: National Air Traffic Services, Human Factors Unit.

Ng, H., Tao, D. & Or, C. K. L. (2013). Age differences in computer input device use: A comparison of touchscreen, trackball, and mouse. In Rocha, A. et al., (Eds.) Advances in Information Systems and Technologies (1015 – 1024). Berlin: Springer-Verlag.






Prinzo, O. V. & Morrow, D. G. (2009) Improving pilot/air traffic control voice communication in general aviation. International Journal of Aviation Psychology, 12, 341 – 357. doi: 10.1207/S15327108IJAP1204_2

Sanders, M. S. & McCormick, E. J. (1993) Human factors engineering and design. New York: McGraw-Hill.

Siegenthaler, E., Wurtz, P., Bergamin, P. & Groner, R. (2011) Comparing reading processes on e-ink displays and print. Displays, 5, 268 – 273. doi:10.1016/j.displa.2011.05.005

Smith, S. L. & Mosier, J. N. (1986) Guidelines for designing user interface software (ESD-TR- 86-278). Hanscom AFB, MA: Electronic Systems Division.

Spencer, R. H. & Johnston, R. P. (2003) Technology Best Practices. Hoboken, NJ: John Wiley

Thomson, W. and Wells, P. N. T. (2013) The perception of visual information, Springer, New York.

Wickens, C. D. (2000) The When and How of Using 2-D and 3-D Displays for Operational Tasks. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 44, pp. 3–403–3–406). http://doi.org/10.1177/154193120004402107

Wilkins, A., Veitch, J., & Lehman, B. (2010). LED lighting flicker and potential health concerns: IEEE Standards PAR1789 Update. Retrieved from http://www.essex.ac.uk/psychology/overlays/2010-195.pdf

Yau, Y. J., Chao, C. J. & Hwang, S. L. (2008) Effects of input device and motion type on a cursor-positioning task. Perceptual and Motor Skills, 106, 76 – 90.






7 ABBREVIATIONS

2D Two dimensional 3D Three dimensional

AMOLED Active-Matrix, Organic, Light Emitting Diode ANSI American National Standards Institute AR Augmented Reality

Bio Biological BSR Board of Standards Review

C2 Command and Control COA Course of Action

dB decibels DOD Department of Defence DOE Department of the Environment DOF Degrees of Freedom DRDC Defence Research and Development Canada

EEG Electro-Encephalogram

GUI Graphical User Interface

HFDG Human Factors Design Guide HFES Human Factors and Ergonomics Society HMD Head Mounted Display HSI Human-System Integration HUD Head Up Display Hz Hertz

IP Ingress Protection IRFID Implanted Radio Frequency Identification ISO International Organization for Standards

LAN Local Area Network LCD Liquid Crystal Display LED Light Emitting Diode

MA&D Micro Analysis and Design (Alion Scientific) MIL Military MIW Maritime Information Warfare mm Millimetre ms Milliseconds






NASA National Aeronautics and Space Administration

OLED Organic Light Emitting Diode

PPE Personal Protective Equipment

RCN Royal Canadian Navy RFID Radio Frequency Identification

SA Situation Awareness SHINCOM Shipboard Internal Communications SOW Statement Of Work STD Standard

UI User Interface USB Universal Serial Bus

VR Virtual Reality






APPENDIX A ANALYSIS OF A SAMPLE OF NAVY COMMAND STAFF TASKS

The following are a selection of task descriptions developed by Lamoureux and Banbury (2011) for the Halifax Class frigate and a subsequent decomposition of those tasks into components used to develop the task description entries of the framework.

Generic Task Inventory Task Description Initiating Conditions Information Required Decision

Component

Cognitive Demand Job Aids or References Required Action

Requirements Accuracy Required

Feedback Required

Communication Required

Interaction Required

Task Duration

Task Frequency

V A C P

Assign tasks to team or individual

5 After receiving handover/team briefings and reviewing the watch schedule, the operator assigns tasks to specific teams or members of the team

• Coming on watch • Receipt of

direction from ORO/Command

• Conducted periodically as required

• ORO/Command’s directions/ intentions

• Mission requirements and objectives

• Details of ongoing and/or planned activities

• Specifics of the tactical situation

This task has a significant decisional component that overshadows the other cognitive content.

• Required information is provided in a briefing

• Required information is available in the form drill procedures/checklist

• Required information is available in the form of written notes

• Required information is available via the CCS/SSD

• Key/dial SHINCOM and speak or

• Conduct face-to-face conversation

• Interpret displays

Medium • Verbal acknowledgement

• Voice (SHINCOM)

• Face-to-Face (Verbal)

• Other crew members

Time: Frequently

Perception Cognition Behaviour Environmental Visual Auditory Text/Verbal Spatial/Graphical Comprehension Projection Motor Speech

Read paper orders, schedules

Receive verbal sitrep

Large amounts Complex Judgement of workload

Control SHINCOM Communicate to remote operators

Communication mode: Aided speech, Access to SHINCOM

Read paper ROE Receive verbal orders

Assessment of tactical situation (Graphical/Textual)

Implications of tactical situation (Graphical/Textual)

F2F conversations with team leads

Communication mode: Direct access or shared video

Review checklists Understanding of RMP (Graphical)

Implications of RMP (Graphical)

Assign tasks verbally through speech

Communication mode: Unaided speech, networked speech

Read state/status boards

Understanding CO intent

Assign tasks verbally through text

Communication mode: networked

Review timeline Interact with graphical displays

Communication mode: Direct access or shared video

Review notes on formal communications

Verbal acknowledgement

Communication mode: SHINCOM and unaided speech

Observe teams at work

Request sitreps Communications mode: text by network

Review drill checklist

Request sitreps Communication mode: unaided or aided speech SHINCOM, Internal nets,

Review written notes

Mobility: Important (observe teamwork)

Review online information (text)

Frequency of use: High

Accuracy requirement: Medium

10 February 2016 – A-1 – 5878-001 Version 02

© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence, 2016 © Sa Majesté la Reine (en droit du Canada), telle que représentée par le ministre de la Défense nationale, 2016



Generic Task Inventory Task Description Initiating Conditions

Information Required

Decision Component

Cognitive Demand

Job Aids or References Required Action


Feedback Required



Task Duration Task Frequency

V A C P

Receive non-urgent information

8 The operator receives non-urgent information from the appropriate operator so that proper actions can be taken

• A pipe is made

• A message has been received

• None required This task includes a visual, auditory or psychomotor component, with only modest decisional aspects to any cognitive content

• None • Listen


• Medium • Feedback cues are not required

• Voice (SHINCOM) • Other crew members

Time: Occasionally


Alert (pipe) Communication mode: SHINCOM Verbal

(message) Verbal Simple Message

Spatial Complex (cords?)

Minor Minor?: Interpret implications

Frequency of use: Low

Accuracy requirement: Medium Mobility: Desirable







Decision Component

Cognitive Demand Job Aids or References

Required Action


Feedback Required



Task Duration

Task Frequency

V A C P

Conduct simple procedures - Non CCS

35 The operator will be required to conduct simple procedures such as issuing keys, mustering documents, or conducting EMCON procedures. This can also include performing corrective action. The operator takes action to correct an error that could adversely affect the mission or personnel safety. The operator or any other member of the operations team can initiate this task in response to the identification of an error. For EMCON procedures the operator, having been alerted to an emission control (EMCON) violation that could compromise security, takes action IAW SOP's to enforce the EMCON policy. For an unauthorized radio transmission, for example, the operator may send a code-word by radio message to remind the other operator that radio silence is in effect. This important task may take seconds or minutes, depending on the nature of the EMCON violation. The operator refers to various displays, logs or written notes as required to remedy the situation in a timely manner. Communication may be required.

• Requirement in accordance with drill sequence

• Corrective action

• Routine procedure

• SOPS • EMCON restrictions in place

• Planned activities • Specifics of the tactical situation (e.g. threat/friendly forces, ROE, weather and sea state, oceanographic information, etc.)

• Mission requirements or objectives

This task has a significant decisional component that overshadows the other cognitive elements.

• Aide memoirs • Decision aids • SOPS

• Listen – comprehend

• Read • Manipulate SHINCOM

Medium Aural or visual indication of procedure initiation

• Face-to-face • Voice (SHINCOM)

Person-to-person

Time: Rare


Monitor displays Detect transmissions

Simple messages

Simple symbols Interpretation of symbol/message: Low

Text entry message Verbal message Communication mode: Direct; SHINCOM;

Monitor alerts Monitor alerts Plan and ROE status Frequency: Low Read messages Tactical situation Read Aide memoirs

Read SOPs







Decision Component


Required Action


Feedback Required




V A C P

Reading 37 The operator, while in position, reads messages as they arrive during the watch. Messages received may be in any operational format (OPERATIONSTAT, OPGEN, OTASK, etc), or on any topic, AAW, ASW, MIO, RMP, INTEL, ROE's etc which pertain to the mission. Message received may also amplify previously received messages. The operator usually marks the message as read and may make notes as required either separately or on the indicated message.

Receipt of messages

• Status of message • Urgency to read / comprehend

This task has a significant decisional component that overshadows the other cognitive elements.

None • Read • Write

Medium No immediate feedback cues required, however eventual feedback is expected

None None Time: Occasional


Read text messages

Text (Complex) High Implication to RMP Annotate messages (writing; typing)

Frequency: Moderate

Implication to schedule events

Mobility: Desirable







Decision Component


Required Action


Feedback Required




V A C P

Select optimum weapon/sensor range scale given current requirements

57 Operator uses the CCS to select between the different pre-defined sensor and weapon systems options in order to best satisfy the tactical objectives given the prevailing conditions and tactical environment. For example: Select radar to be used.

TBD • Mission requirements and objectives

• Specifics of the tactical situation (e.g., threat/friendly forces, ROE, weather and sea state, oceanographic information, etc.)

• ORO’s/Command’s directions/intentions

• Detailed of on-going and / or planned activities

This task is a decisional task consisting mostly of cognitive activity.

• SOPS

• Decision aids

• Aide memoirs

• Notes

• Manipulate SSD settings and controls


• Read

• Analyze

High Change in CCS display

• None Interaction with CCS information

Time: Frequent


Read display (text)

Large Large (RMP) Moderate (Commander’s intent)

Manipulate SSD controls

Frequency of use: High

Read stateboard (text)

Low (weather) High (Tactical situation)

High (Tactical situation) Manipulate CCS Accuracy requirement: High

Read orders (text)

Moderate/High (schedule)

Mobility: None

Read SOPS Moderate: Analyze displays

Read Aide Memoires (checklist)

High (Integration of disparate information)







Decision Component


Required Action


Feedback Required



Task Duration

Task Frequency

V A C P

Monitor operational task performance of individual or team

62 The operator will visually and/or auditorally monitor both the actions and communications of those individual crew or operational teams reporting to them to ensure that they have recognized incoming information appropriately and are responding to it (i.e.: research, interpretation, actions & communications) appropriately for the current tactical requirements. The operator will confirm that procedures are being adhered to. It they are not, the operator will take action to correct the noted deficiency.

• Occurrence of significant event


• Coming on watch/change of watch

• ORO/Command’s directions/ intentions



• Status of sensors/systems

• Details of on-going and/or planned activities

• This task has a significant decisional component that forms a large part of the cognitive content.

• Required information is provided by ORO/Command

• Required information is available in the form of hard copy SOPS/references/ documents

• Required information is provided by radio/telephone or face-to-face communication


• Organize • Conduct face-to-

face conversation

• Monitor & interpret displays, or read

• Physically move to different workstation or team areas in Operations Room

High • Verbal acknowledgement

• Verbal reporting

• CCS plot

• Voice (SHINCOM) • Face-to-Face

(Verbal)

• Complex – cross teams and within sub-teams

Continuous Frequently

Perception Cognition Behaviour Environmental

Visual Auditory Text/Verbal Spatial/Graphical Comprehension Projection Motor Speech Observe individuals

Listen to discussions

Moderate (discussions)

Moderate (Interpret displays)

Moderate: Behaviour actions correspond to expectations (visual)

Move to operator workstation

Verbal (issue orders) Communication mode: Direct

Observe/read displays

Complex (intra and inter team interactions)

Communication mode: Aide voice (SHINCOM)

Review of plan Low (review plan)

High: Decision making Moderate (review plan) Verbal acknowledgement

Review orders Verbal reporting

Display manipulation (CCS plot)

Communication mode: Graphical

Mobility: Desirable

Frequency of use: Often

Accuracy requirement: High





Generic Task Inventory Task Description Initiating Conditions Information Required Decision

Component


Required Action


Feedback Required



Task Duration

Task Frequency

V A C P

Monitor performance of operational procedures

63 The operator will visually or auditorally monitor both the actions and communications of those individual crew or operational teams reporting to them to ensure that appropriate procedures are being followed given the current tactical context. The operator will confirm that procedures are being adhered to and may be required to mirror the operation to ensure errors can be corrected immediately. If they are not, the operator will take action to correct the noted deficiency. Operational procedures include ASMD reactions, TCMs, Resolve Procedure, and Warning Shot drill.

• Occurrence of significant event


• Coming on watch/change of watch

• ORO/Command’s directions/intentions



• Status of sensors/systems

• Details of on-going and/or planned activities

• This task has a significant decisional component that forms a large part of the cognitive content.

• Required information is provided by ORO/Command

• Required information is available in the form of hard copy SOPS/references/ documents

• Required information is provided by radio/telephone or face-to-face communication


• Organize • Conduct face-to-

face conversation • Monitor & interpret

displays, or read • Physically move to

different workstation or team areas in Operations Room

High • Verbal acknowledgement

• Verbal reporting • CCS plot • Stateboard

information

• Voice (SHINCOM) • Face-to-Face

(Verbal)

• Interaction with command and other sub teams will be required

• Review & monitoring of CCS information

Time: Continuously

Perception Cognition Behaviour Environmental

Visual Auditory Text/verbal Spatial/Graphical Comprehension Projection Motor Speech

Observe individuals Listen to individuals colocated

Moderate Minor High (understanding discussion)

High (understanding implications)

Move to operator’s workstation

F2F conversation Communication mode: unaided or aided speech (SHINCOM)

Read orders Listen to individuals remote

High (Understanding of RMP)

High (Implications for RMP)

Interact with CCS displays

Verbal conversation Communication mode: Direct access or shared display

Read SOPs High (Implications for schedule)

Verbal acknowledgement

Mobility: Important

Read information online

Verbal reporting Frequency of use: High

Observe graphical displays

Accuracy requirement: High

Read stateboard Communications mode: Radio/telephone

Communications mode: CCS display (Text/graphics)





APPENDIX B TASK DESCRIPTION FIELDS

Communication and Collaboration

• Task involves communication with other operators

o Task involves Face-face communication

o Task involves verbal communication

• Task involves text communication

o Task involves hand written

• Task involves SHINCOM/other communication hardware

• Task involves Synchronous, Asynchronous, or both types of communication

• Task involves multiple people

o Co-located or networked

o multiple operator input

o multiple operators viewing

• Task involves the need to privately view information

Information and Data Description

• Task involves receiving information

• Task involves Unclassified, Sensitive, or Classified Information

o Task involves audio information/hearing

Task involves aural alerts

if yes, the task involves simple tonal alerts or complex verbal alerts

Task involves adjusting the volume of audio

Task involves private audio for single user

Task involves large amount of audio information

Task involves listening to multiple sources of audio

o Task involves viewing visual information/vision

Task involves viewing textual information

If yes, the task involves viewing minimal, moderate, or large amounts of text

o Task involves viewing graphical information

If yes, the task involves viewing minimal, moderate, or large amounts of graphics

If yes, the task involves viewing simple line or complex textured graphics

The visual quality required for the task is fine detail or coarse detail

Task involves coloured information

10 February 2016 – B-1 – 5878-001 Version 02

© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence, 2016 © Sa Majesté la Reine (en droit du Canada), telle que représentée par le ministre de la Défense

nationale, 2016



Task involves storage of text messages

• Task involves the ability to stream/play video

• Information is changing rapidly

• Information must be quickly accessible

• Task must have 100% up time

• Task requires hands free operation

Actions and Information Manipulation

• Task involves user input or information manipulation

o Task involves text input

Speech

Task involves typing text

If yes, the task involves typing minimal, moderate, or large amounts of text

o Task involves editing or annotating existing messages

o Task involves selecting items on a screen

o Task involves zooming in/out on graphics

o Task involves scrolling through text

o Task involves panning across images

o Task involves continuous input

o Task involves a high level of input precision

if yes, errors are unacceptable or undesirable

o Task involves fast recovery from errors if they occur

o Task involves recording video/pictures

o Task involves following checklists or procedures

o Operator error tolerance is Low, Medium, or High

• Task must be completed while moving between work stations

• Task occurs rarely, occasionally, or often

Decision Making and SA Description

• Task involves data monitoring

• Task involves Scheduling tasks, alerts, or reminders

• The density of information in this task is Low, Med, or High

• Task involves evaluation of tactical situation

• Task involves managing operators

o Observe individuals



nationale, 2016



o Monitor conversations

• Task involves understanding remote environmental conditions

• Task involves a high level of concentration

• Task involves multi-tasking

Task Environment and Operator Clothing

• Ambient working conditions must be considered

o Task may be completed in direct external light (glare)

o Task is completed in variable environmental lighting conditions

o Ambient noise environment is low, high, or variable

Task must not generate unnecessary noise

o Operating temperature range

o Task is subject to external motion

• Task puts any technology at risk of impact

• Task has a risk of exposure to liquids

o Spills

o Spray

o Immersion

o POL

• Task involves PPE

o Task involves a respirators

o Task involves a mask

o Task involves protective goggles

o Task involves gloves

o Task involves flash gear

o Task involves environmental weather clothing

• Connectivity available in environment

o Wifi

o Bluetooth

o NFC

o Network access

o Local

o IFRD



nationale, 2016



o Cell phone network

o Radio

• Task has space restrictions

o Area restriction

o Height restriction



nationale, 2016



APPENDIX C TECHNOLOGY CHARACTERISTIC FIELDS

• Display technology Yes/No

• Input technology Yes/No

• Communication technology Yes/No

• Monitoring technology Yes/No

• Physical Specifications

o Weight kg or g

o Length cm

o Width cm

o Height cm

o Power Consumption w

o Inputs [list]

o Outputs [list]

o Storage Yes/No

o Built-in Capacity GB

o Storage type [Hard disk/SSD/Flash]

o Memory card reader Yes/No

o RAM Size GB

o Hot-swappable HD Yes/No

o Processor speed Ghz

o Processor Cores [Num]

o Battery Yes/No

o Capacity mAh

o In-use Life Hours

o Charge time Hours

o Min Op. Temp Degrees C

o Max Op. Temp Degrees C

o Wired Op. Yes/No

o Water proof rating IPS

o Dust proof Yes/No

o Totable Yes/No

o Wearable Yes/No

10 February 2016 – C-1 – 5878-001 Version 02


nationale, 2016



o Tactile Feedback Yes/No

• Display

o Display Type [LED/LCD/OLED/SOLED/E-INK]

o Backlight Type [BacklitLED/SidelitLED]

o Viewing Angle Degrees

o Screen Size cm

o Resolution Pixels

o Aspect Ratio [Ratio Length:Height]

o Refresh Rate Hz

o Brightness cd/m2

o Adjustable Brightness Yes/No

o Dynamic Contrast Ratio [Ratio]

o Static Contrast Ratio [Ratio]

o Colour Yes/No

o 3D Yes/No

o HMD Yes/No

o Active Glasses Yes/No

o Autostereoscopic Yes/No

o Colour Gamut NTSC

o Colour Depth Bits

o Screen Finish [Glossy/Matte]

o Anti-glare Yes/No

o Screen Material [Glass/Plastic/Sapphire/Gorilla Glass]

• Audio

o Built-in speakers Yes/No

o Built-in microphone Yes/No

o Volume control Yes/No

o Max volume dB

o Output power W

o Headphone Jack Yes/No

o Jack Size mm



nationale, 2016



• Input

o Response time ms

o Touch Yes/No

o Touch Type [Capacitive/projective capacitive/resistive]

o Multi-touch Yes/No

o Hands free Yes/No

o Camera Yes/No

o Sensors [list]

o Bio sensors [list]

o Hard-key QWERTY Keyboard Yes/No

o Stylus Yes/No

o Curser Control Device Yes/No

o Hard buttons Yes/No

o Other input [list]

• Connectivity

o Wi-Fi Yes/No

o Bluetooth Yes/No

o 3G Yes/No

o 4G Yes/No

o GPS Yes/No

o NFC Yes/No

o IR Yes/No

o Radio Yes/No



nationale, 2016



APPENDIX D TECHNOLOGIES CURRENTLY REPRESENTED IN THE FRAMEWORK TECHNOLOGY DATABASE

The following subsections provide a brief description of each technology currently included in the framework as well as an explanation for exclusion of some technologies.

D.1 Wearable

A wearable technology is defined as any portable technology that is designed to be worn on the body. Examples of current wearable technologies include the following:

1. head mounted displays (HMDs) (e.g., Oculus Rift or Google glass)

2. smart watches (e.g., Apple Watch or Samsung Gear S2)

3. wristbands (e.g., Sony Smart Band or Fitbit)

4. smart clothing (e.g., Hexoskin wearable body metrics shirts)

5. smart earphones (e.g., Bragi Dash)

These wearable technologies are briefly described below.

HMD –AR HMD augmented reality (AR) devices are a means for displaying data superimposed over a display of the real world environment or displaying information in a way that allows the wearer to simultaneously view the environment. HMD-AR is generally lower resolution than the VR variant.

HMD – VR Head mounted display virtual reality (VR) devices occlude the outside environment view, although some devices incorporate cameras to provide the environmental view indirectly through a computer-mediated composite image (aka pass-thru video). HMD-VR devices generally have a mid-range of display resolution. However, this attribute may cause vergence problems for the user (e.g., eye strain, fatigue, etc.) as the devices typically provide an infinite focal depth but the viewing angle between the eyes is oriented around a point much closer.

Smart Watch Smart watches are capable of some basic tasks available on smart phones. Smart watches generally have a small touch screen (1 to 2 inch diagonal) on the face of the watch. Some smart watches also have physical buttons and varying levels of water resistance. Smart watches require regular charging (approximately every day or two) and some form of wireless network connectivity to receive notifications.

Wristband Wristbands typically have no display or input capability, although some products may provide such capabilities intermediate between the wristband and the smart watch. Most wristbands are passive monitoring or location trackers. Some wristbands are now developing the ability to provide simple notifications or alerts. Wristbands require regular charging (approximately weekly) and some form of wireless network connectivity to receive alerts.

Smart Clothing Smart clothing typically has no display or input capability. Most smart clothing is intended primarily for passive monitoring or location tracking with the potential for providing notifications in the future. Smart clothing requires some form of wireless network connectivity to send data to a receiver station.

Smart Earphone Smart earphones generally provide three functions: Audio playback, tracking (activity level or location) and communication. Smart earphones are lightweight and discrete – they generally sit in-ear. Some models also have the ability to store audio messages. Smart earphones require regular charging (every few hours) and some form of wireless connectivity.

10 February 2016 – D-1 – 5878-001 Version 02





D.2 Totable

A totable is defined as any technology that can be reasonably carried by an operator. These devices are generally lightweight and battery powered. Examples of current totable technologies include smartphones, laptops, tablets and e-Readers. These four totable types are described in this section.

Smartphone Smartphones have touch screens with computing power equal to tablets and some laptops. They are capable of producing audio and tactile alerts, sending and receiving data, and are capable of both voice and text communication. Commercial smartphones may be fragile, require regular charging (typically at least every two to three days) and some form of wireless connectivity (network or cellular communication.)

Laptop Laptops have the highest computing power of all the totables. They have high resolution, moderately large screens and physical keyboards; some provide touch screen capabilities. Laptops are the heaviest of the totables and typically require frequent charging (daily) but they can be operated as wired devices (plugged into hydroelectric mains). Laptops benefit from some form of wireless network connectivity but they can be used effectively as stand-alone computational devices.

Tablet Tablets have the similar capabilities to smartphones; however, they have bigger screens and larger capacity batteries (typically requiring recharging daily) making the presentation and manipulation of information easier in some applications. Tablets often have wireless network capability and may plug into computer USB (Universal Serial Bus) ports.

E-Reader E-Readers differ from tablets by using an e-ink technology screen that is intended to improve reading of text. However, e-Readers have minimal computing power and are optimized to display static information (e.g., text.) The displays on e-Readers are typically monochrome, touch capable and can be used with gestures or typing with an onscreen keyboard. E-Readers require occasional charging (typically days to weeks) and some form of wireless network connectivity if they require updating of stored information.

D.3 Fixed

A fixed technology is defined as any technology that is not readily portable during use including:

• 2D monitors

• passive, active and glasses-free 3D monitors

• active multi-view monitors

• transparent monitors

• Head Up Displays (HUDs)

• work stations or kiosks

• speakers

• wall displays

These fixed types of technologies are briefly described below.






2D LCD/LED Monitor 2D (two-dimensional) LCD (Liquid Crystal Display) and LED (Light Emitting Diode) monitors are capable of extremely high resolution (up to 4k or a display 3840 pixels wide by 2160 pixels high.) Both technologies come in a wide range of sizes. Commercial monitors can refresh at rates above 120Hz (far above the frequency that humans are capable of detecting) and some have touch screen input capabilities. 2D monitors have a robust viewing angle allowing multiple users to view displayed information from oblique angles.

3D monitor Three-dimensional (3D) monitors come in three main types: active, passive and glasses-free. Both active and passive technologies require each viewer to wear special glasses to filter the displayed images. The biggest advantage of 3D monitors over traditional monitors is their ability to convey depth in a scene. This could be useful when information is being used to build an accurate spatial picture of an environment. Wickens (2000) suggests that the use of 3D ego and exocentric views support general situational awareness and can reduce scanning costs in search tasks, but 3D presentation can incur other cognitive costs due to ambiguities or uncertainties that translate into a lack of precision. It should also be noted that presentation of 3D images from the egocentric view may increase the occurrence of tunnel vision (attention fixation on some detail to the exclusion of other details) actually reducing situational awareness in some cases.

Active 3D relies on battery-powered glasses that are synchronized with the monitor to present images to one eye at a time, rapidly alternating between eyes, creating the illusion of binocular viewing of physical objects. These monitors produce bright video and high picture resolution, generally superior to passive 3D technologies (Burks et al., 2014). However, the glasses require charging and the shuttering effect reduces the monitor’s refresh rate per eye to one-half of the maximum rate. They are also known to cause headaches for some users.

Passive 3D presents two images to the viewer simultaneously and relies on differentially polarized lenses that block one of the stereo images so that each eye receives a slightly different image. The glasses do not require charging and are much cheaper than active 3D glasses. Passive 3D glasses do not alter the frame rate; however, because they are using polarization, they can reduce the luminance of the displayed image.

Glass-free 3D uses autostereoscopic technology that creates a parallax barrier that causes each eye to see one of two (stereo) images presented together. The major benefit to this 3D technology is the lack of glasses. This technology can produce picture distortion and has a limited viewing angle (almost straight on) making it most suitable for a single user only.

Multi-view monitor Multi-view technology is new to the commercial market but it is essentially an active 3D display system that, rather than displaying two, stereo images of the same scene, it simultaneously displays two different images from separate display sources. This allows users wearing synchronized glasses to shutter in time with one input source to view one of the two input sources on a single monitor. There is no loss of brightness or resolution but there is a halving of the frame rate because the monitor is alternating between the feeds.

Transparent monitor Transparent monitors present information on a see-through display (typically Organic Light Emitting Diode, OLED) that is sandwiched between glass layers. Transparent monitors provide high resolution, colour displays. They are dimmer than regular OLEDs and there can be some viewing difficulties depending on who is behind the monitor.

Speakers Speakers can be integrated or stand-alone technologies. Standalone speakers can be used to produce the experience of 3D audio, which allows the user to spatially discriminate sound sources although perceptual ambiguities may occur (e.g., front-back reversal.) These technologies could be useful as a means for using aural alerts to draw the user’s attention to the appropriate interface location.






Wall display Wall displays can include large 2D or even 3D monitors, projector systems, etc. The distinguishing characteristic here is that the displays are large or a collection of multiple screens, each displaying a portion of an overall image.

D.4 Input devices

Input devices are defined as any piece of technology used solely to record a user’s input for interaction with some other system or technology. The subsections below explain their exclusion from the framework’s technology database. Although they were not included in this version of the framework, they could be added with a few modifications to the framework structure and rules.

Keyboard∗ Keyboards can be either physical (e.g., desktop or laptop keyboard) or on-screen (e.g., tablet or smartphone). On-screen keyboards are not themselves a user interface technology but rather a feature of a higher order technology reflecting software UI design. Physical keyboards, however, have been extensively used in the naval environment (Lamoureux & Banbury, 2011) and their replacement is not anticipated, particularly for fixed workstations. For these reasons, keyboards were not included in this version of the framework.

Trackball* Trackballs are often physically fixed to a workstation with multiple electro-mechanical buttons for selection and hot-key functions; add-on trackball devices also exist. Integrated trackballs have the advantage over traditional computer mice in that they cannot roll away in a moving environment. Trackballs are often used for traditional display navigation in dynamic environments such as on naval platforms. Their performance has also been extensively studied in comparison to traditional computer mice (Cockburn, Ahlstrom & Gutwin, 2012; Ng et al., 2013) and specifically in moving work environments. For these reasons, trackballs were not included in this version of the framework.

Mouse* The traditional computer mouse is at a disadvantage in the naval context because of its dynamic untethered properties. Mice have been omitted from the framework for similar reasons to trackball input devices.

Force Touch* Force touch is an interaction type within a supported technology; that is, forced touch is the ability complete differing actions by applying different amounts of force to a touch screen. One problem with regular touch screens is unintended interaction resulting from incidental contact with the screen. With force touch, users can set a pressure threshold for key presses that require more intentional inputs for data entry and should reduce incidental contact errors. However, because this is a feature or characteristic of a higher order technology (e.g., tablet) force touch was excluded from the current version of the framework.

Voice Recognition* Voice recognition allows for the hands free dictation or the execution of simple commands by speech. The precision of this technology is highly variable dependent on environmental noise, users’ quality of input and knowledge of a discrete set of verbal commands – this could be challenging in a naval environment. Voice recognition is a functionality of higher order computer technology rather than a device and may be a capability implemented in a device. For these reasons, voice recognition was not included in this version of the framework.

Gestures* Gestures are commands made on touch screens by using multi-touch interface software. Gestures were originally popularized by Apple and include pinch-to-zoom, two finger pan, three finger page turn, etc. Gestures allow functionality without reliance on dedicated buttons or delving into dropdown menus for accessing tools. Full body gestures should be avoided as they can lead to physical fatigue quite quickly (Hinkley & Wigdor, 2012). Gesture based input is also a feature of higher order technologies rather than a technology itself. For these reasons, gestures were not included in this version of the framework.

∗ These devices are not included in the framework technology database. 10 February 2016 – D-4 – 5878-001 Version 02





D.5 Miscellaneous

The miscellaneous category is used to catalogue technologies or devices that do not fit conveniently into the other high-level categories.

Bio-monitoring Bio-monitoring uses standalone sensors or those integrated into other technologies to measure physiological variables like heart rate, blood pressure, galvanic skin conductivity, electrical activity at the scalp (Electro-Encephalogram, EEG) or blood oxygen level.

Eye tracking Eye tracking can be used to monitor an operator’s attention and direct warnings or alerts to where the operator is looking to enhance detection.

Physical morphing displays Physical morphing displays, like those made by Phorm (https://www.phorm.com/), are actually thin overlays that can be incorporated into touch screens. A gel or liquid is pumped through micro channels that bulge on top of the on-screen keys or buttons to provide haptic feedback about where the user is touching. The benefit of these overlays is that they can be turned on or off and remember multiple button layouts to fit whatever GUI (Graphical User Interface) is currently presented. When not in use the bulges deflate and the screen returns to its smooth surface.

Flexible displays Flexible displays include displays that can be molded by the user into a curved surface, from a wearable screen warped around the wrist to a screen at a console curved to provide a large viewing angle. These displays are generally monochrome e-ink displays with poor image quality when compared to other modern display technologies. However, flexible active-matrix, organic, light-emitting diode (AMOLED) screens are emerging that promise high-resolution image quality.

Holographic displays Holographic displays use electro-holography for recording and reconstructing 3D objects. These displays have an advantage over other 3D displays in that they can reproduce 3D images with full parallax. There are currently no holographic displays on the market, however, prototypes have been built in the laboratory. In 2010, the most advanced holographic display had a refresh rate of 2 seconds. Some researchers suggest holographic televisions could be on the market within 10 years.

e-Ink e-Ink is a monochrome display technology that produces high contrast images and provide a wide viewing angle. e-Ink displays can produce relatively high-resolution images and can be produced in almost any size display (portable or large wall mount). Mostly known because of their use in e-readers, e-Ink displays provide an excellent means for presenting text and typically have ultra-low power requirements compared to other types of display technologies.

Asymmetric interactions* Asymmetric interaction is less of a technology and more of a strategy for implementing VR and AR technologies. In the gaming community, asymmetric interactions usually refer to a type of interaction where one person is playing (or working) in virtual reality while the other person is supporting the task in the real world. An example of an asymmetric interaction would be a pilot operating a drone using VR goggles that obscure real world viewing while a second operator could be reading a map and providing direction to the pilot.

Implanted RFID tag Radio Frequency Identification (RFID) tags can be active or passive devices that communicate to external monitoring systems. The interaction range is typically 10 cm to 200 m. RFID tags provide a means of transferring persistent data automatically (e.g., passwords, employee number, part number, banking data, etc.) RFID tags can be implanted under the skin but are more commonly found as fobs or keys cards. The most obvious use for these tags would be controlled entry into locked rooms or unlocking computers by proximity to the appropriate user.






Retinal projection Retinal projectors draw a raster image (the image is created using a dot matrix structure) directly onto the user’s retina, which gives the illusion of a conventional display floating in space. Some advantages of retinal projection are:

• scanning to one eye allows for overlaying information in the real world

• scanning to both eyes with an offset angle allows for presentation of 3D images

• images are seen only by the intended viewer

Retinal projectors require minimal space and could be used as a station for presentation of classified information or for multiple stations as a means for reducing the amount of ambient light emitted into the environment.






APPENDIX E RULE DESCRIPTIONS

E.1 Communication and Collaboration

Task involves communication with others The importance of technology in this section depends on whether or not the technology is directly involved in the communication interaction. The first scenario involves the participants communicating via a given technology (e.g., video call), while the second involves direct in-person communication where the technology is involved in a secondary fashion (e.g. monitoring data while communicating orders to others.)

Task involves face-face communication If a given technology is directly required to support face-face communication between people remote from each other (e.g., video call), then that technology must have a means to transmit audio and video signals. It may also be desirable to record and store audio and video messages in some situations.

Task involves verbal communication If a given technology is directly required to support verbal communication between people apart from each other (e.g., radio call) that technology must have a means to transmit audio signals. It may also be desirable to record and store audio signals.

Microphones used for the transmission of speech should have a frequency range between 200 – 6100Hz to provide maximum intelligibility as well as a dynamic range of at least 50dB (MIL-STD-1472F, 1999).

Task involves text communication If a task requires text communication, any technology under consideration must have a means for inputting text (e.g., hard keyboard or onscreen keyboard), accept handwriting using a stylus or employ speech recognition, as well as a display for input feedback to the user.

All text must subtend a vertical visual angle of at least 15 arc-min (MIL-STD-1472F, 1999). This means that any technology considered must be capable of displaying a reasonable amount of text at or above this size. Depending on the nature of the task, a smaller display should be considered only if a minimal amount of text is required (e.g., notifications).

Any display used for presenting text should be capable of producing a minimum luminance ratio of 5:1 between text and background (MIL-STD-1472F, 1999).

For any keyboard used for text communication the minimum horizontal width of a key should be 12mm (ANSI, 1988), however, when environmental motion is present this minimum can increase to as great at 30mm especially when considering on screen keyboards.

Task involves hand written text Some tasks may benefit from the use of hand written communication. If a technology is to be used for hand written communications (as opposed to paper and pen/pencil) a stylus would be the likely mode of input. The most important thing to consider in this case is the refresh rate or the draw rate of the stylus. The refresh rate for the stylus should be sufficiently high to assure the appearance of a continuous track with no or minimal draw delay (NASA-STD-3000A, 1989).

Task involves SHINCOM/ other communication hardware If a task requires the use of SHINCOM, the technology under consideration should not interfere with the users’ ability to operate the SHINCOM and the technology must communicate with the SHINCOM technology in a supported mode.

Task involves Synchronous or Asynchronous communication If a task requires synchronous communication over a network the technology under consideration much provide minimal lag between users. However, in some cases asynchronous communication is acceptable (e.g., email) and the lag between users is no longer a concern. The recommended maximum lag is 150 ms (ITU-T G.114, 2003).

10 February 2016 – E-1 – 5878-001 Version 02





Task involves multiple people Some tasks will involve multiple people, either co-located or remote, interacting via a network.

Co-located or Networked If users are co-located, any technology under consideration for communication need not support network connectivity by default unless there is a specific requirement for indirect exchange of digital information. However, if users are not co-located than any technology under consideration must support the network required for collaboration (e.g., Wifi, 4G, LAN etc.)

Multiple operator input If multiple user input is required, any technology under consideration must have (a) the required inputs to support multiple input devices, or (b) support for multi-touch interactions. However, if touch is the primary interaction, it may be more practical for each user involved to have a dedicated, touch capable device with a shared screen or application.

Multiple operator viewing Some tasks may require multiple operators to view the same information on a shared display. For single operator displays the minimum recommended viewing angle is 40 degrees (ISO, 1992) meaning any display technology under consideration to support multiple viewers must have a viewing angle large enough to support multiple operators allowing each to view text subtending a viewing angle of at least 15 arc-min (MIL-STD-1472F, 1992).

Task involves the need to privately view information Information to be privately viewed by one individual may be presented on a small, portable display as long as text subtends a vertical visual angle of at least 15 arc-min (MIL-STD-1472F, 1999).

E.2 Information and Data Description

Task involves receiving information Tasks involve receiving information require technologies capable of updating via a wired connection or a wireless connection when appropriate.

Task involves Unclassified, Sensitive or Classified information The classification or information may change from task to task and the security of any technology under consideration must be assessed.

Task involves audio information/hearing Tasks involving audio information require technologies that have built in speakers or a means for connecting headphones either through a wired connection or wirelessly.

Task involves aural alerts Any technology under consideration that is required to provide aural alerts should be capable of producing audio at volumes 10dB above background and within a frequency range of 200 – 5000 Hz (MIL-STD-1472F, 1999).

When considering portable or totable technologies it is worth noting that alerts in the range of 1500 – 3000 Hz can be difficult to localize and should be avoided (Sanders & McCormick, 1993).

Task involves simple tones or complex verbal messages If complex verbal messages are to be presented, wearable or portable devices should be avoided because of their smaller, lower quality speakers as well as their potential to be out of range of the user (e.g., in pocket or carrying case).

If a task requires complex verbal messages (rather than simple tones) technologies capable of pause and play-back of audio should be considered as operators are limited in the amount of information they can remember in a given voice communication (Prinzo & Morrow, 2009).

Task involves adjusting the volume of audio Tasks completed in dynamic noise environments require a means for easily adjusting the volume on any technology under consideration.






Task involves private audio for a single user If task requires private audio for a single user, then any technology under consideration should support the use of a headset. However, if a headset is not an option, audio playback above 1000Hz is less intelligible at a distance (MIL-STD-1472F, 1999); therefore, higher frequency playback along with physical distance may be sufficient for privacy.

Task involves listing to multiple audio sources Some tasks may require users to monitor multiple audio sources. Humans are often unable to process the meaning of multiple audio sources simultaneously (dichotic listening). To increase the users’ ability to monitor the content of each source accurately, the two audio sources should vary in frequency and location of presentation (Cherry, 1953).

Task involves visual information/vision Tasks requiring visual information will require technologies with some form of display. Generally, no display should have an apparent flicker to at least 90% of a sample population of users, which translates into a minimum refresh rate of approximately 70 Hz (Wilkins, Veitch, & Lehman, 2010). Image formation time should not exceed 10ms to avoid motion artifacts (BSR/HFES, 2002).

Task involves viewing textual information All text must subtend a vertical visual angle of at least 15 arc-min (MIL-STD-1472F, 1999). This means that any technology considered must be capable of displaying a reasonable amount of text at or above this visual angle. Depending on the nature of the task, a smaller display should be considered only if a minimal amount of text is required (e.g., notifications rather than lengthy descriptions.)

Any display used for presenting text should be capable of producing a minimum luminance ratio of 5:1 between text and background (MIL-STD-1472F, 1999.)

Task involves viewing minimal, moderate, or large amounts of text The amount of text to be presented, combined with the above minimum text size will dictate the size of the display required - tasks with large amounts of text will benefit from larger screens.

When large amounts of text must be read (e.g., reading previous logs or manuals) E-ink technologies should be considered over LED or LCD displays. Reading speed and comprehension on E-ink devices is indistinguishable from paper reading and leads to less eye fatigue than on LED/LCD screens (Siegenthaler, et al., 2011).

Task requires viewing graphical information Technologies in this section should be investigated based on resolution, screen size, dynamic and static contrast ratios, display type, colour depth, and brightness.

Task involves viewing minimal, moderate, or large amounts of graphics Display size will limit the amount of graphical information that can be displayed. Environmental motion must also be considered as small displays become increasingly less effective in increasingly moving environments.

Task involves viewing simple or complex graphics Simple graphics can be drawn on a range of displays from e-ink to LED. However, high resolution LED or LCD displays are better suited for drawing graphics that are more complex.

The visual quality required for the task is low or high According to Gong et al., (2013), perceptually, the most important factors that constitute image quality on LED/LCD screens are resolution and colour accuracy – these should be considered when the task require high quality visuals.

Task involves coloured information Any technology under consideration for displaying information that involves colour should have a colour display, although some monochrome displays can render colour information into gray scales. Additionally, higher colour gamut and colour depth can increase the differentiability of objects and data in search tasks as well as adding additional means of spectrally separating unrelated data.






Task requires storage of text information Technologies under consideration must have some amount of internal storage available for storing and retrieval of text information.

Task involves the ability to stream/play video Displays used to play video should have a refresh rate of at least 60 Hz (Cardosi & Murphy, 1995).

Information is changing rapidly With the exception of display types like e-ink and holographs, display frame rate would not be a limiting factor for the ability to update information – some high-end consumer displays can refresh at 120 Hz, well above visual flicker in most cases (Wilkins, Veitch, & Lehman, 2010). The flow of information through communications channels may be a more likely limitation for displaying dynamic imagery or information flow, so the bandwidth (bit-rate) of the communications technology should be considered in conjunction with the information density flow rate.

Information must be quickly accessible Handheld, portable and wearable technology provide a means for accessing minimal to moderate amounts of information extremely quickly through the use of notifications and quickly accessible, intuitive touch menus (Hinkley & Wigdor, 2012).

Task must have 100% up time Devices requiring battery should not be considered if this is true for a given task. In cases where a battery is required, the behaviour of the device as the battery becomes depleted must be well understood such that information for the ongoing task is not lost (Lamoureux & Banbury, 2011).

Task requires hands-free operation If hands-free operation is to be achieved by means of voice commands, an alternative input device should be available for cases where voice input is not effective (e.g. noisy environments.) Also, no-voice commands should be mandatory for tasks that cannot be easily un-done or may compromise safety (MIL-STD-1472F, 1999).

E.3 Actions and Information Manipulation

Task involves user input or information manipulation For tasks requiring user input or information manipulation, any technology under consideration must possess an input device (e.g., keyboard or trackball) or some other means of inputting information (e.g., touch screen or voice command).

Task involves text input For tasks involving text input a keyboard must be available or, for smaller amounts of text, a means for voice-to-text input.

Speech When voice-to-text is used an alternative input device should be available for cases where voice input is not successful. Any voice-to-text input should be kept to a minimum particularly in loud environments (MIL-STD-1472F, 1999).

Task involves typing For any keyboard used for text communication the minimum horizontal width of a key should be 12 mm (ANSI, 1988), however, when environmental motion is present this minimum can increase to as great at 30mm especially when considering virtual, on-screen keyboards.

Task involves typing minimal, moderate, or large amounts of text For tasks requiring minimal to moderate amounts of typing, an on-screen keyboard may be sufficient. QWERTY keyboards are still the most appropriate for large amounts text input because of prior learning. Standard keyboards allow procedural memory to accomplish the typing task, which leads to minimal attention requirements (Hinkley & Wigdor, 2012).






Two thumb QWERTY keyboards offer data entry rates around 60 words per minute making them a better choice than one-handed keyboards (Mackenzie and Soukoreff, 2002). If touch screen keyboards are to be used, they should be on screens large enough to have adequately sized and spaced buttons at least 18mm apart (ANSI, 1988).

Task involves editing or annotating existing messages Technologies must have a means for selecting text, either through touch, keyboard or cursor control device. The amount of text to be edited should be considered. If a large amount of text is to be edited, it may be beneficial to use a physical keyboard instead of an on-screen, touch keyboard.

Task involves selecting items on a screen A positive indication of touch activation should be provided to acknowledge the system response to the control action (MIL-STD-1472D, 1989).

Touch screens afford faster single-item selection than mouse or trackball. Direct input offered by touch screens is less cognitively demanding to use (Ng, Tao & Or, 2013).

Touchscreens provide the fastest means for selection of single items but the slowest means (compared to mouse/trackball) for dragging actions (Cockburn, Ahlstrom & Gutwin, 2012).

Task involves zooming in/out on graphics The following four task elements can be accomplished using a trackball, mouse or touch input methods. On small screens where minimal input is required, touch is sufficient; however, on larger screens touch input can quickly lead to wrist and arm fatigue (Farhadi-Niaki, Etemad & Arya. 2013).

Task involves scrolling through text The following four task elements can be accomplished using a trackball, mouse or touch input methods. On small screens where minimal input is required, touch is sufficient; however, on larger screens touch input can quickly lead to wrist and arm fatigue (Farhadi-Niaki, Etemad & Arya. 2013).

Task involves panning across images The following four task elements can be accomplished using a trackball, mouse or touch input methods. On small screens where minimal input is required, touch is sufficient; however, on larger screens touch input can quickly lead to wrist and arm fatigue (Farhadi-Niaki, Etemad & Arya. 2013).

Task involves continuous inputs (sliders or dials) The following four task elements can be accomplished using a trackball, mouse or touch input methods. On small screens where minimal input is required, touch is sufficient; however, on larger screens touch input can quickly lead to wrist and arm fatigue (Farhadi-Niaki, Etemad & Arya. 2013).

Task involves a high level of input precision Touch screen and mouse input produce fewer selection errors than a trackball in a stationary environment (Ng, Tao & Or, 2013). However, in a moving environment, the error rate on touch screens can increase. One way to mitigate this is to provide tactile or audio feedback for touch screen buttons (Hinkley & Wigdor, 2012; O’Brien, Rogers & Fisk, 2008).

Errors are undesirable or unacceptable Touch screen and mouse input produce fewer selection errors than a trackball in a stationary environment (Ng, Tao & Or, 2013). However, in a moving environment, the error rate on touch screens can increase. One way to mitigate this is to provide tactile or audio feedback for touch screen buttons (Hinkley & Wigdor, 2012; O’Brien, Rogers & Fisk, 2008).

Task needs fast recovery from errors Touch screen and mouse input produce fewer selection errors than a trackball in a stationary environment (Ng, Tao & Or, 2013). However, in a moving environment, the error rate on touch screens can increase. One way to mitigate this is to provide tactile or audio feedback for touch screen buttons (Hinkley & Wigdor, 2012; O’Brien, Rogers & Fisk, 2008).






Task involves following checklists Technology displaying checklist should present text in a clear, easily readable fashion. Because checklists should not require text editing or input e-ink devices would allow converting manual paper lists to electronic checklists.

When large amounts of text must be read (e.g., reading previous logs) E-ink technologies should be considered over LED or LCD displays. Reading speed and comprehension on E-ink devices is indistinguishable from paper reading and leads to lower eye fatigue than with LED/LCD screens (Siegenthaler, et al., 2011).

Task error tolerance is low, medium or high See 3.3.3.3.1.8.

Task involves moving between work stations Two scenarios are identified in this section 1) moving between workstations with the ability to see data at both stations, and 2) moving between stations and bringing information with you. The first scenario would require a display capable of presenting information at a size, resolution, contrast ratio and brightness that is usable to operators at a reasonable distance from their workstation.

If information portability is required, devices that are portable or wearable should be considered. Smartphones and tablets may be useful when up-to-date information is required in a timely manner but the risk/reward benefit of handheld device batteries is a consideration. Also, presentation of large amounts of information should be avoided on small screens such as typically found in smartphones (Hinkley & Wigdor, 2012).

Task is completed rarely, occasionally or often Tasks frequently should have less skill fade associated with them than tasks that are completed rarely. This is important to consider when selecting novel input technologies. Technologies selected for tasks that are completed rarely should be intuitive to most users – the complexity of the technology should not interfere with completion of the task. However, for frequent tasks, often the technology learning curve can be greater because of the frequent interaction; however, this can lead to errors and user frustration so there should be significant benefits that accrue from the more complex technology.

E.4 Decision Making and SA

Task involves data monitoring For tasks that require focused monitoring of data, technologies should be selected based on their ability to make relevant data easily accusable while avoiding presentation of irrelevant data, i.e., clutter (Moacdieh, Prinet & Sarter, 2013).

When operators are monitoring dynamic data, there should be a means for presenting this data in graphical form to support the operator’s ability to perceive changes more easily (Smith & Mosier, 1986). From a technology stand point, this means that any technology under consideration for data monitoring tasks should have a means for displaying graphical images as opposed to text only.

The density of information in this task is Low, Med or High Any display technology under consideration should be capable of presenting all relevant information simultaneously while avoiding clutter and optimizing the accessibility of required information. From a technological standpoint, some possible ways to avoid clutter when displaying high density information is to reduce the global density on the screen (larger display), decrease the similarity of items on the screen (large color pallet, high contrast or brightness) and increase the clarity of information (higher display resolution) (Kaber, et al., 2008.)

Beyond these hardware characteristics, the majority of the considerations related to information density should be addressed at the software and operational level i.e., information density should be minimized by presenting only information that is essential to the operator at any given time. In addition, the hardware limitations of any given technology should not require a ratio of characters to blank spaces greater than 60% in order to effectively present all required information (DOE HFDG ATCCS V2.0, 1992.)






The use of colour on displays with a high information density is beneficial when the colour of specific targets is known; however, for search tasks where the colour of the target(s) is not known, the use of colour may become distracting and increase search time (National Air Traffic Services, 1999.)

Task involves evaluation of tactical situation Technologies should have an appropriate means of interfacing with the necessary ship systems and sensors.

Task involves managing operators If the technologies under consideration for this task description are secondary to the managing task, they should be unobtrusive (i.e., not virtual reality) as to allow monitoring of others. Portable or wearable communication technologies should be considered as they afford the ability to communicate with others while allowing freedom of movement through the ship.

Observe individuals Technologies under consideration should not interfere with the users’ vision to observe other aspects of the environment.

Monitor conversations Technologies under consideration should not interfere with the users’ auditory senses to detect other sounds in the environment, notably alerts.

Task involves understanding remote environmental conditions Technologies should have an appropriate means of interfacing with the necessary ship systems and sensors.

Task involves a high level of concentration Certain technological characteristics should be considered when attempting to increase concentration on a specific task. However, these will be largely task and environment specific e.g., wearing headphones may increase concentration on a task requiring monitoring of audio information; however, it will reduce the operator’s ability to monitor audio information in their surroundings.

For tasks requiring concentration on auditory information, efforts should be made to reduce environmental noise, specifically other speech noise. Humans are fundamentally unable to process two streams of speech at the same time (Cherry, 1953) so the presence of speech unrelated to the primary task has the potential to reduce an operator’s concentration/focus on the primary task. Therefore, technologies with headphone compatibility are favored over open-air speakers.

Task involves multi-tasking If the technologies under consideration is required during concurrent task execution (multi-tasking), the technology should be unobtrusive to minimize interference between tasks (i.e. not a VR HMD). Technologies that rely heavily on one sense should be avoided (i.e. smart earphones). Ideally, the mode of input (sense modality) related to the technology interaction should be different for each of the technologies being used.

E.5 Task Environment and Operator Clothing

Ambient working conditions must be considered The following subsections describe the human factors considerations when considering environmental factors.

Task may be completed in direct external light (glare) When glare is possible, figures and text should be presented in black or dark colours on a light background to limit effects of glare (DOE HFDG ATCCS V2.0 1992).






The magnitude of glare varies by the luminance, location, size of the light source and the luminance the operator’s eyes are currently adapted to. The most significant issue with glare is the user’s reflexive change in attention from information on a display to the glare itself. Glare can be reduced at a technology level by increased display contrast, the ability to change the position of the display and the application of anti-glare display covers (some technologies surface feature this built-in) (Thomson & Wells, 2013).

Task is completed in variable environmental lighting conditions Displays must be readable under all task-related environmental conditions (Lamoureux & Banbury, 2011).

When environmental light is variable, technologies under consideration should have a high contrast ratio (to combat glare) and a high brightness range (to accommodate for light/dark adaptation but the user). Also, display intensity should be an easy-to-use control that provides multiple steps or continuously variable brightness and contrast control (DOD, 1999).

In situations where operators’ eyes are required to be adapted to dark environments but they still need to monitor bright displays, technologies with the ability to display light further in the red spectrum should be considered (Mantiuk, Rempel & Heidrich, 2009) as they reduce light adaptation when switching from the display to the external environment.

Ambient noise environment is low, high or variable In environments with ambient noise, any audio information or feedback must be 10 dB louder than background to reliably detect and recognize the signal. In environments where the ambient noise prevents effective listening (greater than 90 dB,) audition should not be the primary source of information presentation (MIL STD 1472F, 1999).

All speech volume from any technology should have a signal to noise ratio of at least 5:1 (MIL-HDBK-761A, 1989).

Headphones should not be used in any environment with ambient noise below 85 dB if that environment contains sounds that provide the user with useful information and that information cannot be directed to the user's headset (MIL STD 1472F, 1999).

Task must not generate unnecessary noise If the tasks require audio, any technologies under consideration should have a headphone jack available or, minimally, have an easily accessible volume control to reduce interference with other activities.

Operating temperature range The operating specification for any technology under consideration should allow full functionality at any temperature throughout the range of expected operating temperatures.

Task is subject to external motion Lin et al. (2010) measured touch screen, mouse, and trackball input performance in a stationary versus vibrating environment (vibrations matched those of a 150-ton ship in a variety of tidal conditions) created using a 6-DOF Stewart platform. The most accurate means of input in the static environment was the touch screen while the most accurate means of input in the vibrating environment was the mouse (trackball was the least accurate in both).

A study conducted by Yau, Chao, and Hwang (2008) in which participants input accuracy using touch screens verses tack balls was recorded in a moving environment (this time in the pitch/roll direction) revealed that trackball input was the most accurate method.

Eye-based controls (eye tracking) shall not be used in any vibrating environments (MIL-STD-1472G, 2012).

Task puts any technology at risk of impact The materials used in construction of the technology should be considered when it is to be used in environments that put the technology at a high risk being dropped or struck. Technologies that have a readily available means for ruggedizing are preferred.






Task has a risk of exposure to liquids The potential for contact with liquids should be considered for each technology and they should have the appropriate ingress protection (IP) rating.

Task involves PPE Some tasks require that people wear Personal Protective Equipment (PPE) such as gloves, respirators, visors, etc. while in hazardous environments (Lamoureux & Banbury, 2011). If a device must be used while wearing PPE, some technologies may not function adequately. For instance, a device with a touch screen keyboard intended for use by a user wearing gloves should have buttons of a preferred dimension: 18mm x 18mm with a maximum separation of 11mm; other touch screen buttons (e.g. field selection, device control, etc.) should be no smaller than 20mm x 20mm with a minimum separation of 8mm (MIL-STD-1472G, 2012).

Connectivity available in environment Technologies should have an appropriate means of connecting with the necessary ship systems and sensors.

Task has space restrictions The available space should be considered when selection technologies.




Documents

Framework for Selecting Appropriate Interface Technologies