HCI - Lesson 2 Interaction

07

HCI - Lesson 2

Interaction

07

Outline• What is Interaction Design

– A multidisciplinary field

• Terminology– Interaction “Metaphors” and “Paradigms”

– Ergonomics

– Interface “types”

• Interaction Models (theoretical approaches)

• Users and Stakeholders

• The Interaction Design Cycle and its role in the conventional sw process

Recap• HCI has moved beyond desktop machines

and designing interfaces for the– See “Readings” Economist Oct 8, 2011 «Special

Report: Personal Technology»– About extending and supporting all manner of

human activities in all manner of places– Facilitating user experiences through designing

interactions• Make work effective, efficient and safer• Improve and enhance learning and training• Provide enjoyable and exciting entertainment• Enhance communication and understanding• Support new forms of creativity and expression

4

What is interaction design?

• Designing interactive products to support the way people communicate and interact in their everyday and working lives

– Sharp, Rogers and Preece (2011)

• The design of spaces for human communication and interaction

– Winograd (1997)

5

Goals of interaction design

• Develop usable products– Usability means easy to learn, effective to use and

provide an enjoyable experience• Involve all stakeholders (end-users + ….) in the

design process

6

Scope of interaction design• Number of other terms used emphasizing what is

being designed, e.g.– user interface design, software design, user-centered design, product

design, web design, experience design (UX)

• Interaction design is the umbrella term covering all of these aspects– fundamental to all disciplines, fields, and approaches concerned with

researching and designing computer-based systems for people

7

HCI and interaction design

8

Relationship between ID, HCI and other fields

• Academic disciplines contributing to ID: – Psychology– Social Sciences– Computing Sciences– Engineering– Ergonomics– Informatics

9


• Design practices contributing to ID:– Graphic design– Product design– Artist-design– Industrial design– Film industry

10


• Interdisciplinary fields in interaction design:

– HCI– Ubiquitous Computing– Human Factors– Cognitive Engineering– Cognitive Ergonomics– Computer Supported Co-operative Work– Information Systems

Working in multidisciplinary teams• Many people from different

backgrounds involved

• Different perspectives and ways of seeing and talking about things

• Benefits– more ideas and designs

generated• Disadvantages

– difficult to communicate and progress forward the designs being create

12

Interaction metaphors• Metaphor = a literary figure of speech that uses an image,

story or tangible thing to represent a less tangible thing or some intangible quality or idea; e.g., "Her eyes were glistening jewels."

• In HCI: – helpful to conceptualize what we are doing with an interactive system,

e.g. “surfing the web”– A “conceptual model” instantiated at the interface, e.g. the desktop

metaphor

13

Interaction metaphors

• Interaction designed to be similar to the one with physical entity but also has own properties– e.g. desktop metaphor

• Can be based on “activities” (what you can DO with the systems), “objects” (what you manage during an activity) or a combination of both

• Exploit user’s familiar knowledge, helping them to understand ‘the unfamiliar’

• Conjures up the essence of the unfamiliar activity, enabling users to leverage of this to understand more aspects of the unfamiliar functionality

14

Benefits of interaction metaphors

• Makes learning new systems easier• Helps users understand the underlying

conceptual model• Can be very innovative and enable the realm

of computers and their applications to be made more accessible to a greater diversity of users

15

Problems with interaction metaphors

• Break conventional and cultural rules– e.g. recycle bin placed on desktop

• Can constrain designers in the way they conceptualize a problem space

• Forces users to only understand the system in terms of the metaphor

• Designers can inadvertently use bad existing designs and transfer the bad parts over

• Limits designers’ imagination in coming up with new conceptual models

16

Different Interaction Metaphors

• Instructing– issuing commands and selecting options

• Conversing– interacting with a system as if having a conversation

• Manipulating– interacting with objects in a virtual or physical space by

manipulating them • Exploring

– moving through a virtual environment or a physical space

17

1. Instructing• Where users instruct a system and tell it what to

do– e.g. tell the time, print a file, save a file

• Very common metaphor, underlying a diversity of devices and systems– e.g. word processors, VCRs, vending machines

• Main benefit is that instructing supports quick and efficient interaction– good for repetitive kinds of actions performed on

multiple objects

18

Example

19

2. Conversing• Underlying model of having a conversation with

another human

• Range from simple voice recognition menu-driven systems to more complex ‘natural language’ dialogs

• Examples include timetables, search engines, advice-giving systems, help systems

• Also virtual agents, toys and pet robots designed to converse with you

20

Examplehttps://areaclienti187.telecomitalia.it/auth/registrautente.do?access=portal

21

Pros and cons of conversational metaphor

• Allows users, especially novices and technophobes, to interact with the system in a way that is familiar– makes them feel comfortable, at ease and less scared

• Misunderstandings can arise when the system does not know how to parse what the user says

22

3. Manipulating• Involves dragging, selecting, opening, closing and zooming

actions on virtual objects

• Exploit’s users’ knowledge of how they move and manipulate in the physical world

• Can involve actions using physical controllers (e.g. Wii) or air gestures (e.g. Kinect) to control the movements of an on screen avatar

• Tagged physical objects (e.g. balls) that are manipulated in a physical world result in physical/digital events (e.g. animation)

23

Direct Manipulation• Shneiderman (1983) coined the term DM, came from

his fascination with computer games at the time

– Continuous representation of objects and actions of interest

– Physical actions and button pressing instead of issuing commands with complex syntax

– Rapid reversible actions with immediate feedback on object of interest

24

Why is DM metaphor enjoyable?

• Novices can learn the basic functionality quickly• Experienced users can work extremely rapidly to carry

out a wide range of tasks, even defining new functions

• Intermittent users can retain operational concepts over time

• Error messages rarely needed• Users can immediately see if their actions are

furthering their goals and if not do something else• Users experience less anxiety• Users gain confidence and mastery and feel in control

25

What are the disadvantages with DM?

• Some people take the metaphor of direct manipulation too literally

• Not all tasks can be described by objects and not all actions can be done directly

• Some tasks are better achieved through delegating– e.g. spell checking

• Moving a mouse around the screen can be slower than pressing function keys to do same actions

26

4. Exploring• Involves users moving

through virtual or physical environments– E.g., browsing the web– Moving in a 3D virtual

space

• Physical environments with sensing technologies, e.g., – Context aware

computing– Wearable computing

27

Which metaphor is best?• Direct manipulation is good for ‘doing’ types of tasks,

e.g. designing, drawing, flying, driving, sizing windows

• Issuing instructions is good for repetitive tasks, e.g. spell-checking, file management

• Having a conversation is good for children, computer-phobic, disabled users and specialised applications (e.g. phone services)

• Hybrid conceptual models are often employed, where different ways of carrying out the same actions is supported at the interface - but can take longer to learn

28

Further Terminology: “Paradigm”

• General approach adopted by a community for carrying out research – shared assumptions, concepts, values, and

practices – e.g. desktop, ubiquitous computing, in the wild

29

Examples of new paradigms

• Ubiquitous computing (mother of them all)• Pervasive computing• Wearable computing• Tangible bits, augmented reality• Attentive environments• Transparent computing

– and many more….

Further terminology: Ergonomicsthe study of designing equipment and devices that fit the human body, its movements, and its cognitive abilities.

Ergonomics - examples• arrangement of controls and displays

e.g. controls grouped according to function or frequency of use, or sequentially

• surrounding environmente.g. seating arrangements adaptable to cope with all sizes of user

• health issuese.g. physical position, environmental conditions (temperature,

humidity), lighting, noise,• use of colour

e.g. use of red for warning, green for okay,awareness of colour-blindness etc.

Ergonomics and Interaction Design

• Ergonomics: Study of the physical characteristics of interaction

• Also known as human factors – but this can also be used to mean much of HCI!

• Ergonomics good at defining standards and guidelines for constraining the way we design certain aspects of systems

From Interaction to Interface

34

From interaction to interface

• Interaction metaphor: – what the user is doing when interacting with a system, e.g.

instructing, talking, browsing or other• Interface type:

– the kind of interface used to support the metaphor, e.g. speech, menu-based, gesture

35

Many kinds of interface types available…

• Command line• menus• speech and natural language• question/answer and query dialogue• data-entry & form-fills • WIMP• Three–dimensional interfaces• Multitouch ((see dedicated lesson)• Gesture (see dedicated lesson)• Tangible (see dedicated lesson)• Augmented reality• …

Command line interface• Way of expressing instructions to the computer

directly– function keys, single characters, short abbreviations, whole

words, or a combination

• suitable for repetitive tasks• better for expert users than novices• offers direct access to system functionality• command names/abbreviations should be

meaningful!

Typical example: the Unix system36

Menus• Set of options displayed on the screen• Options visible

– less recall - easier to use– rely on recognition so names should be meaningful

• Selection by: – numbers, letters, arrow keys, mouse– combination (e.g. mouse plus accelerators)

• Often options hierarchically grouped– sensible grouping is needed

• Restricted form of full WIMP system (see next slides)

37

Speech & Natural language• Speech recognition or typed natural language • Familiar to user• Rapidly improving …

… but still inaccurate

• Problems (see phone services!)– vague– ambiguous– hard to do well!

• Solutions– try to understand only a subset– pick on key words – Provide feedbacks

e.g. airline reservation:reliable “yes” and “no”+ system reflects back its understanding“you want a ticket from New York to Boston?”

38

Error and repair (in speech interaction, and in general)

can’t always avoid errors …… but we can put them right

make it easy to detect errors… then the user can repair them

hello, this is the Go Faster booking systemwhat would you like?(user) I want to fly from New York to Londonyou want a ticket from New York to Boston(user) nosorry, please confirm one at a timedo you want to fly from New York(user) yes… … …

39

Q&A and Query interfaces

• Question/answer interfaces– user led through interaction via series of questions– suitable for novice users but restricted functionality– often used in information systems

• Query languages (e.g. SQL)– used to retrieve information from database– requires understanding of database structure and

language syntax, hence requires some expertise

40

Form-fills• Primarily for data entry or data retrieval• Screen like paper form.• Data put in relevant place• Requires

– good design– obvious correction

facilities

41

Spreadsheets

• first spreadsheet VISICALC, followed by Lotus 1-2-3MS Excel most common today

• sophisticated variation of form-filling.– grid of cells contain a value or a formula– formula can involve values of other cells

e.g. sum of all cells in this column

– user can enter and alter data spreadsheet maintains consistency

42

WIMP Interface Windows Icons Menus Pointers

… or windows, icons, mice, and pull-down menus!

• default style for majority of interactive computer systems, especially PCs and desktop machines

• Point&Click mechanism– just click something!

• icons, text links or location on map

– minimal typing43

elements of the WIMP interface

windows, icons, menus, pointers+++

buttons, toolbars, palettes, dialog boxes

also see supplementary materialon choosing wimp elements44

WIMP: Windows• Areas of the screen that behave as if they were

independent– can contain text or graphics– can be moved or resized– can overlap and obscure each other, or can be laid out

next to one another (tiled)

• scrollbars– allow the user to move the contents of the window up and

down or from side to side• title bars

– describe the interaction context (e.g., the name of the window)45

WIMP: Icons

• small picture or image• represents some object in the interface

– often a window or action• windows can be closed down (iconised)

– small representation fi many accessible windows• icons can be many and various

– highly stylized– realistic representations.

46

WIMP: Pointers• important component

– WIMP style relies on pointing and selecting things• uses mouse, trackpad, joystick, trackball, cursor keys or

keyboard shortcuts• wide variety of graphical images

47

WIMP: Menus• Choice of operations or services offered on the screen• Required option selected with pointer

problem – take a lot of screen spacesolution – pop-up: menu appears when needed

File Edit Options Typewriter Screen Times

Font

48

WIMP: Kinds of Menus • Menu Bar at top of screen (normally), menu drags

down– pull-down menu - mouse hold and drag down menu– drop-down menu - mouse click reveals menu– fall-down menus - mouse just moves over bar!

• Contextual menu appears where you are– pop-up menus - actions for selected object– pie menus - arranged in a circle

• easier to select item (larger target area)• quicker (same distance to any option)

… but not widely used!

49

WIMP: Menus extras

• Cascading menus– hierarchical menu structure– menu selection opens new menu– and so in ad infinitum

• Keyboard accelerators– key combinations - same effect as menu item– two kinds

• active when menu open – usually first letter• active when menu closed – usually Ctrl + letter

usually different !!!

50

WIMP: Menus design issues

• which kind to use

• what to include in menus at all

• words to use (action or description)

• how to group items

• choice of keyboard accelerators

51

WIMP: Buttons

• individual and isolated regions within a display that can be selected to invoke an action

• Special kinds– radio buttons

– set of mutually exclusive choices– check boxes

– set of non-exclusive choices

52

WIMP: Toolbars

• long lines of icons …… but what do they do?

• fast access to common actions

• often customizable:– choose which toolbars to see– choose what options are on it

53

WIMP: Palettes and tear-off menus• Problem

menu not there when you want it

• Solutionpalettes – little windows of actions– shown/hidden via menu option

e.g. available shapes in drawing package tear-off and pin-up menus

– menu ‘tears off’ to become palette

54

WIMP: Dialogue boxes

• information windows that pop up to inform of an important event or request information.

e.g: when saving a file, a dialogue box is displayed to allow the user to specify the filename and location. Once the file is saved, the box disappears.

55

WIMP: Look and … feel• WIMP systems have the same elements:

windows, icons., menus, pointers, buttons, etc.

• but different window systems… behave differently

e.g. MacOS vs Windows menus

appearance + behaviour = look and feel

56

Beyond WIMP: Three dimensional interfaces

• 3D sometimes used in ‘ordinary’ WIMP systems– highlighting– visual affordance– indiscriminate use

just confusing!• More frequently: 3D workspaces

– use for extra virtual space– light and occlusion give depth– distance effects

flat buttons …

… or sculptured

click me!

57

Beyond WIMP • Multitouch, Gesture, Tangible: see dedicated lessons• Augmented reality and wearable interaction• And many others…

59

Which interaction and interface type to choose?

• Need to determine requirements and user needs

• Take budget and other constraints into account

• Also will depend on suitability of technology for activity being supported

“Models” of interaction: theoretical approaches

60

Norman’s execution/evaluation loop

• user establishes the goal• formulates intention• specifies actions at interface• executes action• perceives system state• interprets system state• evaluates system state with respect to goal

system

evaluationexecution

goal

61

execution/evaluation loop


system

evaluationexecution

goal

62


• user establishes the goal

• formulates intention• specifies actions at interface• executes action• perceives system state• interprets system state• evaluates system state with respect to goal

system

evaluationexecution

goal

63



system

evaluationexecution

goal

64

Using Norman’s model as an interpretative tool

Some systems are harder to use than others

Gulf of Executionuser’s formulation of actions

≠ actions allowed by the system

Gulf of Evaluationuser’s expectation of changed system state

≠ actual presentation of this state

65

Human error - slips and mistakes

Slip (=svista) understand system and goal correct formulation of action incorrect action

mistake may not even have right goal!

Fixing things?slip – better interface designmistake – better understanding of system

66

Abowd & Beale’s modelextension of Norman…their interaction framework has 4 parts

– user– input– system– output

each has its own unique language

interaction translation between languages

problems in interaction = problems in translation

Score

Utask

Ooutput

Iinput

67

Using Abowd & Beale’s modeluser intentions

translated into actions at the interface translated into alterations of system state reflected in the output display interpreted by the user

general framework for understanding interaction– not restricted to electronic computer systems– identifies all major components involved in interaction– allows comparative assessment of systems– an abstraction

68

Recap

• Terminology– Interaction Design– Ergonomics– Interaction metaphor– Interface types– Cognitive models of interaction (Norman’s, Abow

& Beale’s)

69

70

The Interaction Design ProcessOUTLINE• Phases• What is involved in Interaction Design?

– Importance of involving users– Degrees of user involvement– What is a user-centered approach?

• Some practical issues– Who are the users?– What are ‘needs’?– Where do alternatives come from?– How do you choose among alternatives?

• The ID process in the sw development life cycle

71

Interaction Design PROCESS:Four basic activities

1. Establishing requirements

2. Designing alternatives

3. Prototyping

4. Evaluating71

72

Iterative Activities:The interaction design lifecycle model

Design

Requirements Management

Prototyping

Evaluation

73

Interaction Design is a user-centered approach

based on:– Early focus on users, tasks, and context of use: directly studying

cognitive, behavioral, cultural, attitudinal characteristics of the

people who (will) use the product and their context of use – Empirical observation: users’ reactions and performance to

scenarios, manuals, simulations & prototypes are observed, recorded and analysed

– Iterative process: when problems are found, fix them and carry out more tests

74

Some practical issues• Who are the users?• What do we mean by ‘needs’?• How to generate alternatives

• How to choose among alternatives

• How to integrate interaction design activities with other models?

75

Who are the “users”?• Not as obvious as you think:

– those who interact directly with the product– those who manage direct users– those who receive output from the product – those who make the purchasing decision – those who use competitor’s products

• Three categories of user (Eason, 1987): – primary: frequent hands-on– secondary: occasional or via someone else– tertiary: affected by its introduction, or will influence its

purchase

Improving our terminology

Users= end users of the product (primary or secondary users)

Stakeholders= all kind of users (primary or secondary or tertiary): Any subject how is interested to the interactive product

76

77

Check-out operators

CustomersManagers and owners

• Suppliers• Local shop owners

Who are the stakeholders?

The programmers

78

Importance of involving stakeholders

• Expectation management – Realistic expectations – No surprises, no disappointments– Timely training– Communication, but no hype

• Ownership – Make the stakeholders active participants– More likely to forgive or accept problems– Can make a big difference to acceptance and

success of product

79

Degrees of stakeholders involvement• Member of the design team

– When? • During requirements management• During design and prototyping• During pre-release evaluation

– For how long?• Full time (constant input)• Part time (patchy input)• Short term : in specific phases only• Long term (along the whole design process, and after – see below)

• Involvement after product is released– Evaluation, maintenance and update

• Combination of these approaches

80

What do we mean by ‘needs’?• Stakeholders rarely know what is possible• Stakeholders can’t tell you what they ‘need’ to

help them achieve their goals • Instead, look at existing tasks:

– their context– what information do they require?– who collaborates to achieve the task?– why is the task achieved the way it is?

• Envisioned tasks:– can be rooted in existing behaviour– can be described as future scenarios

81

How to generate alternatives

• Humans stick to what they know works• But considering alternatives is important to ‘break

out of the box’• Designers are trained to consider alternatives,

software people generally are not• How do you generate alternatives?

—‘Flair and creativity’: research and synthesis —Seek inspiration: look at similar products or

look at very different products

82

How to choose among alternatives

• Evaluation with users or with peers, e.g. prototypes• Technical feasibility: some not possible• Quality thresholds: Usability goals lead to usability criteria set

early on and check regularly—safety: how safe?—utility: which functions are superfluous? —effectiveness: appropriate support? task coverage, information

available—efficiency: performance measurements

83

Testing prototypes to choose among alternatives

84

How to integrate interaction design in other models (e.g. for sw development)

• Software engineering is the discipline for understanding the software design process, or life cycle

• Interaction design occurs at many stages of the life cycle, not as a single isolated activity

Sw process models: The waterfall model

Requirementsspecification

Architecturaldesign

Detaileddesign

Coding andunit testing

Integrationand testing

Operation andmaintenance

85

Activities in the sw life cycleRequirements specification

designer and customer try capture what the system is expected to provide can be expressed in natural language or more precise languages, such as a task analysis would provide

Architectural designhigh-level description of how the system will provide the services required factor system into major components of the system and how they are interrelated needs to satisfy both functional and nonfunctional requirements

Detailed designrefinement of architectural components and interrelations to identify modules to be implemented separately the refinement is governed by the non-functional requirements

86

The life cycle for interactive systems

cannot assume a linearsequence of activities

as in the waterfall model

lots of feedback!


Architecturaldesign

Detaileddesign




87

Sw process: Bohem’s model

cctr.umkc.edu/~kennethjuwng/spiral.htm 88

Where and How Interaction Design plays?


Architecturaldesign

Detaileddesign




A new, complementary perspective in a number of phases

Design & Implementation: focus on interface and interaction mechanisms

Requirements: focus on all stakeholders and their needInteraction

and Interface Design

Interaction and Interface

Implementation

Testing: focus on features directly perceived by end-users

89

90

Recap: Interaction Design Process is

• a goal-directed problem solving activity informed by intended use, target domain, materials, cost, and feasibility– a creative activity– a decision-making activity to balance trade-offs

• a user centered process

Documents

HCI - Lesson 2 Interaction