Heuristic Evaluation Tutorial

1

Heuristic Evaluation

• Goal: Identifying usability problems in existing systems

• Checking compliance with a set of usability principles

• Performed individually • Results will be aggregated

2

Heuristic Evaluation

• The scope is limited to scenarios • You can go through the interface several times• You should specify: – The problem – The task– The heuristic(s)

3

IT security management (ITSM) tools

• IT security management tools are components in the design, development, and maintenance of a secure information technology infrastructure. – Examples: network firewall, intrusion detection

system, identity and access management system

• Characteristics of the ITSM environment– Complex, collaborative, people with different

backgrounds

4

Example ITSM Tool

• Firewall as a running example• Filtering packet based on certain

characteristics• Firewall use rules for this purpose– E.g. block every connection from 123.123.100.100

5

HEURISTICS

6

1 - Visibility of system status

The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.

7

1 - Visibility of system status : example

Firewall rules:Block all connections to the range(192.168.0.1 - 192.168.0.255)Allow connections to 192.168.0.1 port 22

8

Firewall Security Admin

2 - Match between system and the real world

The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. It should follow real-world conventions, making information appear in a natural and logical order.

9

2 - Match between system and the real world : example

Firewall rules:eth0 inbound on port 22 block

eth 0eth 1

10

Security Admin

3 - User control and freedom

Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

11

3 - User control and freedom : example

Firewall rules:A XB XC X

12

Firewall

4 - Consistency and standards

Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions and terminology which are familiar to the user.

13

4 - Consistency and standards : example

Firewall rules file:eth0 inbound … block

Block all incoming connections on eth 0Block all incoming connections on eth 1Block all incoming connections on eth 2

…

Firewall UI:

14

Firewall

Security Admin

5 - Error prevention

Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

15

5 - Error prevention : example

Firewall rules:Block all connections on port 22

Allow connections on port 22 for subnetwork1

Are you sure … ?

16

Security Admin

6 - Recognition rather than recall

Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

17

6 - Recognition rather than recall : example

Write the rule in the following text box:Write the rule in the following:

interface From To Port

18

Firewall Security Admin

7 - Flexibility and efficiency of use

Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

19

7 - Flexibility and efficiency of use : example

Write the rule in the following text box:

Write the rule in the following:

interface From To Port

20

Security Admin

8 - Aesthetic and minimalist design

Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

21

8 - Aesthetic and minimalist design : example

22

Security Admin

9 - Help users recognize, diagnose, and recover from errors

Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

23

9 - Help users recognize, diagnose, and recover from errors : example

Allow connections on port 22 for subnetwork1

• Error• Error 22• Error – rule can’t be added

• Error – Rule is in conflict with the rule number 34• Error – The port number is not valid• Error – The port number should between 1~1024

24

Firewall Security Admin

10 - Help and documentation

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.

25

10 - Help and documentation : example

Write the rule in the following text box:Write the rule in the following:

interface From To Port ?

26

Security adminFirewall

Firewall UI

IDENTITY MANAGEMENT SYSTEM The target system for heuristic evaluation

27

Identity management system

Neteauto

Sales rep. 28

Sour

ce: I

denti

ty L

ifecy

cle

Man

agem

ent F

lash

Dem

o (h

ttp:

//w

ww

.ca.

com

/us/

iden

tity-

man

agem

ent.a

spx#

docu

men

ts)

Identity management system

Sales rep.

VP Sales

Security admin

Previous sales reports 29

IdM System

Identity management systemCertification Process

Notification

30

IdM System

VP Sales

Security admin

Employees

System Demo

31