IMSE: 8810 Human Factor Instructor: Dr. Linsey Baker Student:
Qiuyue Zhao
Slide 2
Outlines 1. Introduction of usability What is usability?
Usability in mobile phone design Why is usability important? How do
we do usability? 2. Discussion of papers Paper 1: A framework for
evaluating the usability of mobile phones Paper 2: Systematic
evaluation methodology for cell phone user interfaces Paper 3:
Chinese character entry for mobile phones 3. Final conclusion
Slide 3
Usability: What is it?
Slide 4
Usability is a measure of quality Its defined by six quality
components: Effectiveness Learnability Efficiency Memorability
Error Prevention Satisfaction Jakob Nielsen Ben Shneiderman
Slide 5
Main mobile problems Three features or limitations of mobile
phone Small screens Awkward input, especially for typing Processing
power and available memory
Slide 6
Usability in Mobile phone design Very heavy and awkward Smaller
and more portable The first mobile phone with internal antenna Full
QWERTY keyboard Multi-touch
Slide 7
Why is usability important in Mobile phone design? Usability is
often associated with the functionalities of the product If its
difficult to use, people wont buy If users get lost, they wont buy
If its hard to read or doesnt answer users key question, they wont
buy
Slide 8
How do we do usability? Early Focus on Users and Tasks Know who
the users are Know what tasks the users will perform Know Which are
most important Empirical Measurement (Evaluation methods) Empirical
Measurement Before starting the new design, test the old design.
Test your competitors' designs Iterative Design a cyclic process of
prototyping, testing, analyzing, refining a product or process
Paper 1 :A framework for evaluating the usability of mobile
phone What is this paper about? Propose a structuralized usability
evaluation model in order to help practitioners identify and
organize critical usability problems in a systematic way,
understand the relationships among usability factors, and generate
better design ideas. Why? Pervious studies only focus on what could
constitute the usability factors, seldom organize them in a
systematically. Most concerned with software products, not mobile
phone. So do not reflect the feature of mobile phone
appropriately.
Slide 11
Something about Usability evaluation 1. Importance of usability
evaluation 2. Usability design problems: Interface features design
(task-independent) LUI: logistical user interface GUI: Graphic user
interface PUI: Physical user interface Users tasks design
(task-dependent) 3. Usability Evaluation Methods(UEM)
Slide 12
Section 1: Research method Step 1: collection and analysis of
usability problems First: classify usability problems into 2
categories Task-independent problems (interface design features)
Task-dependent problems (user tasks) Second: typically choose 28
mobile phone tasks Third: associate usability problems with mobile
phone tasks
Slide 13
Step 2: Identification of design features to be evaluated What
interface features to evaluate Step 3: Evaluation strategy How to
evaluate interface feature LUI evaluation GUI evaluation PUI
evaluation Task-based evaluation Section 1: Research method Choose
from four kinds of evaluations
Slide 14
Section 2:Proposed evaluation framework Hierarchical model of
usability factors Aim to evaluate Usability indicator Usability
criteria Usability property property
Slide 15
Usability indicators Visual support of task goals Support of
cognitive iteration Support of efficient interaction Functional
support of user needs Ergonomic support Physical aspect of
interaction Cognitive interaction affective
Slide 16
Partial list of Usability criteria
Slide 17
Criteria checklist
Slide 18
Section3:Usability evaluation framework Visual support of task
goals
Slide 19
Conclusion This study developed an evaluation framework for
supporting usability practitioners to test the suability of mobile
phones in an analytical way. A hierarchical model of usability Four
sets of checklists A quantification method An evaluation process
Limitation Mapping relationship between evaluation areas and
usability indicators and its consequent evaluation items of the
checklists. The goal-mean relationship need to be further
examined
Slide 20
Paper 2: Systematic evaluation methodology for cell phone user
interfaces Object Develop a Systematic Evaluation Methodology for
Cell Phone User interface(SEM-CPU). SEM-CPU is a specifically
designed to integrate five empirical methods into a
laboratory-based test in order to evaluate cell phone UIs.
Slide 21
Three stages comprised to construct SEM-CPU Stage 1 Stage 2
Stage 3
Slide 22
Stage1: Data collection in SEM-CPU Five method Scenario-based
task performance Benchmark tasks: tasks that reflect the realistic
context of use found outside of the laboratory Voice Activated
Dialing (VAD) Short Messaging Service(SMS) Phone Book(PB)
Questionnaires (six usability attributes: icons clarity, text
label, ease of use, ease of locating functions, quality of feed
back and overall satisfaction.) Retrospective think-aloud instead
of concurrent think aloud User observation for critical incidents
Post-task interview
Slide 23
Procedures of data collection
Slide 24
Stage 2: Data analysis Step 1: quantitative analysis
Effectiveness Efficiency Satisfaction Learnability Step 2:
qualitative data transformation Qualitative Data logging(QDL)
template step 3:navigation flow diagram analysis Step 4:verbal
protocol Task completion time(s) Error rates Task completion time
and task completion ratio at repeated trails Ratings of
satisfaction Task completion ratio(%)
Slide 25
An example of the table with the reorganized results of verbal
protocol/critical analysis
Slide 26
Step 3: Navigation flow diagram analysis No apparent disability
Cognitive disability Physical disability Advantage: can identify
the cause of usability problems
Slide 27
Stage 3: Data integration Triangulation strategy
Slide 28
Example of design specification
Slide 29
Conclusion This paper present a systematic methodology, called
SEM- CPU, to evaluation cell phone UIs in laboratory-based testing.
SEM-CPU guides usability engineers integrating five empirical
methods to discover valid usability problems, and generate proper
design specifications. Efficiently conduct laboratory-based testing
with multiple empirical methods Collect and measure necessary
usability attributes Identify determinants of usability problems
Generate proper solution Limitation Quality analysis relies heavily
on experts knowledge.
Slide 30
Paper 3: Chinese Character entry for mobile phones Object A
longitudinal experiment to evaluate character entry performance
using both objective and subjective measures for a new design and
the existing cell phone keypad.
Slide 31
Background Knowledge Chinese is an iconographic language Two
primary methods for entering Chinese Characters Pronunciation-based
methods (Pinyin) Stroke-based method Stroke: each stroke is written
with a single action Chinese Characters: the minimum functional
unit. Most characters are composed of two or more strokes.
character strokes
Slide 32
Current stroke input method A single stroke is used as the
legend for each key to represent the corresponding group of strokes
Problems: Failed to convey the necessary information to all users
to understand which strokes could be entered using each key. Select
the wrong key Separate a single stroke into two or more pieces and
attempt to enter each piece separately Try to enter two or more
strokes using a single key
Slide 33
A new stroke-based solution New: combining abstract symbols and
concrete stroke examples
Slide 34
Evaluation method Participants: born and raised in China, lived
in the US for more than 4 years. 24 volunteers(12 males, 12 females
) Tasks Text entry tasks: enter 5 sentences that were based on
headlines from a Chinese news website each of the 6 days Distracter
task Questionnaire Procedures Pre-test on the first day 6 days
survey Nine-point scale Post-test on the last day
Slide 35
Dependent variables Entry speed Character-level failures
Stroke-level accuracy Overall satisfaction Further usage
Effectiveness of two keypad design How well participants understood
the system
Slide 36
Result Entry speed increased by 32% eliminate 80%
Character-level errors for the first time users higher stroke-level
accuracy rates than the original design Overall satisfaction
increased More interested in using the new system The author uses
ANCOVA to prove the interaction between the keypad design and the
changes of variables.