1

Product Development & User Product Development & User TestingTesting

Dr. Minjuan Wang

Department of Educational Technology, SDSUbelleminjuan@gmail.com

ADDIEADDIE

Somewhat time and labor intensive

A more flexible ADDIE model?

Been accused of being too linear

SAMSAM

Start End

The Successive Approximation Model:An agile design method

Start

End

4

Summative EvaluationSummative Evaluation

Traditional Instructional Design Models

– Linear– Tasks performed in sequential fashion– Summative evaluation (ADDIE)– Long development phases– Time consuming and expensive to make

changes

5

Formative EvaluationFormative Evaluation

Simultaneous design and development Ongoing feedback from end-users Iterative design cycle

DesignDesigncommunitycommunity

UserUser community community

Formative EvaluationFormative EvaluationIterative DesignIterative Design

revisions

testingprototypeconcept

product

7

Designer BlindnessDesigner Blindness

8

Prototype TestingPrototype Testing

Main Entry: pro·to·type 1 : an original model on which something is patterned : ARCHETYPE3 : a standard or typical example4 : a first full-scale and usually functional form of a new type or

– design of a construction (an airplane)

http://www.m-w.com/cgi-bin/dictionary

9

Rapid PrototypingRapid Prototyping

Early stages

– does the product meet the users' needs and expectations?

Start with a crude prototype and refine Prototypes may be discarded rather than carried

forward as a kernel for further development.

10

Formative Evaluation may…

Be done very early in the design processwhen ~10% of project resources have

been expended Provide solid measurements of task

performance Help designers empathize with user’s

needs and context Uncover problems

11

Types of Prototypes

Low fidelitypaper and pencil sketching; rapid cut-and-

paste.

Mid fidelityscreen printout.

High fidelityworking example, similar in appearance and

functionality to the proposed final product.

12

When to Test?When to Test?assess needs

define goals

generate specs

Create a product

Identify a piece

Fully develop it

alpha test

beta test (field test)

user

testing

13

Design Gel

TIME

needs specsprototyp

e

design

freeze

FormativeFormative

EvaluationEvaluation

SummativeSummative

EvaluationEvaluation

14

What to test

Instructional EffectivenessIs this proposed design effective in

meeting the instructional goals?ComprehensionRetentionTransfer

15

Test on UsabilityTest on UsabilityLearnability: How easy it is to learn to

use the system when referring to a novice user’s experience at the initial part of the learning curve.

Efficiency of Use: This attribute refers to expert users and the level of their performance.

Memorability: How easy the system is to remember when casual users return to it in infrequent periods of time.

Errors: How often do users make errors while using the system, how serious are these errors, and how do users recover from these errors

Satisfaction: How pleasant the system is according to the the individual

satisfaction of each user. Efficiency of Use

Satisf

action time on task

Errors

Memorability

Learnability

Nielsen (1993) defines usability with the use of measurable components so that usability can be systematically be evaluated and improved.

16

Formative Evaluation Questions

Are the instructional activities clearly structured?

Is the content perceived as useful and relevant by the learner?

Are the practice exercises easy to understand and complete?

Is your delivery system well organized (web navigation, blackboard setup, moodle layout) easy to use?

Is the instructor guide easy to follow?

17

Abbreviated Formative Evaluation

Focus on potential weaknesses in your design

Formulate formative evaluation questionsprobe for and evaluate potential

weaknesses

18

How many test subjects?

http://www.useit.com/papers/heuristic/heuristic_evaluation.html

19

How To Test? Here is one approach:

A few at a timeReviseTest it againTest more if you wishYour time is the limit!!!You can also do one big testing session

(with all participants)I am sure reality will help you decide

20

TT33R:R: Test, Tweak, Test, ReportTest, Tweak, Test, Report

Test Tweak Test

Makerecommendations

based on your tests

21

Thank YouThank You

From crystaljingsrFlickr-CC