HISTORY OF HCIREQUIREMENTS

DESIGNUSER CENTEREDDESIGN PROCESSDATA GATHERING

EVALUATION

Midterm: 10/2What do you want it to be?

Midterm Project Presentations

FORMAT:  15 minutes to present the problem, design space, how

formative evaluation affected your design, and the design including illustrations.

5 minutes of feedback from the class. The presentation should be based on your written team

reports from project steps 2 (req. analysis) and 3 (design & formative). Initial Idea Needs/problem statement, users (primary, secondary, tertiary), client

(if applicable) Requirements Design Space Mock Up/prototype Formative Evaluation (how it effected your design)

10/14: Teams 4, 5, 6 10/16: Teams 1, 2, 3

Notes on Midterm Presentation

Describe changes / iterations Feel free to reuse images and diagrams that were

developed for the reports. Organize Presentation (AND REPORTS !!!!)

Introduction Topic, Initial Idea, Origin, Needs, Problem statement (story, question, etc.)

Body What you have so far and why (design, evaluation, etc.)

Conclusion What you learned, where you are going, what is left to do.

Crisp and clear presentations. Enlist feedback, questions, etc. that will help YOU.

What do YOU need. Be honest (and professional) and don’t be afraid to discuss your weaknesses and real concerns.

NOTE TAKING:  Each team must designate one member to record class feedback, put notes in Design Log

Group 1 Group 2 Group 3 Group 4 Group 5 Group 6

Colonel Panic AwesomeAwesome Code MonkeysCode Monkeys Subscribe De GamaDe Gama OnSite

Music Textbooks Teaching OO Recruiting Map Police

OrganizationOpen ended

questionnaireContextual

InquiryUsers, claims, HTA, scenario

SurveyUnstructured

Interview

Erickson, Benjamin

Weisberg, Alexander

DeForge, Jason

Bourne, Joseph

Zimmer, Andrew

Harris, Morgan

Mey, Henjo Lilly, Kyle Braley, Colin Probus,

Stephen DelValle, Eric

Lindner, William

Everett, Mark

Dove, Andrew Cammarata,

Matthew Cline, James Chelko, Jared

Scott, Benjamin

Lin, Yu-Hsun McFarland,

Daniel Eltahir, Idris Willis, Brad Cox, William  

Data gathering

Overview

Useful data?

Your needs vs. Participant’s

answers

Interviews

Questionnaires

Observation

Choosing and combining

techniques

Interviews

• Unstructured - are not directed by a script. Rich but not replicable.

• Structured - are tightly scripted, often like a questionnaire. Replicable but may lack richness.

• Semi-structured - guided by a script but interesting issues can be explored in more depth. Can provide a good balance between richness and replicability.

Interview questions• Two types:

− ‘closed questions’ have a predetermined answer format, e.g., ‘yes’ or ‘no’

− ‘open questions’ do not have a predetermined format

• Closed questions are easier to analyze Avoid:

− Long questions− Compound sentences - split them into two− Jargon and language that the interviewee may not

understand − Leading questions that make assumptions e.g., why do

you like …?− Unconscious biases e.g., gender stereotypes

Questionnaires

• Questions can be closed or open• Closed questions are easier to analyze, and may

be done by computer• Can be administered to large populations• Paper, email and the web used for dissemination• Sampling can be a problem when the size of a

population is unknown as is common online

Questionnaire design

• The impact of a question can be influenced by question order.

• Do you need different versions of the questionnaire for different populations?

• Provide clear instructions on how to complete the questionnaire.

• Decide on whether phrases will all be positive, all negative or mixed.

Question and response format

• ‘Yes’ and ‘No’ checkboxes

• Checkboxes that offer many options

• Rating scales

– Likert scales

– semantic scales

– 3, 5, 7 or more points?

• Open-ended responses

Observation

Direct observation in the field Structuring frameworks Degree of participation (insider or outsider) Ethnography

Direct observation in controlled environmentsIndirect observation: tracking users’ activities

Diaries Interaction logging

Structuring frameworks to guide observation

- The person. Who? - The place. Where?- The thing. What?

The Goetz and LeCompte (1984) framework:- Who is present? - What is their role? - What is happening? - When does the activity occur?- Where is it happening? - Why is it happening? - How is the activity organized?

Direct observation in a controlled setting

Think-aloud technique

Indirect observation

• Diaries• Interaction logs

Choosing and combining techniques

Depends on The focus of the study The participants involved The nature of the technique The resources available

Summary

Three main data gathering methods: interviews, questionnaires, observation

Four key issues of data gathering: goals, triangulation, participant relationship, pilot

Interviews may be structured, semi-structured or unstructured

Questionnaires may be on paper, online or telephone

Observation may be direct or indirect, in the field or in controlled setting

Techniques can be combined depending on study focus, participants, nature of technique and available resources

Interviewing17

What role do they play? Managers and clerks may view the system completely different.

Prepare yourself as well as the user. Send them warm-up materials.

Depending on their role and availability, meet in a neutral place.◦ Meeting room or closed off room. Reduces distractions for

both parties. No email or ringing phones. Turn off your phone/pager.

Bring:◦ tape recorder◦ Camera◦ legal pad for both of you◦ colored pens◦ business cards◦ laptop. Be careful not to use your computer if it cannot be

seen by all attendees. Don’t hide behind it.

Interviewing Continued...18

Give yourself enough time for the interview and plan for breaks.

Role play if the user’s explanation is unclear. Users cannot articulate the procedures they follow Impossible for you to ask every possible question and

for any user to know questions the developer should be asking.

Draw simple diagrams or decision trees. Complex diagrams will turn off key non-technical users.

Send a summary as soon as possible to the interviewees.

Storyboarding/PrototypingStoryboarding/Prototyping

19

Passive storyboards - sketches, pictures, screenshots, Powerpoint presentations, or sample outputs. Use stick figures.

Active storyboards - automated slide presentation or movie describing the way the system behaves.

Interactive storyboardsInteractive storyboards - require participation by the user. Throw away code. Sample interface or reporting outputs; very close to a throwaway prototype.

Storyboard elements:◦ Who are the players◦ What happens to them◦ How it happens

Storyboarding/PrototypingStoryboarding/Prototyping

20

Don’t invest too much in the prototype. A throwaway prototype shouldn’t take more

than a week to build. Users will be intimidated to make changes if

it looks to close to the real thing.“If you didn’t change anything, you

didn’t learn anything.”If the prototype looks too good, users

will want it now or assume that you are almost done.

Questionnaire Resources21

Questionnaires in Usability Engineering FAQ

Web-based UI Evaluation Questionnaire perl CGI script