Embed Size (px)
DESCRIPTION
Presentation on what you can learn from usability testing and projects in ballot and elections. Dana Chisnell and Whitney Quesenbery at EVN 2012
Citation preview
Need a little usability?What you can learn from usability testing
Whitney QuesenberyDana Chisnell
Usability in Civic Life
2
What questions do you have?
3
Usability testing answers questions, so we have to understand the question we are asking
If we want to know:
Which ballot design helps voters be more accurate?
We are asking a question that is
Comparative, quantitative (errors, time)
(and why they made those mistakes)
If we want to know:
Do voters understand how to mark their ballot?
We are asking a question that is abut
Mental models, variability, qualitative insights(and how prevalent each model is)
4
Helpful models for thinking about usability
A’s Ability, aptitude, attitude
(what voters bring to the election)
E’s Efficient, effective, engaging, error-tolerant, easy to learn
(dimensions of usability)
Use Useful, usable (operable), desirable
(how we define success)
5
Location and context: formal informal
Recruiting: defined opportunistic
Activities: instructed tasks free tasks
Questions: structured unstructured
Data collection:observation task/data
only
Results quantitative qualitative
The recipe for planning a usability test
6
A few examples
7Which instructions work better?Traditional ballot instructions vs. plain language version
About this project 45 participants 3 lab setting, 8 days 2 moderators Instructions for voting Careful observation Satisfaction survey
We learned about Voter accuracy Order effects Voter preference
Results Statistical analysis Participant preference
8
Which layout works better?Left aligned vs. right aligned names?
About this project 100 participants 2 locations, 1 day 4 moderators + 6 others Instructions for voting Minimal observation
We learned about Voter preference Time to vote Number of errors
Results Statistical analysis of
marked ballots
9
What do voters do with unusual options?How do voters react to “double-vote” options?
About this project 200 participants 5 locations, 1 day 12 moderators + 5 others Self-defined tasks Minimal observation
We learned about Voter awareness of party Types of marks and errors Need for voter ed.
Results Inspection of ballots for
marking patterns
10What problems might voters have with a new ballot?What problems might voters have with a new ballot design?
About this project 1 of 5 similar tests 10 participants 1 locations, 1 day 2 moderators + 1 others Self-defined tasks Observation Post-task questionnaire
We learned about Navigation on the ballot Instructions
Results Found pattern of errors for
one interaction Time on task
11
What do voters think about different systems?Which accessible voting system do voters prefer?
About this project 496 surveys; multi-disabilit 11 systems Multiday “voting fair”
They learned about Voter assessment of
usability Voter comments about
accessibility Differences by disability
Results Statistical analysis survey
results Qualitative analysis of
comments
12
Do voters understand how their votes are counted?Can voters explain and act on that understanding?
About this project 20 participants 4 locations, 1 day 8 moderators Self-defined tasks Minimal observation
We learned about Navigation on the ballot Instructions Mental models of counting Whether there’s a problem
Results Qualitative insights
13
Do voters understand a new voting system?What is their reaction to new procedures?
About this project Focus group Mock election 1 moderators
They learned about Attitudes and reactions
Results Qualitative insights
In comparison with traditional
optical scan, how easy was it to
mark and cast ballots?
Vs.
Thinking about other elections you
have voted in, how did this
compare? Was it easier or harder to
mark and cast your ballot?
Did you perform a “print audit?”
Vs.
Did you check that your ballot was
correctly printed?
14
Things to watch out for when planning a usability test
15
Treating participants with respect
Language and cultural issues
Managing context
Participant protections and informed consent
16
Avoiding bias
Leading questions
Closed vs. open questions
Sampling bias
Moderator influence
Observer (or moderator reactions)
17
Thoughts?
18
Whitney [email protected]
Dana [email protected]
Field guides to ensuring voter intenthttp://tinyurl.com/ensuringvoterintent
Usability in Civic Lifehttp://usabilityinciviclife.org
Civic Designhttp://civicdesigning.org
LEO Usability Testing Kithttp://www.usabilityinciviclife.org/voting/leo-testing-kit/
