Improving the Usability and Accessibility of Voting Systems and Products

Dr. Sharon Laskowski

sharon.laskowski@nist.gov

July 9, 2004

TDGC Meeting

2

3

Human Factors/Usability Perspective on Voting Systems: Voters Cognitive and physical nature of the voters Physical environment Psychological environment Voting product Usability is determined by the demands of

the system and the voter’s ability to perform under those demands

4

Measuring Accessibility and Usability

Accessibility The degree to which a system is available to and usable by

individuals with disabilities Usability

A measure of the effectiveness, efficiency, and satisfaction achieved by a specified set of users performing specified tasks with a given product

Metrics: errors causing a vote cast not as intended or a vote not cast, (errors prior to success), and time to cast vote

Designing and measuring process User-centered design Diagnostic usability evaluation Testing performance—usability testing

5

State of Usability of US Voting Systems In general, voting systems have not been

measured for usability nor have they been developed using a user-centered design process

We do not know the degree to which voters cast their vote NOT as they intended due to confusion with the user interface

Note observations by , CalTech/MIT Voting Technology Project, Herrnson et.al., and others such as Doug Jones

6

Design and Performance Standards

Design Standards—how the product is designed For example, font size, ballot instructions

Performance Standards—how the product functions No overvoting, test by demonstration Time to cast vote, failures in casting vote as intended

Requires: measuring with users against benchmarks, Sample ballots of different complexity, and Well-defined test protocols and user groups

7

We currently cannot measure usability of voting systems E.g., select/deselect Need high degree of usability Following design guidelines does not necessarily insure usability Usability engineering provides measurement methods, but not necessarily to the

degree we need specifically for voting

We need standards and conformance tests that do measure degree of usability and accessibility, if systems are going to be qualified and certified for usability and accessibility

Measurement for Qualification and Certification

rigorousresearch &experiments

informalevaluation

Easy, variable Complex, reliable

feasiblereproducible

conformance testing

8

Current Voting Standards and Testing Current VSS has some accessibility standards, but only a usability

appendix ITAs currently perform qualification tests Can we test for usability and accessibility?

Standards must be clear, unambiguous, testable Requires procedures for testing the voting product against the standards

(conformance testing) For example, inspection , demonstration, operation

IEEE P1583 draft standards Task Group 3 has made some progress

But, lack of resources and small vendor base have been a barrier to developing standards that are performance-based standards, benchmarks and conformance tests

9

The HF Report

Our report recommends an approach that will produce measurable voting systems standards for usability and accessibility Doesn’t need a lot of research Does need:

Expertise in conformance test development Some applied research to develop user testing protocols Neutral third parties to accomplish this

No cheap, quick fixes Could require some usability testing to avoid major

usability blunders, but this is no guarantee

10

10 Recommendations1. Performance-based, high-level usability standards2. Complete set of user-related functional requirements3. Avoid low-level design specifications. Use only product design

requirements that have been validated as necessary4. Applied research to support the development of usability and

accessibility standards5. Review current requirements (Access Board, the current VSS, draft

IEEE standards) for possible adoption6. Ballot design guidelines7. Guidelines for facility and equipment layout; design and usability

testing guidelines for vendor- and state-supplied documentation and training materials

8. Vendors should incorporate a user-centered design approach 9. Conformance tests for voting products against the applicable

accessibility requirements.10. Valid, reliable, repeatable, and reproducible process for usability

conformance testing of voting products against the standards described in recommendation 1) with agreed upon usability pass/fail requirements.

11

Most Critical Need

A set of usability standards for voting systems that are performance-based, with Objective measures Conformance test procedures

Then voting products and systems can be certified that they meet the usability standards

This is the only way to guarantee high levels of usability

12

RoadMap(Details in Report) Short term: encourage usability and user-

centered design Long term:

Use best of IEEE and other standards …and ballot design guidance

Develop user test procedures Collect user data to define performance baselines Develop performance standards and conformance

tests