Product Accessibility Assessment

Prioritizing and Making Recommendations in an Imperfect

World

http://go.ncsu.edu/ahg2012-procurement

Presenters

Greg KrausUniversity IT Accessibility CoordinatorNorth Carolina State Universitygreg_kraus@ncsu.edu

Mark TurnerDirector, Center for Accessible MediaCSU, Office of the Chancellormturner@calstate.edu

Accessibility Throughout the Procurement Process

• Two institutions process for incorporating accessibility at key points in the procurement process– North Carolina State University• Synchronous Learning Management System

– California State University System• Digital Rentals

Topics Covered

• Prioritizing finite resources on key IT products• Developing contract language• Defining vendor responsibilities• Conducting accessibility evaluations• Factoring accessibility into vendor selection• Implementation planning for inaccessible

products

NC State University

Purchasing – Instructions to Bidders

• Language specifies “Enterprise-Level IT Systems or Technologies”

• Vendors must: – Assure all features, fully comply with Section 508 OR – Detail why and in what way any feature does not fully comply

with Section 508– If VPAT is submitted it must include compliance checklists for:

Technical Standards, Function and Performance Criteria, Documentation and Support

– The product is subject to an accessibility evaluation by the University.

• http://www.fis.ncsu.edu/materialsmgmt/purchasing/vendor/bid_instruct.pdf

The Problem Isn’t Policy, It’s Process

Starting the Process

• Three ways to incorporate accessibility into the procurement process– Business process– Personal Relationships– Listening (being nosy)

Prioritization – Learning About The Technology

• What does this software do?• VPAT and other accessibility documentation?• Is the technology part of a required academic or institutional

purpose?• How many people are expected to use this technology?• What is the scope of access to this technology? Does the

technology have any publicly available access or is it limited to a set of users?

• Is the primary audience going to include people with disabilities?

• Is the technology likely to be reused in the future or used for an extended period of time?

Prioritization – Digging Deeper

• List out all of the functions of this technology, indicating:– Other Commercial or Free Solutions?– Comparable Alternatives Already on Campus?– Essential to a Business Need (Yes or No)?– Accessibility Problems?

• Examples from SLMS– Video conferencing– Application sharing

Defining Your Business Needs

• What educational or business needs are you trying to meet?

• In other words, what problem are you trying to solve?• Rank your needs as “Essential, High, Medium, and Low”• Include accessibility as one of your Essential

requirements• http

://oit.ncsu.edu/itaccess/ict-accessibility-regulation-faqs#procurement

Accessibility is not the quintessential criterion in product assessment

Our Search For a New SLMS

• Had been an Elluminate Live! customer for years

• Moved to Blackboard Collaborate for the short term

• Time to reassess the market• (This presentation is not a detailed overview

of the accessibility of SLMS systems. It is a detail of the process we went through.)

Software Considerations From Users and Support

• Cross Platform • Support (Help Desk) • Similar features to those currently available/highly used in Elluminate • Moodle plug-in • Hosting • Pricing/Cost (including $ and staff resources) • Accessibility • System wide licensing (unlimited users/access) • Recording • Migration of current recorded content • Ease of transition (cost to us in staff hours) • Mobile access

Essential Features

• Text chat• Audio chat• Present a PowerPoint presentation• Recording• Cross-platform/cross-browser support• Does not require software installation for participants• High or no limit on participants per session• Accessibility• Adoption at other universities

Initial Products

• Blackboard Collaborate• DimDim• Adobe Connect• WebEx Meeting Center• WebEx Training Center• GoToMeeting• Big Blue Button• Saba Centra• Microsoft LYNC

(teleconference)• iLinc

• Polycom CMA-D (teleconference)

• Skype (people may ask)• Google+• CCC Confer• LiveOn• FastViewer• omNovia• Merit WebConnect• Talking Communities

Passed the Initial Evaluation

• Largely using accessibility (brief testing) and usage at other campuses, we narrowed the list to– Blackboard Collaborate – Adobe Connect – WebEx Meeting Center – WebEx Training Center – Saba Centra – iLinc

How the Finalists Were Determined

• Survey of Elluminate users – what they use• Accessibility options for users with disabilities • Benchmarking from other regional

universities’ SLMS search results

Three Finalists

• Blackboard Collaborate• WebEx Training Center • Saba Centra• Accessibility was not the overriding factor in

this step– One candidate with accessibility support was

eliminated for other reasons

The Final Rubric

• 14 meta tasks we tested– Which was broken down to approximately 80 tasks

• Ranked each task in importance and frequency of use from 1 to 3 (3 being highest or most frequent)

Meta Tasks We Tested

• Create a meeting• Join a meeting• Audio• Chat Tool• Personal Response

Tools• Whiteboard• Polling

• Web Tour• Session Management• Breakout Rooms• File Transfer• Video• Application Sharing• Other

Sample of Ranking of Task Importance

• 3 (Most Important)– Text chat– Audio chat

• 2 (Moderately important)– File transfers– Room timer

• 1 (Least Important)– Rearrange slides while in session– Participate in the session anonymously

Sample Detailed Tasks

• Chat Tool– Send a chat message– Read previous chat messages– Determine when a new chat message appears– Switch between chat rooms– Send a private chat– Read a private chat

Technology Used in Accessibility Testing

• Keyboard-only input • JAWS screen reader • VoiceOver screen reader • ZoomText screen magnifier and reader • Built-in OS contrast tools • Dragon Naturally Speaking

At the end of the day…

• …nothing was perfect• Every product had failures in the “Essential”

category• Do you just add up the number of successful

functions and go with the product that was the most successful?

• No, this is where you earn your keep as an “accessibility expert”

Conclusion

• “Of the three products tested, Blackboard Collaborate provides the most accessible user experience. None of the products are perfectly accessible, and Saba Centra and WebEx Meeting Center are accessible to a degree, however, because of the level of accessibility support present in Collaborate, a strong business case would need to be made to choose either Centra or Meeting Center over Collaborate from an accessibility perspective.”

Elaborating on the Conclusion

• “It is noticeable that significantly more thought has gone into the user experience of Blackboard Collaborate than the other products from an assistive technology user’s perspective. Rather than finding an easy way for a user to simply achieve a basic level of functionality, Blackboard Collaborate seems to consider the optimal way for an assistive technology user to use the system.”

Final SLMS Report

• https://wolfware.ncsu.edu/sops/slms_search/SLMSRecommendationReport.pdf

Planning for Problems

• Optimizations• Workarounds• Accommodations

Optimizations And Workarounds?

• Optimizations– actions the author or institution can take to make

the technology more accessible• Workarounds– actions the end-user can take to make the

technology more accessible

Emerging Technology• “NC State will support the consideration of using emerging

technologies as they become available or prior to purchase, while also advocating for their accessibility from their inception and as they are further developed.”

• You can use (potentially inaccessible) emerging technologies if– The learning objective can only be met by using the emerging technology, OR– No one in your group needs an accessible technology, BUT you must have a

plan in place for providing an equally effective and integrated experience for all users

• http://policies.ncsu.edu/regulation/reg-04-25-05 (1.3)• http://oit.ncsu.edu/itaccess/ict-accessibility-regulation-faqs#emerging-

technologies

Sample Optimization and Workaround

• Google Apps– Optimization• Email notifications for new calendar events by default

– Workaround• Google Doc Download Tool