PROTEA: A Process for procuring accessible software

PROTEA: A Process for procuring accessible software

Dr Richard WalkerE-Learning Development Team Manager & VLE Service Group Leader University of York

Professor Helen PetrieProfessor of Human Computer InteractionUniversity of York

ABOUT THE UNIVERSITY OF YORK

Ranked #7 in The Times Higher Education 100 Under 50 world ranked universities (2013)

Home to more than 15,000 students and 3,000 staff

Blackboard users since 2005

Currently on Blackboard Learn v9.1 Service Pack 12 (licensing community, content and learning management system and mobile learning building block)

ABOUT THE HCI RESEARCH GROUP AT YORK

One of the oldest research groups studying human-computer interaction in the UK, if not the world

Founded in 1984 – centred in Computer Science, but with members in Archaeology, Electronics, Health Sciences, History, Management, Psychology, Sociology and Theatre Film TVKey values – putting humans at the centre of human-computer interactionParticular interest in people with disabilities, older people, e-learning

BACKGROUND – FROM THE E-LEARNING DEVELOPMENT TEAM

Have a strong commitment to accessibility and usability

Institutional staff and student VLE surveys have highlighted usability issues, for example:

• In designing modules

• Uploading content

• EARL reading list application

• Tools felt to be “clunky”

BACKGROUND – FROM THE E-LEARNING DEVELOPMENT TEAM

Have worked with the HCI Group on two occasions:

• On upgrade from v8 of Blackboard Academic Suite to the next generation release 9.1 Service Pack 5 (in 2011)

• To review version 9.1 Service Pack 12 – as an additional action to the VLE retender (see below)

• We report our findings to the University’s E-Accessibility Forum and use the information to inform our training and support to system users

BACKGROUND – FROM THE HCI RESEARCH GROUP

We have done many evaluations of accessibility and usability of websites and web-based systems, in particular:

A formal investigation of websites for the Disability Rights Commission of Great Britain

We evaluated 100 websites with 50 people with disabilities, 10 people per website

Asked them to conduct typical tasks on each websites

We found that if we evaluated with blind, partially sighted and dyslexic participants we found 80% of accessibility problems

OUR CHALLENGE

We wanted to support the university in providing more accessible software, for both students and staff

One of the difficulties is in procuring new software – how do we know it will be accessible, most software does not come with any details of accessibility

If it does, it is usually a statement of conformance to guidelines, perhaps Web Content Accessibility Guidelines (WCAG) – is this sufficient?

OUR CHALLENGE

Two problems:

• WCAG guidelines are not relevant to all software and quite hard to apply to very complex systems

• Guidelines such as WCAG do not tell you definitely that students/staff with disabilities will be able to use the software, they are not precise enough, you need testing with users

OUR SOLUTION:PROTEA: PROTOCOL FOR TESTING E-ACCESSIBILITY

So we created a protocol for user testing of software:

• Can be applied to any kind of software, to be used by students or staff

• Can be conducted by our group, other groups in the university or by the software provider (who would provide evidence that it has been followed)


Need to evaluate the software with 3 blind people, 3 partially sighted people and 3 people with dyslexia, as close as possible to the real users of the software being evaluated

The blind and partially sighted people should use a mix of different assistive technologies to access the software

• Blind participants – JAWS, WindowEyes, VoiceOver, NVDA

• Partially sighted participants – SuperNOVA, ZoomText

• Dyslexia participants – may change screen configuration


Need to pick a series of 3 - 5 “typical tasks” for the participants to do:

• “typical task” means something quite “meaty” • Send an email to your tutor asking for a meeting

• Upload an essay

• Find the reference list for your archaeology course next term

• Start with easy tasks, work up to harder ones, perhaps weave into a story


A facilitator sits with the participant as they go through the tasks, to assist if needed

If possible, sessions should be recorded for later analysis and discussion

Participant talks through what they are doing (“think aloud” or “verbal protocol”)

OUR SOLUTION:PROTEA: PROTOCOL FOR TESTING E-ACCESSIBILITYIf the participant finds a problem, they pause and rate it on a scale 1 (= cosmetic, small annoyance) to 4 (= major, I’m really stuck)

If the participant gets really stuck (e.g. the screen reader cannot interpret the page), the facilitator helps move the participant on

BUT participant should not be “lead” through the task – not told where to click or the particular names of button, links etc


Occasionally might have some of the development team sit in

BUT only if the participant is comfortable with that and will not feel intimidated

Gather all the problems encountered, analyse for which groups of users/assistive technologies are being affected

OUR RESULTS

We applied PROTEA as part of the recent procurement process for the VLE at York

We did find some unexpected accessibility problems that we fed back to Blackboard

We also highlighted some issues where we need to educate our staff members who develop content to ensure accessibility

OUR RESULTS

Issue 1: Assistive Technology unable to give access to functionality

Problem: screen reader users could not interact fully with the Content Editor

Consequences: could not send emails, contribute to fora, answer quiz questions

OUR RESULTS

Issue 2: Navigation within software

Problem: Moving between frames was difficult and inconsistent for screen reader users

Consequences: users could not navigate to where they wanted to be, became “trapped” in frames

OUR RESULTS

Issue 3: Auditory information needs different handling to visual information

Problem: In character limited text boxes, screen readers announced the character count after every keypress

OUTCOMES AND RECOMMENDATIONS

Blackboard have been very responsive in investigating these issues

They are the kinds of problems which will not necessarily be picked up by testing to guidelines

Need really realistic evaluation with users and a range of assistive technologies


Other useful outcomes of using this kind of evaluation:

VLE support staff become much more aware of problems that students with disabilities may face

Can create better support materials, training for students/staff who are creating content


We have used the PROTEA method not only with Blackboard, but with several systems being procured by the university and created in-house

Both systems for students and for staff

We have also worked with the UK government (on DirectGov) and a major bank

THANK YOU!

If you would like more information, we will be setting up a website in the next weeks:

www.yorkhci.org/protea

We are happy to provide general advice for free and of course can undertake evaluations

[email protected]

Documents

PROTEA: A Process for procuring accessible software