Embed Size (px)
Citation preview
EVALUATING ACCESSIBILITY AND USABILITY FOR
DIFFERENT APPLICATIONS
by
GEORGE CHALHOUB
ABSTRACT
In this report, we briefly evaluate software and web applications for their accessibility and their usability
according to official W3 standards. Two software applications (VLC media player, Norton Internet Security)
and two web applications (Steam, Reddit) were evaluated. Numerous assistive technologies and online
resources were used to validate the accessibility and usability of those websites.
January 3, 2018
3,112 words1
1 Count from page 7 until page 17 (excluding headers and figure captions).
Back to top
1 | P a g e
This page intentionally left blank.
Back to top
2 | P a g e
Table of Contents Integrity statement ............................................................................................................................................ 4
Report Accessibility ............................................................................................................................................ 5
Introduction ....................................................................................................................................................... 6
Steam ............................................................................................................................................................. 6
Reddit ............................................................................................................................................................. 6
Norton Internet Security ................................................................................................................................ 6
VLC media player............................................................................................................................................ 6
Software Accessibility Evaluation ...................................................................................................................... 7
Norton Internet Security ................................................................................................................................ 7
VLC Media Player ........................................................................................................................................... 7
Software Usability Evaluation ............................................................................................................................ 8
Norton Internet Security ................................................................................................................................ 8
VLC Media Player ........................................................................................................................................... 8
Web 2.0 Accessibility Evaluation ....................................................................................................................... 9
Steam ............................................................................................................................................................. 9
Reddit ........................................................................................................................................................... 10
Web 2.0 Designing to Standards Evaluation .................................................................................................... 10
Steam ........................................................................................................................................................... 10
Reddit ........................................................................................................................................................... 11
Web 2.0 Usability Evaluation ........................................................................................................................... 12
Steam ........................................................................................................................................................... 12
Reddit ........................................................................................................................................................... 12
Web 2.0 Browser and OS compatibility evaluation ......................................................................................... 13
Steam ........................................................................................................................................................... 13
Reddit ........................................................................................................................................................... 13
Code Validation ................................................................................................................................................ 13
Reddit ........................................................................................................................................................... 14
Steam ........................................................................................................................................................... 15
Assistive technology ......................................................................................................................................... 15
Wave ............................................................................................................................................................ 15
NVDA ............................................................................................................................................................ 16
BrowserStack ................................................................................................................................................ 16
ATbar ............................................................................................................................................................ 16
Back to top
3 | P a g e
Microsoft Edge’s Accessibility Tree .............................................................................................................. 16
W3C Markup Validation Service .................................................................................................................. 17
W3C CSS Validation Service ......................................................................................................................... 17
Appendix .......................................................................................................................................................... 18
Appendix A – Assistive technology use ........................................................................................................ 18
Wave ........................................................................................................................................................ 18
BrowserStack ............................................................................................................................................ 20
Microsoft Edge’s Accessibility Tree .......................................................................................................... 21
Appendix B – Accessibility and usability tests ............................................................................................. 23
Software Accessibility tests ...................................................................................................................... 23
Web Accessibility tests ............................................................................................................................. 27
Appendix C – Other screenshots .................................................................................................................. 28
References ........................................................................................................................................................ 35
Back to top
4 | P a g e
Integrity statement
I, George Chalhoub, confirm that this report is wholly my work and I haven’t received any assistance when
doing it. I am fully aware of the requirements of good academic practice, and the potential penalties for
any breaches.
/george chalhoub/
Back to top
5 | P a g e
Report Accessibility
Great efforts have been made to make this report accessible in numerous formats and many platforms. In
this page we will list the formats where the report will be available along with the URLs sending you to the
format. All the files will be downloaded once you click on the links.:
• Portable Document Format (PDF): WWW link for the accessible report in PDF format.
• Microsoft Word (DOCX): WWW link for the accessible report in DOCX format.
• Microsoft Word 97-2003 (DOC): WWW link for the accessible report in DOC format.
• Microsoft Word Template (DOTX): WWW link for the accessible report in DOTX format.
• OpenDocument Text (ODT): WWW link for the accessible report in ODT format.
• Plain Text (TXT): WWW link for the accessible report in TXT format.
• Rich Text Format (RTF): WWW link for the accessible report in RTF format.
• Single File Web Page (MHT): WWW link for the accessible report in MHT format.
Back to top
6 | P a g e
Introduction
In this assignment, we evaluate two software applications and two web applications for their usability and
accessibility. My choice was to evaluate applications that I have used significantly in the past years. Here
are my choices for the two web applications:
Steam
Steam Community is the world’s largest gaming digital distribution platform. Steam was estimated to have
75% of the game market in October 2013, and in 2015 it had around 125 million registered accounts.
Steam can be directly accessible as a web application at any time at this URL: www.steamcommunity.com.
Reddit.com is one of the world’s most famous websites that encompasses social news discussion, web
content rating, and discussions. Reddit is the 8th most visited website in the world according to Alexa Rank.
Reddit is a web application that can be accessed by typing reddit.com on a web browser.
As for the software applications, we choose:
Norton Internet Security
Norton Internet Security is a prominent malware prevention and removal software developed by
Symantec Corporation in 1991. It uses many technologies to identify viruses on numerous platforms.
Services offered by Norton Antivirus include email spam filtering and browser protection.
VLC media player
Released 16 years ago, VLC media player is one of the world’s best open-source media players used by
millions of users in the world. VLC is available on all major operating systems such as Windows, macOS,
and Linux. It is also available for Android and iOS.
Back to top
7 | P a g e
Software Accessibility Evaluation
Using the Department of Justice’s software accessibility checklist [1], we evaluate Norton Internet Security
(Screenshot B2) and VLC media player (Screenshot B1) for their accessibility.
Norton Internet Security
While the software allows the use of keyboard functions, it does not provide any documentation nor any
instructions what-so-ever on how to use them. It also seems that keyboard shortcut buttons are not
available for drop-down menus.
Some objects and icons do not contain any descriptive labels or text which includes but is not limited to:
search icon, help icon, and hide/show icon.
Furthermore, the software seems to be using its company-made controls/elements which might confuse
users with disabilities.
As for the fonts, defaults fonts cannot be altered for texting and printing displays. Moving on, no colour
settings are present which prevents the user from altering any colours on the screen.
Generated reports from the software are not available to print in an ASCII file format (Screenshot C10).
Lastly, the company does not provide any training to people with disabilities to use their software. The
word ‘accessibility’ is not found in the help centre.
VLC Media Player
Some mouse actions such as getting through the software header, editing hotkeys in the preferences do
not have any keyboard equivalence.
Tabbing does not work to navigate the elements of the media player. If hotkeys are not known, navigating
the software with a tab would not be possible. While the preferences are navigable with UP and DOWN
arrows, using ENTER or SPACE, do not save the settings but close the preference page.
Icon objects do not have a clear named label visible unless the mouse hovers over the objects. However,
VLC does not offer any means to display labels on the icons.
Furthermore, VLC media player does not support the ‘show sounds’ feature offered by Windows operating
system. As for the colours, highlighting is not viewable with inverted colours.
Documentation is provided online in a .html format (Screenshot C11) but does not offer ASCII file format.
There is no way to access the document from within the software (offline mode).
Finally, VLC media does not also offer any training for users with disabilities to access the software.
Back to top
8 | P a g e
Software Usability Evaluation
According to the software usability guidelines reported by Whitney Quesenbery [2], we evaluate the
usability of Norton Internet Security and VLC media player. Quesenbery lists five Es for usability which are
used to assess the applications: Effective, Efficient, Engaging, Error-tolerant and Easy to learn.
Norton Internet Security
According to specific guidelines [3] that were published to evaluate the usability of internet security tools,
we evaluate Norton Internet Security for its usability.
Despite having a strong reputation, this software has a history of draining the speed of computers.
Symantec even confirmed that ‘there was some truth to this years ago’ [4]. Gamers or movie watchers
with Norton Internet Security activated, are advised to activate ‘Full-Screen Detection’ mode to prevent
Norton from slowing down the machine (Screenshot C13). There is no valid reason why users need to go
through the process of activating this mode.
In many occasions, Norton Internet Security only uses graphs and images to inform the customer about
the gravity of the security threat or the trust of a particular program (Screenshot C12). For example, the
trust of a program on Norton’s Program Control Interface is seen only via coloured icons (Screenshot C12).
A person who is colour-blind would not be able to grasp information. Written confirmation and words
should be directly visible in the first alert shown or any security information present.
In other instances, the software is reported to prevent genuine users from activating the product due to
anti-piracy practice implemented by the company [5]. For instance, users report small font size, no
description, and other usability issues.
Moving on, recovering from user decisions is not easy. To test this, we added a Trojan Horse on a Windows
10-laptop with Norton Internet Security’s protection. Upon viewing a security threat notification, we have
clicked on ‘Ignore threat’. However, recovering or undoing the click was no easy task. Many steps were
needed to recover from such actions including reading the documentation. A user might press a button by
mistake or might rapidly respond to a security alert without anticipating the impact of a click.
VLC Media Player
The most prominent issue with VLC Media Player is that it does not use the QT accessibility API which
makes it very hard for blind people to use [6].
The search functionality on VLC Media Player is misguiding. It appears to be doing a universal search, but
in fact, it is searching a directory of a playlist [7].
VLC also seems to contain keyboard shortcuts that are very hard to guess or even find in the
documentation. For example, Shift+Arrows are used to rewind or go forward in a movie. The community
Back to top
9 | P a g e
of VLC media player have requested to have the forward and rewinding tasks done with only the Arrows
(excluding the shift) [6].
Furthermore, playlists running in full-screen mode halt the mode every time a video plays; so a user would
have to trigger the full-screen mode again. There doesn’t seem to be an option to fix this configuration.
When initiating an update for VLC media player, the software abruptly closes itself to install the update.
Updates should be installed after the VLC is closed and not during a session. Lastly, Windows users report
a shortcut for VLC on the start menu tab without their permission.
Web 2.0 Accessibility Evaluation
Based on official Web Content Accessibility Guidelines (WCAG) [8], we evaluate Steam and Reddit based
on their accessibility. An accessibility review was also published on University of Southampton’s
Web2Access [9] website and is pending approval.
Steam
Despite being accessible with tabbing, Steam’s registration and login forms fail a screen reader test by
NVDA. Title input fields do not uniquely identify with text input fields resulting in failing a screen reader
test. Placeholders are not found in text input fields.
Furthermore, the website rarely uses any alt tag attributes to describe images (Screenshot A2). There is no
use of alt tags except on the logo of the parent company and the website.
Moving on, the website fails to establish link target definitions. It is full of non-defined links such as ‘Install
Steam’ or ‘Play now’ lacking title attributes as well. Non-defined links such as ‘Learn more’ or just
‘download’ are found everywhere failing to indicate activation.
While content remains accessible, the website loses its structure when the styling is removed with WAVE
(Screenshot A1). Icons and buttons start to appear weirdly. Video of gameplay is found on some pages, but
when the character speaks, no subtitles are found and cannot be enabled. There is no subtitle description
of what is being heard as well.
Tab order does not work correctly in the Customer Reviews section (Screenshot C4). Focus moves in a
logical order except with some left sidebars on the main page steampowered.com.
Lastly, the website has plenty of low contrast colours between foreground and background as reported by
WAVE (Screenshot A3).
Even though the website zooms in (Screenshot C1) and out (Screenshot C2) correctly, a person with a
visual disability would find it hard to access steam website successfully. A person with a hearing disability
would not be able to comprehend the media content on the website.
Back to top
10 | P a g e
The website scores an average score of 46% using University of Southampton’s Web2Access tool
(Appendix B – Table 2) [11].
While the registration page is fully accessible by a screen reader and keyboard tabbing, a captcha is
present (Screenshot C6) without any alternatives offered.
All subreddits on the main website contain images with empty alt attributes (Screenshot A5). Those
images would have benefitted from alt tags. Other subreddits and posts on the website contain images
without alt tags as well.
Some link target definitions such as ‘read more’ and ‘view more’ are found on the index page. Some icon
links have no accessible names like the search icon.
Iframes inside Reddit do not have any titles. The same issue happens with tables where they have no
captions nor headers for rows and columns.
Some content becomes out of order when the styling is removed (Screenshot A4).
The tab order works well except for the title of a subreddit which could never be selected using a Tab.
The page functionality with the keyboard is not always consistent. Some elements of the page become
inaccessible without a mouse. For example, clicking on 'share' would open a toolbox with an 'X' that can
only be clicked with a mouse and is inaccessible with the keyboard.
The website contains a text editor with absolutely no keyboard shortcuts and instructions that are not
adequately readable by a screen reader.
WAVE also reports issues with very low contrast (Screenshot A6). When Reddit’s main page window is
shrunk, the website does not resize correctly.
The website scores an average score of 46% using University of Southampton’s Web2Access tool
(Appendix B – Table 1) [10].
Web 2.0 Designing to Standards Evaluation
Using the four primary guiding principles (Perceivable, Operable, Understandable, Robust) of accessibility
in WCAG 2.0, we evaluate Steam and Reddit for standards [12].
Steam
A test by WAVE shows that the main content of the website and its structure are linked to its style. The
structure of the website is not independent of its style, and thus they are not separated.
Back to top
11 | P a g e
Moving on, when adding a credit card to the steam community, the website gives only 5 minutes to fill the
secure form before it expires and logs out a user. Unlimited time should be given instead to a user on the
form.
As for the media on the website, it does contain play/pause, but it does not allow the user to rewind or
fast-forward the content.
User profiles of gamers have animations that can be freely set. There are no ways for a user to disable or
hide them. Some users might be at risk and experience seizures because of this. For example, if you visit
the profile of a gamer called ‘Elite’, you can see a large flashing animation (Screenshot C5) on the page
[13].
Lastly, navigating the website is not easy, the user must pass through multiple steps or stages to be able to
download a game (including credit card adding verification, sign up, verification, etc..). The points of
interaction in the website are not clear or visible to give the best user experience possible.
Using Microsoft Edge’s accessibility tree, we can see that a screen reader would never be able to describe
steam (Screenshot A9). The structure seems to suffer from many inconsistencies and the accessibility
tree’s content cannot be understood by inspection.
The header bar on the top is filled with titles of subreddits in a tiny font (size 5 or 6) and are all in caps.
Furthermore, links on Reddit are not given any highlighting after receiving keyboard focus.
The website contains many subreddits with flashing content, Aww [13] and Jokes [14], without taking into
consideration that this content includes flashing and might trigger seizures for some people. There are no
warnings on the site either about their flashings.
The website apparently fails navigation standards. Reddit is continuously sending users to external images
on imgur.com, videos on youtube.com, and other websites found online. Without any warnings, those
users are redirected to another website without having any control, warnings or guidance for the users.
The website contains some misleading advertising that would confuse main content with advertised
content (Screenshot C9). Despite having ‘promoted’ on it, it has posts containing the same layout of
regular entries possibly creating confusion and driving users to places where they do not intend to go.
Using Microsoft Edge’s accessibility tree, we can see that a screen reader would fail to describe the
website adequately (Screenshot A10). While the screen reader would be able to describe the login and
registration forms; the search form and main content would not be appropriately read.
Back to top
12 | P a g e
Web 2.0 Usability Evaluation
According to an official paper posted by W3 called ‘Exploring Usability Enhancements’ by May and
Koiveunen [16], we are evaluating the usability of Steam and Reddit.
Steam
The website’s navigation fails to put the user in a context where the position of the user is clear and
concise. Going back, for example, would have to be done from the browser and not from the website.
Searching for a game redirects the user to store.steampowered.com without having an easy option to
return back.
The website is not consistent with its template layout. For example, when a user visits the help support
page, the header’s spacing differs and the footer disappears completely.
The main website and the news website has infinite scrolling, preventing the user to click on or even read
any parts of the footer.
On some search forms, the website fails to provide feedback on user actions. On the ‘Find Hubs’ search
form, if you type ‘soton’, you will get no feedback and pressing on the search button will not generate any
action or message.
While the search page is comprehensive, it does not offer metadata to enhance the search queries of a
user and invoke an advanced search experience.
Moving on, using the website for the first time to download games is not an easy task as many steps are
needed: signing up, signing in, adding a credit card, making the purchase, installing the steam client, and
downloading the game.
The website does not make an effort to correct errors committed by the user. For example, misspellings in
search queries are not corrected and return no results (e.g. searching for ‘toomb raider’ returns nothing).
In other cases, the steam client does not warn macOS users when they are buying Windows-only-
compatible games since the game will not run on their platform.
In addition to May and Koiveunen’s paper [16], usability heuristics published by Jakob Nielsen were used
to evaluate the usability of the Reddit website [17].
The website sends users to other websites, without warning thus minimising the control that the user has
over a webpage and hindering the user experience. For example, the website uses imgur.com to host the
images uploaded by the users. When images are posted and linked, instead of dynamically loading the
image into the browser, the user is redirected to the imgur.com website.
Back to top
13 | P a g e
Each subreddit has its different styling despite having structured content like other subreddits, thus
creating confusion and difficulty in navigating or finding common elements of the header; lacking
consistency.
The website fails to use simple and clear dialogues at some points; there is a ‘<3’ subheading found in the
footer page failing to inform some users what it would mean clearly. The website also uses unknown
wording such as ‘reddiquette’.
Web 2.0 Browser and OS compatibility evaluation
Steam
The website’s store page is not compatible with Internet Explorer 11 and below. According to tests done
on NetRenderer, the store page is not either compatible with earlier versions of Internet Explorer such as
Internet Explorer 8 (Screenshot C3), 9, and 10.
Using cross-browser commercial tool BrowserStack [18], the website’s store page was tested on hundreds
of browsers and devices, and the results were published publicly in the form of screenshots (Screenshot
A7) [19][20].
The website’s logo and thumbnails fail to load on Samsung Galaxy Tab 2, Galaxy Tab 4 and Galaxy 10.1
(Portrait) whereas on the iPad fourth (Portrait) the navigation page does not work.
The website also appears to be entirely non-functional on Sony Xperia Tipo (portrait) as well as on
Samsung Galaxy Note 10.1 (portrait).
A test with BrowserStack shows that Reddit fails to load on some mobile devices (Screenshot A8) [21].
Examples include Samsung Galaxy S3, Samsung Galaxy S2 and Google Nexus 4. The website is compatible
with Internet Explorer except for versions 5.5 and 6 as reported by NetRenderer.
There seems to be numerous compatibility issues with Reddit that are reported on a bug tracker on Reddit
[22]. For example, users have reported not being able to post a reply to comments from iPad Safari [23].
After attempting to reproduce this bug, it seems to be still not fixed.
Other users have reported JavaScript issues with Chrome running on Mac [26]. Whereas on Firefox Mac,
users are having difficulties with posting comments and upvoting [24]. Another user reports that the
scrollbar on iPad is conflicted between two cards [25] on all iPads.
Code Validation
Back to top
14 | P a g e
We validate and check the XHTML and CSS of the home page of the websites Steam and Reddit. For
XHTML, we use W3C’s Markup Validation Service [30] as for the CSS, we use the The W3C CSS
Validation Service [31].
HTML Validation Results:
Screenshot of W3C’s Markup Validation Service’s results on the Reddit website.
CSS Validation Results:
Screenshot of W3C’s CSS Validation Service’s results on the Reddit website.
Reddit’s main page seems to contain 64 CSS Errors and 3 HTML errors.
Back to top
15 | P a g e
Steam
HTML Validation Results:
Screenshot of W3C’s Markup Validation Service’s results on the Steam website.
CSS Validation Results:
Screenshot of W3C’s CSS Validation Service’s results on the Steam website.
Steam’s main page seems to contain 25 CSS Errors and 15 HTML errors.
Assistive technology
Wave
Back to top
16 | P a g e
WAVE is a free-to-use “web accessibility evaluation tool” that was developed by WebAIM and was
launched in 2001. WAVE displays websites in three different forms: styles – no styles – contrast) [27].
‘Styles’ shows the website with all icons and indicators whereas ‘No Styles’ displays the website with no
CSS styling and liberalised tables showing only HTML but with wave icons and indicators. As for the
‘Contrast’ mode, it shows only contrast issues per WACG 2.0 official guidelines.
Wave was used heavily to check many accessibility issues with Steam and Reddit.
NVDA
NVDA (short for NonVisual Desktop Access) is a screen reader which allows people with visual disabilities
to read text from computer screens in a computerised voice. NVDA allows full control of users with the
move of the cursor allowing them to make NVDA only read what they want. NVDA is only compatible with
Microsoft Windows operating systems [28].
NVDA was used to check the accessibility of the forms in Reddit and Steam.
BrowserStack
BrowserStack [18] is a paid online software that allows you to test website compatibility on real browsers. Instead of having to test a website on different devices and operating systems manually, the website link is provided to BrowserStack, and the service does testings on hundreds of devices and operating systems. Browser Stack was used for checking the accessibility of Steam and Reddit on hundreds of devices and
operating systems.
ATbar
ATbar is an open-source toolbar that helps users customise and interact with web pages [29]. The toolbar
combines multiple functionalities and allows you to utilize them by that toolbar. ATbar can help users who
are dyslexic or visually impaired or who want to change the contrast.
ATbar was used for checking the usability of Steam and Reddit and was used to check for the zooming.
Microsoft Edge’s Accessibility Tree
The Accessibility tree is a unique feature of Microsoft recent’s browser Edge. The accessibility tree shows
the structure of a web page in the same way it would appear to a screen reader.
This feature was used with Steam and Reddit to check for their compatibility with screen readers.
Back to top
17 | P a g e
W3C Markup Validation Service
This W3C Markup Validation Service [30] checks the markup validity of Web documents in HTML or
XHTML. This tool was used to check the HTML validity of Steam and Reddit.
W3C CSS Validation Service
This W3C CSS Validation Service [31] checks the validity of Check Cascading Style Sheets (CSS)
and/or(X)HTML documents with style sheets on websites. This tool was used to check the CSS validity of
Steam and Reddit.
Back to top
18 | P a g e
Appendix
In this appendix, we list vital information for this report. In Appendix A, we will list results provided by
assistive technologies when performing tests and experiments on the applications. Whereas in appendix B,
we provide the results of forms filled and checked to test the accessibility and usability of websites and
applications. In appendix C, we list essential screenshots to accompany or document the results of our
findings.
Appendix A – Assistive technology use
Wave
Screenshot (A2) on WAVE website showing
one major error in steam website.
Screenshot (A1) of results of a WAVE test with ‘No Styles’ mode on Steam website. You can
see that the website’s structure is clearly not independent from its style
Back to top
19 | P a g e
Screenshot (A3) on WAVE website showing hundreds of errors with the
contrast of the steam website. WAVE mainly reports very low contrast on the
website.
Screenshot (A4) of results of a WAVE test with ‘No Styles’ mode on the Reddit website. You can
see that the website’s structure is independent from its style but it suffers from some
inconsistencies.
Back to top
20 | P a g e
BrowserStack
Screenshot (A5) of results of a WAVE test with ‘Styles’ mode on the Reddit website. You can
see that the website’s is generally okay but has some inconsistencies.
Screenshot (A6) on WAVE website showing hundreds of errors with the contrast of the Reddit
website. WAVE mainly reports very low contrast on the website.
Back to top
21 | P a g e
Microsoft Edge’s Accessibility Tree
Screenshot (A7) of BrowserStack website showing capture results of the steam website on
many platforms and devices.
Screenshot (A8) of BrowserStack website showing capture results of the Reddit website on
many platforms and devices.
Back to top
22 | P a g e
Screenshot (A9) of the accessibility tree of the steam website on Microsoft Edge. You can
clearly see that the accessibility tree is not proper and would fail a screen reader.
Screenshot (A10) of the accessibility tree of the Reddit website on Microsoft Edge. You can
clearly see that the accessibility tree is not proper and would fail a screen reader.
Back to top
23 | P a g e
Appendix B – Accessibility and usability tests
Software Accessibility tests
Part 1/2 of a screenshot (B1) of a usability checklist
which was provided by the US Department of
Justice and applied on VLC media player.
Back to top
24 | P a g e
Part 2/2 of a screenshot (B1) of a usability checklist
which was provided by the US Department of Justice
and applied on VLC media player.
Back to top
25 | P a g e
Part 1/2 of a screenshot (B2) of a usability checklist
which was provided by the US Department of Justice
and applied on Norton Internet Security.
Back to top
26 | P a g e
Part 2/2 of a screenshot (B2) of a usability checklist
which was provided by the US Department of Justice
and applied on Norton Internet Security.
Back to top
27 | P a g e
Web Accessibility tests
Website: reddit.com
Reviewed: on 03 Dec 2017
Average score: 46%
Table 1 - Summary of evaluation results of Reddit website by Web2Access
# Test Outcome Notes
31 Login, Signup and Other Forms
33% Keyboard accessible. Pass NVDA test. No CAPTCHA alternative offered at all.
32 Image ALT Attributes and Text Alternative
33% Bad usage of null alt attributes of images of trending subreddits. The rest of images contain alt tags appropriately.
33 Link Target Definitions 33% Use of read more on the index page (https://www.reddit.com/wiki/index) without any explanations. Some icon links have no accessible names like the search.
34 Logical Page Structure and Order
33% Pages have titles but not iframes. No skip to main content. Keyboard navigation ok.
35 Removal of Styles 67% Generally ok. Some content out of order.
36 Audio/Video Features N/A Site does not have video or audio.
37 Video/animations - audio descriptions
N/A Does not have audio or video.
38 Appropriate use of Tables 33% Translation table used with no headers for rows or columns. No captions for tables.
39 Tab Order Correct and Logical 67% Works well except for the title of a subreddit. It is never selected.
40 Page Functionality with Keyboard and Touch Screen
67% Clicking on 'share' would open a toolbox with an 'X' that can only be clicked with a mouse and is inaccessible with keyboard
41 Accessibility of Text Editors 0% No keyboard shortcuts provided. Screenreader does not help.
42 Appropriate Feedback 67% Good feedback, timing restrictions start with many failed attempts.
43 Contrast and Colour Check 33% Very low contract issues. Webaim detected.
44 Page Integrity when Zooming 100% Very good.
45 Text size, style, blinking elements and Readability
33% Fonts on top below 10pt. Header text found with all caps.
Back to top
28 | P a g e
Website: steamcommunity.com
Reviewed: on 30 Nov 2017
Average score: 46%
Table 2 - Summary of evaluation results of steam website by Web2Access
# Test Outcome Notes
31 Login, Signup and Other Forms
67% Title fields do not uniquely identify with text input fields due to bad programming.
32 Image ALT Attributes and Text Alternative
0% Alt tags are only found on website logo.
33 Link Target Definitions 0% 'Read more' and 'See more' and 'See all' without any explanation.
34 Logical Page Structure and Order
67% Focus moves in a logical order except with some left sidebars on the main page steampowered.com. The main content is directly loaded, no need for skip to main.
35 Removal of Styles 33% The structure is lost completely. Tested with WAVE. Large icons start appearing.
36 Audio/Video Features 0% No subtitles in videos. Videos autoplay.
37 Video/animations - audio descriptions
0% No option offered.
38 Appropriate use of Tables N/A The website does not use tables.
39 Tab Order Correct and Logical
67% Focus moves in a logical order except with some left sidebars on the main page steampowered.com
40 Page Functionality with Keyboard and Touch Screen
100% 'Read more' on user reviews of games. Button is recognised as an interactive element and can be triggered with a keyboard.
41 Accessibility of Text Editors
N/A The website has no text editors.
42 Appropriate Feedback 100% No redirects or data cleared, works well.
43 Contrast and Colour Check 0% WAVE: Very low contrast between foreground and background colours.
44 Page Integrity when Zooming
100% Works well.
45 Text size, style, blinking elements and Readability
67% Most text is above 10pt and most fonts used are sans-serif.
Appendix C – Other screenshots
Back to top
29 | P a g e
Screenshot (C1) of the Steam website (steamcommunity.com) with 200% zoom. The website
doesn’t lose its integrity when zoomed in.
Screenshot (C2) of the Steam website (steamcommunity.com) with 50% zoom. The website
doesn’t lose its integrity when zoomed out.
Back to top
30 | P a g e
Screenshot (C3) of the Steam Community login webpage on Internet Explorer 8. The
websites loses its integrity and becomes unusable.
Screenshot (C4) of the customer reviews section on the steam website.
Back to top
31 | P a g e
Screenshot (C5) of the profile page of a user on the steam website. The user is using a long
flashing animation as his artwork showcase which might trigger seizure for some users.
Screenshot (C6) of Reddit’s registration form. A captcha is needed to register and no
alternatives are offered.
Back to top
32 | P a g e
Screenshot (C9) of Reddit’s main page showing a misleading advertisement. A person with a
disability might confuse the advertisement with real website content.
Screenshot (C10) of a diagnostic report by Norton Internet Security. There is no option to export
the report into an ASCII format.
Back to top
33 | P a g e
Screenshot (C12) of Norton Internet Security’s Program Control interface.
Screenshot (C11) of VLC Media Player’s documentation about keyboard hotkeys. This is helpful
for all users whether they have any disabilities or not.
Back to top
34 | P a g e
Screenshot (C13) of Norton Internet Security’s Administrative Interface. ‘Full-
Screen Detection’ mode needs to be activate to improve the performance.
Screenshot (C14) of VLC media player software. Search is misleading and
searches only for current playlist.
Back to top
35 | P a g e
References
[1] Department of Justice (2015). Software Accessibility Checklist | CRT | Department of Justice. [online]
Available at: https://www.justice.gov/crt/software-accessibility-checklist [Accessed 26 Dec. 2017].
[2] Whitney, Q. (2004). Balancing the 5Es: Usability. Cutter IT Journal, 17(2).
[3] Ibrahim, T., Furnell, S., Papadaki, M., & Clarke, N. (2009). Assessing the usability of personal internet
security tools. In Proceedings of the 8th European Conference on Information Warfare and Security (ECIW
2009), Military Academy, Lisbon & the University of Minho, Braga, Portugal (July 6-7, 2009).
[4] Support.norton.com. (2017). Does Norton slow down my computer?. [online] Available at:
https://support.norton.com/sp/en/us/home/current/solutions/v120813372_ns_retail_en_us [Accessed 26
Dec. 2017].
[5] Community.norton.com. (2017). Norton Anti-Theft activation bug (usability issue) | Norton Community.
[online] Available at: https://community.norton.com/en/forums/norton-anti-theft-activation-bug-
usability-issue [Accessed 26 Dec. 2017].
[6] Wiki.videolan.org. (2017). Usability Complaints - VideoLAN Wiki. [online] Available at:
https://wiki.videolan.org/Usability_Complaints/ [Accessed 26 Dec. 2017].
[7] Medium. (2017). A Usability Test on VLC Media Player – Usability Testing – Medium. [online] Available
at: https://medium.com/usability-testing/a-usability-test-on-vlc-media-player-cd3ab5175918 [Accessed
26 Dec. 2017].
[8] W3C Web Accessibility Initiative (WAI). (2005). WCAG Overview ◦ Web Accessibility Initiative ◦ W3C.
[online] Available at: https://www.w3.org/WAI/intro/wcag#components [Accessed 26 Dec. 2017].
[9] Web2access.org.uk. (2017). Web2Access. [online] Available at: https://web2access.org.uk/ [Accessed
26 Dec. 2017].
[10] Web2access.org.uk. (2017). Review of Reddit · Web2Access. [online] Available at:
https://web2access.org.uk/products/372/reviews/2017-12-03 [Accessed 26 Dec. 2017].
[11] Web2access.org.uk. (2017). Review of Steam Community · Web2Access. [online] Available at:
https://web2access.org.uk/products/369/reviews/2017-11-30 [Accessed 26 Dec. 2017].
[12] W3.org. (2017). Introduction to Understanding WCAG 2.0 | Understanding WCAG 2.0. [online]
Available at: https://www.w3.org/TR/UNDERSTANDING-WCAG20/intro.html [Accessed 26 Dec. 2017].
[13] Steamcommunity.com. (2017). Steam Community :: Elite. [online] Available at:
http://steamcommunity.com/id/addo37 [Accessed 26 Dec. 2017].
[14] reddit. (2017). A subreddit for cute and cuddly pictures • r/aww. [online] Available at:
https://www.reddit.com/r/aww/ [Accessed 26 Dec. 2017].
[15] reddit. (2017). Jokes: Get Your Funny On! • r/Jokes. [online] Available at:
https://www.reddit.com/r/Jokes/ [Accessed 26 Dec. 2017].
Back to top
36 | P a g e
[16] W3.org. (2002). Exploring Usability Enhancements in W3C Process. [online] Available at:
https://www.w3.org/2002/09/usabilityws.html [Accessed 26 Dec. 2017].
[17] Nielsen Norman Group. (1995). 10 Heuristics for User Interface Design: Article by Jakob Nielsen.
[online] Available at: https://www.nngroup.com/articles/ten-usability-heuristics/ [Accessed 26 Dec. 2017].
[18] Browserstack.com. (2017). Cross Browser Testing Tool. 1000+ Browsers, Mobile, Real IE.. [online]
Available at: https://www.browserstack.com [Accessed 26 Dec. 2017].
[19] Browserstack.com. (2017). Cross Browser Screenshots for http://store.steampowered.com. [online]
Available at: https://www.browserstack.com/screenshots/94b65664a686ae2de21dcf04f2e4d3fda8e039d1
[Accessed 26 Dec. 2017].
[20] Browserstack.com. (2017). Cross Browser Screenshots for http://store.steampowered.com. [online]
Available at: https://www.browserstack.com/screenshots/7964e0c5634f057ceee7ef0cf43baa3c28a03f5f
[Accessed 26 Dec. 2017].
[21] Browserstack.com. (2017). Cross Browser Screenshots for https://www.reddit.com/. [online] Available
at: https://www.browserstack.com/screenshots/fbe3c14d3aa1354e84fc83d4e25dc7c102bce81c
[Accessed 26 Dec. 2017].
[22] reddit. (2017). bugs • r/bugs. [online] Available at: https://www.reddit.com/r/bugs/ [Accessed 26
Dec. 2017].
[23] reddit. (2017). Can’t post comments or reply on iPad from Safari. • r/bugs. [online] Available at:
https://www.reddit.com/r/bugs/comments/7k4lhp/cant_post_comments_or_reply_on_ipad_from_safari/
[Accessed 26 Dec. 2017].
[24] reddit. (2017). Bug with posting, upvoting, etc on Mac Firefox • r/bugs. [online] Available at:
https://www.reddit.com/r/bugs/comments/7iucak/bug_with_posting_upvoting_etc_on_mac_firefox/
[Accessed 26 Dec. 2017].
[25] GIPHY. (2017). Reddit GIF - Scrollbar bug. [online] Available at:
https://giphy.com/gifs/xUOxf0y5jeJh3wB0Iw/html5 [Accessed 26 Dec. 2017].
[26] reddit. (2017). All javascript not working in chrome • r/bugs. [online] Available at:
https://www.reddit.com/r/bugs/comments/69y8wt/all_javascript_not_working_in_chrome/ [Accessed 26
Dec. 2017].
[27] Wave.webaim.org. (2017). WAVE Help. [online] Available at: http://wave.webaim.org/help [Accessed
26 Dec. 2017].
[28] NV Access. (2017). NV Access. [online] Available at: https://www.nvaccess.org/ [Accessed 26 Dec.
2017].
[29] ATbar. (2017). About. [online] Available at: https://www.atbar.org/about/ [Accessed 26 Dec. 2017].
[30] Validator.w3.org. (2018). The W3C Markup Validation Service. [online] Available at:
https://validator.w3.org/ [Accessed 3 Jan. 2018].
Back to top
37 | P a g e
[31] Jigsaw.w3.org. (2018). The W3C CSS Validation Service. [online] Available at:
https://jigsaw.w3.org/css-validator/ [Accessed 3 Jan. 2018].
