Using Acceptance Tests to Validate Accessibility Requirements in RIA

Using Acceptance Tests to Validate

Accessibility Requirements in RIA

Willian Massami WatanabeRenata Pontin de Mattos Fortes

Ana Luiza Dias

9th International Cross-Disciplinary Conference on Web Accessibility – 16/17th April 2012 – Lyon, France.

Agenda

•Research context

•State of art

•Hypothesis

•Acceptance Tests

•Preliminary results

•Advantages and limitations

•Next steps

Agenda

•Research context

•State of art

•Hypothesis

•Acceptance Tests

•Next steps

Research Context

Web 2.0

Perpetual Perpetual betabeta

Research Context

• New features• Web applications

improvements

• New features• Web applications

improvements

Research Context

Less than an hourLess than an hour

Research Context

RequirementsRequirements

DesignDesign

ImplementationImplementation

TestsTests

DeploymentDeployment

Research Context

Continous Integration

Continuouslly compile and test the software to ensure quality

[Kim et al. 2008]

Research Context

Continuous Integration

Developers

Research Context

Developers

Source code repository

Research Context

Developers

• Unit testsUnit tests

• Integration Integration

teststests

• CompileCompile

• Acceptance Acceptance

teststests

• CompileCompile

teststests

Report

Research Context

Developers

teststests

• CompileCompile

teststests

• CompileCompile

teststests

Report

Research Context

Developers

teststests

• CompileCompile

teststests

• CompileCompile

teststests

Report

Research Context

RequirementsRequirements

DesignDesign

ImplementationImplementation

TestsTests

DeploymentDeploymentQuality

Research Context

How to evaluate accessibility?

TestsTests

Research Context

•User testing [Romen e Svanaes, 2008]

•WCAG conformance [W3C, 2008]

•Barrier walkthrough [Brajnik, 2006]

•Automatic accessibility tests

Research Context

•User tests

•WCAG conformance

•Barrier walkthrough

•Automatic accessibility tests

TestsTests

Research Context

Agenda•Research context

•State of art

•Hypothesis

•Acceptance Tests

•Next steps

State of art

JavaScript

RIARIA

State of art

HTML ParserHTML Parser

Static HTML content analysis

[Velasco et al. 2008]

State of art

JavaScript CSS

Not evaluatedNot evaluated

Agenda

•Research context

•State of art

•Hypothesis

•Acceptance Tests

•Next steps

Hypothesis

RIARIA

Hypothesis

RIARIAA

HypothesisContinuous Integration

Developers

teststests

• CompileCompile

teststests

• CompileCompile

teststests

Report

Agenda

•Research context

•State of art

•Hipothesis

•Acceptance Tests

•Next steps

Acceptance Test

Definition

“A formal test that determines if a system satisfies its acceptance criteria and allow the user to identify when a

system will be accepted or not.”

Behaviour Acceptance tests

•Assertions made against the external interface

•Test cases reflects User Stories

Acceptance Test

Focus management [W3C, 2011]

Acceptance Test

TAB key navigation to the element

containing the text “Sports”

Acceptance Test

containing the text “Tennis”

Acceptance Test

Agenda

•Research context

•State of art

•Hipothesis

•Acceptance Tests

•Next steps

flyout menu

accordion menuoverlay menu

tabbed menu

flyout menu

accordion menuoverlay menu

tabbed menu

[Welie, 2008]

Preliminary results

Acceptance tests

HTML validators

All web tools that test navigation accessibility requirementshttp://www.w3.org/WAI/ER/tools/

complete

Cynthia

DaSilva

EvalAccess

Accessibility check

Achecker

FAE report

Preliminary results

HTML validators

http://www.w3.org/WAI/ER/tools/complete

Cynthia

DaSilva

EvalAccess

Accessibility check

Achecker

FAE report

Preliminary results

All web tools that test navigation accessibility requirementsHTML

validatorsHTML

validators

HTML validators

Acceptance tests

Preliminary results

Our hypothesis

DaSilva

EvalAccess

FAE report

HTML ValidatorsHTML ValidatorsAcceptance

testsAcceptance

Preliminary results

testsAcceptance

Projects that present defects that were not identified by

the Tool

Projects that present defects that were not identified by

the Tool

Preliminary results

Developers

teststests

• CompileCompile

teststests

• CompileCompile

teststests

Report

Preliminary results

Integração Contínua

Developers

• Integraiton Integraiton

teststests

• CompileCompile

teststests

• Integraiton Integraiton

teststests

• CompileCompile

teststests

Report

Projects that might be put in production event though they

contain accessibility errors

Projects that might be put in production event though they

contain accessibility errors

Preliminary results

testsAcceptance

Projects that present no accessibility errors

however are not put into production

Projects that present no accessibility errors

however are not put into production

Preliminary results

Developers

teststests

• CompileCompile

teststests

• CompileCompile

teststests

Report

Preliminary results

Integração Contínua

Developers

• Unit tetsUnit tets

TestesIntegration TestesIntegration

teststests

• CompileCompile

teststests

• Unit tetsUnit tets

TestesIntegration TestesIntegration

teststests

• CompileCompile

teststests

Report

Prevent the deployment of a stable

release of the software

Prevent the deployment of a stable

release of the software

Preliminary results

testsAcceptance

HTML Validators HTMLHTML Validators HTMLAcceptance

testsAcceptance

testsWarnings onlyWarnings only

Preliminary results

Validadores HTMLValidadores HTMLAcceptance

testsAcceptance

Do not make distinction between interface components that were

accessible and the ones that were not

Do not make distinction between interface components that were

accessible and the ones that were not

Agenda

•Research context

•State of art

•Hipothesis

•Acceptance Tests

•Next steps

Advantages

•In the experiment, the use of acceptance tests presented a higher number of correct assertions

•Higher effort required in order to execute the tests

•Assertions made agains the DOM structure inside of an use context makes it possible to evaluate ARIA requirements

Limitation

•The limitation of this hyphotesis is the higher cost to describe all contexts of the users

Agenda

•Research context

•State of art

•Hipothesis

•Acceptance Tests

•Next steps

•Implement new use scenarios for the acceptance tests

•Evaluate ARIA requirements

Next steps

REFERENCES

• Seojin Kim, Sungjin Park, Jeonghyun Yun, and Younghoo Lee. 2008. Automated Continuous Integration of Component-Based Software: An Industrial Experience. In Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering (ASE '08). IEEE Computer Society, Washington, DC, USA, 423-426. DOI=10.1109/ASE.2008.64 http://dx.doi.org/10.1109/ASE.2008.64

• ROMEN, D.; SVANAES, D. Evaluating web site accessibility: validating the wai guidelines through usability testing with disabled users. In: NordiCHI ’08: Proceedings of the 5th Nordic conference on Human-computer interaction, New York, NY, USA: ACM, 2008, p. 535–538.

• W3C Conformance evaluation of web sites for accessibility. 2008a.Available at: http://www.w3.org/WAI/eval/conformance.html

REFERENCES

• BRAJNIK, G. Web accessibility testing: When the method is the culprit. In: Computers Helping People with Special Needs, Springer Berlin / Heidelberg, 2006, p. 156–163 (Lecture Notes in Computer Science, v.4061).Available at: http://www.springerlink.com/content/ h2573t61gu3n8533/

• C. A. Velasco, D. Denev, D. Stegemann, and Y. Mohamad. A web compliance engineering framework to support the development of accessible rich internet applications. In Proceedings of the 2008 international cross-disciplinary conference on Web accessibility (W4A), W4A ’08, pages 45–49, New York, NY, USA, 2008. ACM.

• W3C Accessible Rich Internet Applications (WAI-ARIA) 1.0. 2011. Available at: http://www.w3.org/TR/wai-aria/.

• Welie. 2008. Available at: http://www.welie.com/patterns/.

http://www.pyccuracy.orghttps://github.com/watinha/Pyccuracy-Accessibility-

Actions

Thank you very much!

Contacts:watanabe

willian@yahoo.com

anadias@icmc.usp.br

renata@icmc.usp.br

9th International Cross-Disciplinary Conference on Web Accessibility – 16/17th April 2012 – Lyon, France.

http://watinha.com/pyccuracy_test_1/templates/

Using Acceptance Tests to Validate Accessibility Requirements in RIA

Technology

RIA Tutorial.pdf

Ria Avenue

RIA Document

Radioimmunoassay (RIA) - Freemicrobiology.free.fr/Presentations/RIA.pdf · Radioimmunoassay (RIA) • Radioimmunoassays (RIA) are assays which are based on the measurement of radioactivity

RIA Presentation

RIA Dyes & Chemicals Co, Ahmedabad, RIA 'HE' Dyes

Regulatory Impact Analysis (RIA) · Regulatory Impact Analysis (RIA) Public Health (Alcohol) Bill 1 Summary of RIA A summary of the RIA which includes Policy Options, Costs, Benefits

Validate your

RIA material

RIA and the DOE/NNSA National Security Missionmantica/radio-ria/kreisler_radioria.pdf · RIA and the DOE/NNSA National Security Mission Radiochemistry at RIA American Chemical Society,

Ria Enterprise

Tributa Ria

An Evaluation of RIA Frameworkspaginas.fe.up.pt/~dsie10/presentations/session 6/An...RIA Concepts RIA framework: The tools and environments that allow the development of RIA. •Examples:

Yahoo! Experiences with Accessibility in DHTML and RIA Nate Koechley Senior Engineer & Designer, Yahoo! User Interface (YUI) Library Team Platform Engineering

RADIOIMMUNOASSAY (RIA)

Designing RIA Accessibility: A Yahoo UI (YUI) Menu Case Study · Flash, JavaScript, Java ... •Allows ARIA on top of RIA. 46 Faithful and Predictable Ports: Benefits •More options

RIA implementation patterns

Taller Ria

RIA Security

RIA Manual