W14

Test Techniques

10/15/2014 3:00:00 PM

Testing the New Disney World

Website

Presented by:

Leslie Honniball

Walt Disney Parks and Resorts Technology

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Les Honniball

Walt Disney Parks and Resorts Technology As manager of quality assurance for Walt Disney Parks and Resorts Technology, Les Honniball leads a team of QA engineers responsible for testing the web sites of Walt Disney World Resort in Florida, Disneyland Resort in California, the Disney Cruise Line, and the Walt Disney Paris and Hong Kong theme parks and resorts. A twenty-three year veteran of the testing field with experience working for companies large and small, Les has worked with remote QA teams in India, the Philippines, Argentina, and the US using the agile development process. Les works closely with various automation teams at Disney to help define the core automation tests needed to launch a successful website.

Introduction

• Testing the web user interface for a large high-volume global web-site has a unique and complex set of challenges

• Testing the content, meta tags, pages, products and url’s requires attention to detail and careful consideration of changes

• Testing a technical and functional software platform that can handle all of this is: – Complicated – Challenging – Multi-disciplinary – FUN!

Agenda

• Challenges

• Pages, URLs & SEO

• Analytics

• User Experience (UX)

• Designing a global Web Platform

• Testing a global Web Platform

• Take-aways

Unique Challenges for Public Sites Pages, SEO and URLs

Analytics, pages and page flows

Going global: i18n + channel + market + store

A/B Testing and revenue optimization

Availability (someone is always browsing): - 24x7x365 - Rolling deployments - Reduced functionality when system x down

User Experience & Accessibility

Some Characteristics of an SEO friendly site

Well known pages with targeted content for search engines that will resolve in multiple languages

Well-formed URL’s that can be easily bookmarked, indexed by crawlers and mapped onto pages/channels/stores/languages/markets

Make sure each major searchable entity has a page indexed for search

If you are international, Google is not the only search engine (Yahoo is bigger than Google in Japan)

Mapping page URLs to Language and Region

Analytics

Analytics

• Small changes in our commerce flows can affect conversion rates and revenue

• Revenue Management and Optimization for online channel requires extensive tracking and analysis of users as they interact with the website

UX (User Experience)

• Our Guests deserve the best

• Comps, Approvals, Feedback

• Responsive? Adaptive? Redaptive?

Testing a Global Web Platform

a Multi-Brand Multi-Lingual Web Platform requires:

• Site (URL) Management & determining a users context

• Personalized Pages

• Context-aware services

• Application Configurability

• Technology Platform

Determining a Guests Context

A guest’s context Destination Market/Affiliation Guest Store/Channel Pass-holder?

A guest’s context

Personalized, targeted Web Site

Offer 1 Offer 2 Offer n

Web Store 1

Web Store 2

Web Store n

Content a

Content b

Content z

QA for the Global Web Platform

• Test management

• Multi-device/Browser/OS testing

• Automation

Test management & Process

The following process and tools are used for managing a Sprint:

• Agile Development Process

• JIRA for entry and tracking

• ALM

• Google Doc’s

Testing Approach

• Testing begins at the Studio level in the Latest environment with Acceptance and Functional testing performed during the sprint. Once promoted to the Stage environment, initial smoke testing will be conducted by Studio Testers and the Central QA Team.

• Central QA Team may also perform additional ad-hoc tests not detailed in the plan but must remain relevant and within the scope of the project, and done only when all prioritized test scenarios have passed.

• Progress will be communicated based on the percentage of executed test cases and other relevant testing activities. Test results will be captured and stored in HP ALM.

Defect Life Cycle

Defect found by

Tester

Defect created in

JIRA and assigned

to PDM

Defect Prioritized

by PDM

Defect

prioritized for

current

release?

Defect assigned to

Studio backlog

Defect resolved by

Development

Defect verified

by Tester

No

Yes

No

Defect closed by

Tester

Yes

Multi-device/Browser/OS testing

• PC Browsers:

• Chrome (Windows 7)

• Firefox (Windows 7) – All test automation runs primarily on this platform

• IE8 (Windows 7)

• IE9 (Windows 7)

• IE10 (Windows 7)

• Mac Chrome (OS X Lion)

• Mac Safari (OS X Lion)

Multi-device/Browser/OS testing • Mobile Devices:

• iPad (iOS6 / iOS7)

• iPad Mini (iOS7)

• iPhone 4s (iOS6)

• iPhone 5 (iOS6 / iOS7)

• iPod Touch (iOS6)

• HTC One S (Android 4.0.4)

• HTC One X (Android 4.0.3)

• HTC 5435 (Android 4.1.1)

• DNA HTC (Android 4.1.1)

• Galaxy S3 (Android 4.1.2)

• Galaxy S4 (Android 4.2.2 / 4.3)

• Nexus 7 (Android 4.3)

Test Automation

Test Management and Reporting

In Closing

• Testing the user interface for a large high-volume global web-site has a unique and complex set of challenges: – We have to design,code and Test for everyone who wants

to access the site(s) from anywhere on any channel/device – Small changes have large impacts (page flows, UX …)

• Testing a technical and functional software platform that can handle all of this is: – Complicated – Challenging – Multi-disciplinary – FUN!

Q&A