Julien Bensaid Vice President & Head of Global Testing Practice , Atos

Marta Sanfeliu Chief Integrator, Winter Olympics Sochi General Manager, Olympics for Atos

Speakers

Barcelona 1992 • 3 Billion Television viewers • 169 Participating Nations

London 2012 • 8 Billion Connected Devices • 204 Participating Nations

…from a single physical event in ‘92 to the most connected event in history

Rio 2016 • Cloud based infrastructure • 206 Qualified Nations

Evolution of the Games…

Scale - Rio 2016

4 BN Worldwide viewers

37 Competition venues

300,000 Accreditations

30,000 Media Personnel

15,000+ Athletes

3,500 Technology Staff

70,000 Volunteers

200,000 Testing Hours

80 Applications & Systems

500 Servers

>$500M Technology Budget

Total Security Events

Athens ‘04

Beijing ‘08

London ‘12

Sochi ‘14

80 M

201 M

255 M

322 M

219 Security Events Every Second at Sochi !

Culture

Procedures

Language Laws

Location People

• Fixed price contract with the IOC

• Multi-year games contract

• Dealing with many unknowns

• No option of missing deadlines

• 302 events timed to a fraction of a second

• Readiness includes systems & people

• Zero Email Initiative

• Use of Social Media – BlueKiwi

• Dedicated Technical Operation Center

• Alerts and Communication drills

• Global delivery teams in 13 locations

• Multi language adaptation

• Bridging cultures

November 2012 The first team member arrived in Rio after London

2016

2012 2013 2014 2015 2016

Olympics 5 – 21 Aug 2016 Paralympics 7 – 18 Sept. 2016

Our Countdown to Rio 2016

200,000 hours of testing

5 Testers

50-80 Testers

200 Testers

Onsite Staff

A dedicated 1000 sq meters (10,800 sq feet) test lab for the Olympics

54 Test Cells – 36 for sports, cells for S/W distribution & Games Management

Tests by local teams where the event is going to be hosted

Ability to run applications in “Shadow” mode during trials

Real sport events and Shadow events to test performance of people and systems

How do you simulate real life situations during testing?

Hybrid methodology - Agile for development and Waterfall for Integration

9000 test cases grouped by sports, events and systems. E.g. Track and Field has 700 Test Cases

Tools – ALM, Performance Center, UFT, Acunetix (Security)

Virtualization for external interfaces

A script-less Hybrid Automation framework for UFT increases reusability

What tools and processes are used to deliver a zero defect games?

• User Acceptance Tests

• Homologation Tests

• Multi Sport Tests

• Technical Rehearsals

• Delivery Acceptance Test

• End To End Test

• Technical Test

• Test Events

Types of Testing Conducted

Tests across iOS, Android & Windows platforms

Predefined checklist for navigation, visibility by resolution, need for scrolling etc.

Usability tests by sports and events specialist

Usability and Performance deviations are treated as defects

Platforms developed in HTML 5 & CSS 3 for increased user responsiveness

How do you ensure consistent user experience for 4 Billion viewers across devices and operating systems?

A core team with previous Olympics experience. Testers chosen via a rigorous selection process

Sports specialists, Major event technologists and Language experts part of the testing team

Reusable Test Scenario Repository to reduce effort for test design

A structured SIAM process to interface with 3rd parties and external agencies

Continuous learning and retrospection

How do you ensure quality with a new 200+ team every 2 years?

Test Factory Approach

It’s not just the Olympics that Test in the Extremes

Create Extreme Security focus by simulating breaches and disaster scenarios in Security Testing labs to provide peace of mind in sharing data

Extreme Reliability through enterprise grade Performance Testing toolsets, assuring 100% uptime for mission critical applications to protect your brand

Leverage tools, labs, utilities & partners - e.g. in Service Virtualization - to create Extreme Agility and lower TCO