Embed Size (px)
Citation preview
W10 Test Analytics, AI/ ML Wednesday, May 2nd, 2018 1:45 PM
Machine Learning Heralds the End of Selenium
Presented by:
Jason Arbon
Appdiff, Inc.
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073 888-- -268- - -8770 ·· 904- --278-- -0524 - [email protected] - http://www.stareast.techwell.com/
Jason Arbon Appdiff, Inc. Jason Arbon is the CEO of Appdiff, which is redefining how enterprises develop, test, and ship mobile apps with zero code and zero setup required. He was formerly the director of engineering and product at Applause.com/uTest.com, where he led product strategy to deliver crowdsourced testing via more than 250,000 community members and created the app store data analytics service. Jason previously held engineering leadership roles at Google and Microsoft and coauthored How Google Tests Software and App Quality: Secrets for Agile App Teams.
4/25/18
1
Machine Learning Heralds the End of Selenium
Jason Arbon
#STAREAST #AI @jarbon @testdotai
4/25/18
2
#STAREAST #AI @jarbon @testdotai
4/25/18
3
so 2004
#STAREAST #AI @jarbon @testdotai
4 Big Problems with Selenium / Appium
#STAREAST #AI @jarbon @testdotai
4/25/18
4
You Have To Code
#STAREAST #AI @jarbon @testdotai
Element Selection
#STAREAST #AI @jarbon @testdotai
4/25/18
5
Hard-Coded Step Sequencing
#STAREAST #AI @jarbon @testdotai
Little Reuse
#STAREAST #AI @jarbon @testdotai
4/25/18
6
Automation breaks just when you need it.
#STAREAST #AI @jarbon @testdotai
Why AI?
#STAREAST #AI @jarbon @testdotai
4/25/18
7
People can deal with change and the
unexpected.
#STAREAST #AI @jarbon @testdotai
Test code can’t.
#STAREAST #AI @jarbon @testdotai
4/25/18
8
How does AI fix all this?
#STAREAST #AI @jarbon @testdotai
AI works like People
#STAREAST #AI @jarbon @testdotai
4/25/18
9
You Have To Code
#STAREAST #AI @jarbon @testdotai
Approach #2
Use Machine Learning directly instead of Selectors.
Approach #1
Use Machine Learning to smartly update/repair Selectors when they break.
#STAREAST #AI @jarbon @testdotai
4/25/18
10
Approach #1
VS
Element Identification
Test.ai Auto Labeled
Quick Visual Custom Labeling
Selenium IDE Pain Finding Identifiers Have to write code (brittle)
Old / Past
4/25/18
11
Element Selection
#STAREAST #AI @jarbon @testdotai
4/25/18
12
4/25/18
13
Hard-Coded Step Sequencing
#STAREAST #AI @jarbon @testdotai
VS
Step Sequencing
Selenium IDE Have to code exact steps. Brittle when app changes.
Test.ai Familiar Cucumber/Gherkin Only list labels of interesting
steps and verifications. Also, GUI Editor
Old / Past
#STAREAST #AI @jarbon @testdotai
4/25/18
14
#STAREAST #AI @jarbon @testdotai
VS
Test Execution Old / Past
#STAREAST #AI @jarbon @testdotai
4/25/18
15
Little Reuse
#STAREAST #AI @jarbon @testdotai
AIT - Open - Manual Too
#STAREAST #AI @jarbon @testdotai
4/25/18
16
Write Once, Run Cross Platform
#STAREAST #AI @jarbon @testdotai
Game-Changer: Write Once, Run on All Similar Apps
#STAREAST #AI @jarbon @testdotai
4/25/18
17
Many Companies Working on AI for Testing
#STAREAST #AI @jarbon @testdotai
Harnessing Collective Intelligence For Testing
4/25/18
18
#STAREAST #AI @jarbon @testdotai
Leverage AI
Approach #3
Record and look for
Anomalies
Approach #4
Drag and Drop test
creation. AI executes tests like a human.
Approach #1
Record and auto
repair selectors
Approach #2
Visual Differencing Cross Platform
#STAREAST #AI @jarbon @testdotai
4/25/18
19
Approach #1
Approach #2
4/25/18
20
Approach #3
Approach #4
4/25/18
21
Approach #4
VS
Test Reporting
Selenium IDE Many tests flaky/break as app changes Simple pass/fail
Test.ai Q-Learning test step generation
Element Classifiers at runtime See test steps
Presentable
Old / Past
#STAREAST #AI @jarbon @testdotai
4/25/18
22
#STAREAST #AI @jarbon @testdotai
#STAREAST #AI @jarbon @testdotai
