Embed Size (px)
Citation preview
MOVE FAST AND FIX TESTS
Shauvik Roy Choudhary @shauvik http://shauvik.com
GOING FROM MANUAL TO AUTOMATED:
MOVE FAST AND FIX TESTS
Shauvik Roy Choudhary @shauvik http://shauvik.com
GOING FROM MANUAL TO AUTOMATED:
MOVE FAST AND FIX TESTS
Shauvik Roy Choudhary @shauvik http://shauvik.com
GOING FROM MANUAL TO AUTOMATED:
Checks
ABOUT ME• PhD Thesis:
Cross-platform Testing & Maintenance of Web & Mobile Apps
• Industry: Google, Yahoo!, FullStory, Fujitsu Labs, IBM Research, HSBC, Goldman Sachs
• Entrepreneurship: Georgia Tech TI:GER program, VentureLab, ATDC, TechSquare Labs
Shauvik Roy Choudhary PhD, Georgia Tech
Founder, CheckDroid
OUTLINE• MOTIVATION: Why Automate ?
• MOVE FAST: How to Automate Quickly?
• FIX TESTS: How to Resolve Automation Issues?
• SCENARIO: Android UI Testing
• DISCUSSION
WHY AUTOMATE TESTS?
Software
Manual Testing
WHY AUTOMATE TESTS?
Software
Manual Testing
WHY AUTOMATE TESTS?
Software
Manual Testing
WHY AUTOMATE TESTS?
Software
Manual Testing
WHY AUTOMATE TESTS?
Software
Manual Testing
PLATFORMS
Web Tester
PLATFORMS
Web Tester
PLATFORMS
Web Tester
PLATFORMS
Web Tester
PLATFORMS
Web Tester
PLATFORMS
Web Tester
PLATFORMS
Web Tester
PLATFORMS
Web Tester
Mobile Tester
LollipopAndroid 5.0
9
HOW TO AUTOMATE?
HOW TO AUTOMATE?
WHAT
Parts of your App
HOW TO AUTOMATE?
HOW
Infrastructure
Tool Support
WHAT
Parts of your App
WHAT: TO TEST
• Functional Tests Positive and negative scenarios
• Non-Functional Tests Performance, Security, Compatibility…
• Previous Issues (identified Manually) Never to occur in Regressions
HOW: INFRASTRUCTURE
• Unit TestsxUnit for different programming languages
• UI TestsSelenium/WebDriver for Web Apps Appium for Mobile (iOS, Android)Espresso for Android
IDEAL TESTING PYRAMIDby Mike Cohn
ICE-CREAM CONE
HOUR GLASS
source: just-about.net
TESTING DIAMOND
Source: toddlittleweb.com
CHRISTMAS TREE
source: just-about.net
UI TESTS CLOSEST TO
END-USER TESTING
MOVE FASTHow to automate quickly?
RELIABLE INFRASTRUCTURE
VISUAL TEST RECORDING
FIX TESTSHow to fix broken tests?
UI TESTING ISSUES
UI TESTING ISSUES
• FRAGILITY
UI TESTING ISSUES
• FRAGILITY
• SPEED
UI TESTING ISSUES
• FRAGILITY
• SPEED
• FLAKINESS
1. FRAGILE TESTS
REDUCING FRAGILITY
• Use durable selectors
• IDs over XPath
• Relative XPaths over Absolute XPaths
• Other durable labels (class, content-desc..)
2. SLOW TESTS
MAKING TESTS FAST
• Don’t use Sleeps
• If absolutely necessary, sleep while polling
• Replace slow resources with stubs
doSomething()Thread.sleep(1000) checkSomething()
doSomething()do{ Thread.sleep(1000) }while(!loaded) checkSomething()
FLAKINESSUI Thread
Test Thread
Work in Queue
FLAKINESSUI Thread
Test Thread
Work in Queue
click
FLAKINESSUI Thread
Test Thread
Work in Queue
click
Motion down
Motion up
FLAKINESSUI Thread
Test Thread
Work in Queue
click
Motion down
Motion up
assert
FLAKINESSUI Thread
Test Thread
Work in Queue
click
Motion down
Motion up
assert
FLAKINESSUI Thread
Test Thread
Work in Queue
click
Motion down
Motion up
assertsleep
FLAKINESSUI Thread
Test Thread
Work in Queue
click
Motion down
Motion up
assertsleep
TESTS
3. FLAKY TESTS
Image credits: Google, GTAC talk
REDUCING FLAKINESS• Have a hermetic test environment
Source: googletesting.blogspot.com
REDUCING FLAKINESS• Have a hermetic test environment
Source: googletesting.blogspot.com
FIX TESTS
1. FRAGILE TESTS
FIX TESTS
1. FRAGILE TESTS
2. SLOW TESTS
FIX TESTS
1. FRAGILE TESTS
2. SLOW TESTS
3. FLAKY TESTS
Image credits: Google, GTAC talk
FIX TESTS
ANDROID UI TESTING
MOBILE TESTING PYRAMID
Source: Daniel Knott, AdventuresInQAAuthor: Hand-On Mobile App Testing
ANDROID TESTING TOOLS
Image credits: BitBar
ESPRESSO: HOW IT WORKSUI Thread
Test Thread
Work in Queue
Test Case
Motion down
Motion upEspresso
action assert
Blocked
ESPRESSO METHODS
onView(Matcher) .perform(ViewAction) .check(ViewAssertion)
EXAMPLES
onView(withId(R.id.greeting)) .perform(click());
onView(withText("Hello Steve!")) .check(matches(isDisplayed()));
EXAMPLES
onView(withId(R.id.greeting)) .perform(click());
onView(withText("Hello Steve!")) .check(matches(isDisplayed()));
Find
EXAMPLES
onView(withId(R.id.greeting)) .perform(click());
onView(withText("Hello Steve!")) .check(matches(isDisplayed()));
Find
Do Stuff
EXAMPLES
onView(withId(R.id.greeting)) .perform(click());
onView(withText("Hello Steve!")) .check(matches(isDisplayed()));
Find
Do Stuff
Check
onView(Matcher<View>)
● withId ● withText ● withContentDescription ● isDisplayed ● hasFocus ● hasSibling ● ... ● mySpecialMatcher
perform(ViewAction)
● click ● longClick ● doubleClick ● typeText ● scrollTo ● ... ● myCustomAction
check(ViewAssertion)
● matches(...) ● doesNotExist ● myCustomAssertion
HIERARCHYwithParent(Matcher)withChild(Matcher)hasDescendant(Matcher)isDescendantOfA(Matcher)hasSibling(Matcher)isRoot()
UI PROPERTIESisDisplayed()isCompletelyDisplayed()isEnabled()hasFocus()isClickable()isChecked()isNotChecked()withEffectiveVisibility(…)isSelected()
ROOT MATCHERSisFocusable()isTouchable()isDialog()withDecorView(…)isPlatformPopup()
COMMON HAMCRESTMATCHERSallOf(Matchers)anyOf(Matchers)is(...)not(...)endsWith(String)startsWith(String)
SEE ALSOPreference matchersCursor matchers
USER PROPERTIESwithId(…)withText(…)withTagKey(…)withTagValue(…)hasContentDescription(…)withContentDescription(…)withHint(…)withSpinnerText(…)hasLinks()hasEllipsizedText()hasMultilineText()
INPUTsupportsInputMethods(…)hasImeAction(…)
CLASSisAssignableFrom(…)withClassName(…)
MatchersCLICK/PRESSclick()doubleClick()longClick()pressBack()pressImeActionButton()pressKey([int/EspressoKey])pressMenuKey()closeSoftKeyboard()openLink(…)
GESTURESscrollTo()swipeLeft()swipeRight()swipeUp()swipeDown()
TEXTclearText()typeText(String)typeTextIntoFocusedView(String)replaceText(String)
POSITION ASSERTIONSisLeftOf(Matcher)isRightOf(Matcher)isLeftAlignedWith(Matcher)isRightAlignedWith(Matcher)isAbove(Matcher)isBelow(Matcher)isBottomAlignedWith(Matcher)isTopAlignedWith(Matcher)
LAYOUT ASSERTIONSnoEllipsizedText(Matcher)noMultilineButtons()noOverlaps([Matcher])
View Actions
matches(Matcher)doesNotExist()selectedDescendantsMatch(…)
View Assertions
onView(Matcher) .perform(ViewAction) .check(ViewAssertion)
CHEAT SHEET
2.0
DEMO
Tests
TEST CLOUD
WHY AUTOMATE TESTS?
Software
Manual Testing
WHY AUTOMATE TESTS?
Software
Manual Testing
MOVE FASTHow to automate quickly?
WHY AUTOMATE TESTS?
Software
Manual Testing
MOVE FASTHow to automate quickly?
1. FRAGILE TESTS
2. SLOW TESTS
3. FLAKY TESTS
Image credits: Google, GTAC talk
FIX TESTS
WHY AUTOMATE TESTS?
Software
Manual Testing
MOVE FASTHow to automate quickly?
1. FRAGILE TESTS
2. SLOW TESTS
3. FLAKY TESTS
Image credits: Google, GTAC talk
FIX TESTSANDROID UI TESTING
DISCUSSIONAre you currently automating tests?
What are some challenges that you see?
