XCODE: AUTO TESTPhineas

ABOUT AUTO TEST

ABOUT AUTO-TEST

Why?

If we're testing internals we should probably test what user could see.

UI is the first thing users that see, so it is more important than how it does whatever.

Hard to forget to test something.

Much expensive, time, labor, money, boring, etc.

■ Logic tests. These tests check the correct functionality of a unit of code by itself (not in an app). With logic tests you can put together specific test cases to exercise a unit of code. You can also use logic tests to perform stress-testing of your code to ensure that it behaves correctly in extreme situations that are unlikely in a running app. These tests help you produce robust code that works correctly when used in ways that you did not anticipate.

■ Application tests. These tests check units of code in the context of your app. You can use application tests to ensure that the connections of your user-interface controls (outlets and actions) remain in place, and that your controls and controller objects work correctly with your object model as you work on your app. You can also use these tests to perform hardware testing, such as getting the location of the device on which your app is running.

UNIT TEST

TEST SCRIPT

INITIAL UNIT TEST

1. Static function will be executed once

2. General function always run at each test

XCTESTCASE

■ Fundamental Test XCTAssert(expression, format...)

■ Boolean Tests XCTAssertTrue(expression, format...) XCTAssertFalse(expression, format...)

■ Equality Tests XCTAssertEqual(expression1, expression2, format...) XCTAssertNotEqual(expression1, expression2, format...) XCTAssertEqualWithAccuracy(expression1, expression2, accuracy, format...)

XCTAssertNotEqualWithAccuracy(expression1, expression2, accuracy, format...)

■ Nil Tests XCTAssertNil(expression, format...) XCTAssertNotNil(expression, format...)

■ Unconditional Failure XCTFail(format...)

XCTESTCASE

PERFORMATION TESTING

■What are the absolute performance characteristics of

this code? Is the procedure bound by computation or

memory? What is the limiting behavior across different

sample sizes?

■What are the relative performance characteristics of this

code, as compared to its alternatives? Which is faster,

methodA or methodB?

CFTimeInterval startTime = CACurrentMediaTime(); {

. . . . . . . } CFTimeInterval endTime = CACurrentMediaTime(); NSLog(@"Total Runtime: %g s", endTime - startTime);

uint64_t t = dispatch_benchmark(iterations, ^{ @autoreleasepool { ……… } }); NSLog(@"[[NSMutableArray array] addObject:] Avg. Runtime: %llu ns", t);

measureBlock() { ……… }

■A performance test takes a block of code that you want to evaluate and runs it ten times, collecting the average execution time and the standard deviation for the runs. These statistics combine to create a baseline for comparison, a means to evaluate success or failure. To implement performance measuring tests, you write methods using new API from XCTest in Xcode 6 and later.

Test Suite 'Selected tests' started at 2014-12-11 01:21:27 +0000 Test Suite 'testerTests.xctest' started at 2014-12-11 01:21:27 +0000 Test Suite 'testerTests' started at 2014-12-11 01:21:27 +0000 Test Case '-[testerTests testPerformanceExample]' started. 2014-12-11 09:21:27.328 tester[724:13120] test start /Users/sunxiaoshan/Desktop/Report/XCodeTest/tester/tester/testerTests/testerTests.m:164: Test Case '-[testerTests testPerformanceExample]' measured [Time, seconds] average: 0.003, relative standard deviation: 3.742%, values: [0.002862, 0.002867, 0.002876, 0.003214, 0.003048, 0.002881, 0.002917, 0.002864, 0.002859, 0.002884], performanceMetricID:com.apple.XCTPerformanceMetric_WallClockTime, baselineName: "Local Baseline", baselineAverage: 1.810, maxPercentRegression: 10.000%, maxPercentRelativeStandardDeviation: 10.000%, maxRegression: 0.100, maxStandardDeviation: 0.100 2014-12-11 09:21:27.705 tester[724:13120] test end Test Case '-[testerTests testPerformanceExample]' passed (0.378 seconds). Test Suite 'testerTests' passed at 2014-12-11 01:21:27 +0000. Executed 1 test, with 0 failures (0 unexpected) in 0.378 (0.379) seconds Test Suite 'testerTests.xctest' passed at 2014-12-11 01:21:27 +0000. Executed 1 test, with 0 failures (0 unexpected) in 0.378 (0.379) seconds Test Suite 'Selected tests' passed at 2014-12-11 01:21:27 +0000. Executed 1 test, with 0 failures (0 unexpected) in 0.378 (0.381) seconds

XCTESTEXPECTATION

XCTestExpectation *documentOpenExpectation = [self expectationWithDescription:@"document open”];

[documentOpenExpectation fulfill];

[self waitForExpectationsWithTimeout:15 handler:^(NSError *error) { if (error != nil) { XCTFail(@"timeout error: %@", error); } }];

UI AUTOMATION TEST

How to do it

XCode tool: Automation

Script Language : JavaScript

How to get object

window.buttons()[0]

window.buttons()[“login”]

HOW TO START

HOW TO START (I)

Create a new trace document in Instruments using the Automation trace template.

HOW TO START (II)

Click Add > Create.

HOW TO START (III)

In the Detail pane Navigation bar, select Script to enter the code for your script.

VIEW STRUCTURE

ELEMENT HIERARCHY (I)

UIATarget.localTarget()

frontMostApp()

https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html

ELEMENT HIERARCHY (II)

UIATarget.localTarget()

frontMostApp()

mainWindow()

https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html

ELEMENT HIERARCHY (III)

UIATarget.localTarget()

frontMostApp()

mainWindow()

tableViews()[0]

https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html

ELEMENT HIERARCHY (IV)

UIATarget.localTarget()

frontMostApp()

mainWindow()

tableViews()[0]

cells()[6]

https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html

ELEMENT HIERARCHY (V)

UIATarget.localTarget()

frontMostApp()

mainWindow()

tableViews()[0]

cells()[6]

elements()["Name"]

https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html

ELEMENT HIERARCHY (VI)

https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html

INTERACTION

INTERACTION

element.tap()

touchAndHold()

doubleTap()

scrollToVisible()

captureScreenWithName()

JENKINS

It’s a tool that can manager projects easily and write the custom script quickly by yourself.

JENKINS PLUGIN

NEW PROJECT

XCTOOL COMMAND LINE

REFERENCE

REFERENCE

https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html

https://developer.apple.com/library/ios/documentation/DeveloperTools/Reference/UIAutomationRef/_index.html#//apple_ref/doc/uid/TP40009771

http://blog.manbolo.com/2012/04/08/ios-automated-tests-with-uiautomation

http://justbm.blogspot.tw/2011/12/my-ios-ui-automation-testing.html

http://www.slideshare.net/DavidReidy/automated-ui-testing

http://nshipster.com/xctestcase/

https://developer.apple.com/Library/ios/documentation/DeveloperTools/Conceptual/testing_with_xcode/testing_3_writing_test_classes/testing_3_writing_test_classes.html

https://developer.apple.com/legacy/library/documentation/DeveloperTools/Conceptual/UnitTesting/01-Unit-Test_Overview/overview.html

https://developer.apple.com/library/IOs/documentation/DeveloperTools/Conceptual/testing_with_xcode/testing_4_running_tests/testing_4_running_tests.html

http://jenkins-ci.org/