Quality comes free with open source testing tools

Quality comes free! with Open Source ToolsSteven Mak

Sunday, 8 September, 13

Who am I?

• Agile, TDD Coaching, Ugly Code Cleaning Dude• I love coding - Java, C#, Javascript, C/

C++, PHP, Perl, and some weird ones• I speak English, Cantonese, and

Mandarin

2

Odd-e Pte. Ltd.Steven Mak 麥天志Agile CoachHong KongEmail: [email protected]: www.odd-e.comTwitter: stevenmak


mailto:[email protected]


http://www.odd-e.com


What kind of tests in place?

3


Why do you want these tests?

4


Unit Test Example

5

• Arrange-Act-Assert / Given-When-Then• One assertion per test / Test one behaviour per class

@Test public void makesReservationThroughReservationService() { // ARRANGE final Reservation reservation = new Reservation(); Reservations reservations = stubReservationsToReturn(reservation); HouseKeeping houseKeeping = stubHouseKeeping(); HotelRoom room = new HotelRoom(reservations, houseKeeping);

// ACT Reservation reservationReturned = room.bookFor(new Customer());

// ASSERT assertSame(reservation, reservationReturned); }


Four-Phase Test Pattern

6

Test

Setup

Execute

Verify

Clean-up

Also called: Arrange-Act-Assert



6

Test

Setup

Execute

Verify

Clean-up

Might happen partially in the test

fixture




6

Test

Setup

Execute

Verify

Clean-up




6

Test

Setup

Execute

Verify

Clean-up

Avoid mixing up the Execute and

the Verify



JUnit 4

7

import org.junit.Test;import static org.junit.Assert.*;

public class PersonTest {

@Test public void defaultNameOfThePersonIsJohn() throws Exception { Person john = new Person(); assertEquals("John", john.name()); }}


JUnit 4

7




Import JUnit package


JUnit 4

7





Import assertion methods


JUnit 4

7






Test class containing tests


JUnit 4

7







Annotation for test methods


JUnit 4

7








Test method


JUnit 4

7








Test method

Assertion


Test Fixture

8

import org.junit.After;import org.junit.Before;


Person john = new Person();

@Before public void initializePerson() { john.initialize(); } @After public void cleanUpPerson() { john.clean(); } @Test public void defaultNameOfThePersonIsJohn() throws Exception { assertEquals("John", john.name()); }}


Test Fixture

8





Import Before/After annotations


Test Fixture

8






Run before each test


Test Fixture

8







Run after each test


Test Fixture

8







Data needed in each test.

Run after each test


Test Fixture

8







Data needed in each test.

Run after each test

The data and setup / teardown (before, after) is also sometimes

called: test fixture


JUnit lifecycle

@BeforeClassFor each @TestInstantiate test class@BeforeInvoke the test@After@AfterClass

9


FIRST

10

• Fast• Isolates• Repeatable• Self-validating• Timely

and more...

• Readable• Try: One assertion per test

You still need higher-level tests!


Should be fast, very fast

11


Test Driven Development Why don’t we write code to make test pass?

•The one rule to rule them all-Only ever write code to fix a failing test

•3 Steps:-Write a test (which fails “red”)-Code (to make test pass “green”)-Refactor (test still pass “green”)-Repeat forever

12


TDD Cycle

13

Test

Implement

Design

• Short• Rhythmic• Incremental • Design-focused• Disciplined


Functional leveltest as requirement, requirement as test

14


Use Examples

15

With 3 judges giving scores 4, 20, and 18, the displayed score should be 42.

When the first 2 judges have given their scores, e.g. 10 and 5, the intermediate score of 15 should be displayed already.

No scores displayed as a dash (–), not zero.

Maximum score from a judge is 20 points!


Examples, Tests, and Spec

16

Examples Tests

Requirements

can become

elaborate verify


17

Discussin workshop

Developin concurrence

Deliverfor acceptance

I want to show scores for the current pair.

OK. Can you give me an example of these scores?

So if the three judges gave scores 7, 11 and 2 then the

displayed score should be 20. OK. And would those individual judges' scores be shown?

Or just the total?Yes, individual scores.

And they should be shown all the time, even before all judges have

given their scores.Like if first two judges

give a 7 and an 11 then we'll show 18 as an intermediate score?

Exactly.





As a competition organizerin order to show the judging to the

audience on several monitorsI want to have the scores for the

current pair on a web page.

Scoring




Scoring




Scoring




Scoring


18

The examples are translated into executable specification,allowing a computer to run them against the product.

Discussin workshop




19

Discussin workshop








19

Discussin workshop







Robot tests are written in tables so that computers can read them


It's all in the tables

20


Test Tools

Robot Architecture

21

Test Data (Tables)

Robot Framework

Test Libraries

System Under Test

Test Library API

application interfaces

Robot comes with a number of built-in test libraries and you can (should!) add your own.

Test libraries can use any test tool necessary to interact with the system under test.


Test Cases are composed of keyword-driven actions

22

!"#$%&'()*+%),'-./()0



22

!"#$%&'()*+%),'-./()0

this is the name of a test case



22

!"#$%&'()*+%),'-./()0

this is the name of a test casethese keywords form the test case



22

!"#$%&'()*+%),'-./()0


keywords receive arguments


2 types of keywords

23


2 types of keywords

23

We can import keyword libraries for a test case


2 types of keywords

23


...and libraries may be configured, too.


2 types of keywords

23



This keyword comes from the imported library.


2 types of keywords

23



This keyword comes from the imported library.

This is a user keyword, implemented in table format.(Think macros composed of other macros.)


24

Data-driven test cases


keywords receive arguments


Variables

25

!"#$"%&'(

)#*+,-*++"./,&$.'0

!"#$%&'"(()*+,*%-."/012345167&89:&."(()*+,*%-.";400<=2>6?@89>@."A$B'.C'CD8'A-Sunday, 8 September, 13

Technical Activity

Workflow

Specification pyramid

26

RuleClarity

Stability

Specification

Users can understand

AutomationTechnical


An Example

27


RIDE Project

28


Rule Based Test Case

29

Rule


Workflow Definition

30

Workflow


Technical Activity

31

Technical Activity


Rule Based Test Case

32

Rule


Workflow Definition

33

Workflow


Technical Activity

34

Technical Activity


35

import imaplib

class check_mail: ROBOT_LIBRARY_SCOPE = 'GLOBAL' __version__ = '0.1' error_message = "" def open_mail_box(self, server, user, pwd): try: self.mailbox = imaplib.IMAP4_SSL(server) self.mailbox.login(user, pwd) self.mailbox.select() except imaplib.IMAP4.error: self.error_message = 'Could not connect to the mailbox'

def count_mail(self, *args): r, self.item = self.mailbox.search(None, *args) return len(self.item[0].split())

def get_mail_error_message(self): return self.error_message

def close_mailbox(self): self.mailbox.close() self.mailbox.logout()

Implement a LibraryTechnical Activity


36

public class MailLibrary { public static final String ROBOT_LIBRARY_SCOPE = "GLOBAL"; public static final String ROBOT_LIBRARY_VERSION = "1.0";

private Store store; private Folder folder;

public MailLibrary() { }

public void openMailbox(String host, String username, String password) throws MessagingException { Properties properties = new Properties(); properties.setProperty("mail.imap.socketFactory.class", "javax.net.ssl.SSLSocketFactory"); properties.setProperty("mail.imap.socketFactory.fallback", "false"); properties.setProperty("mail.imap.socketFactory.port", "993");

Session session = Session.getDefaultInstance(properties); store = session.getStore("imap"); store.connect(host, username, password); }

public void checkMailbox() throws MessagingException { FlagTerm flagUnseen = new FlagTerm(new Flags(Flags.Flag.SEEN), false); SearchTerm searchTerm = new AndTerm(flagUnseen, new FromStringTerm("[email protected]"));

folder = store.getFolder("INBOX"); folder.open(Folder.READ_ONLY);

Message messages[] = folder.search(searchTerm);

if (messages.length == 0) { throw new MessagingException("No mail found"); } }

public void closeMailbox() throws MessagingException { if (folder != null) { folder.close(false); } if (store != null) { store.close(); } }}

Technical Activity




Other choices• Cucumber• Fitnesse• Robot Framework

37


38

Sonar


39

What does this mean?


Longitudinal study

40http://almossawi.com/firefox/prose/

Anything happened during some point in time?

• Project deadline?• Firefighting?• Policy change?


http://almossawi.com/firefox/prose/

http://almossawi.com/firefox/prose/

Don’t forget your version control system

41http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=16679


http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=16679

http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=16679

More visualization: JUnit project

42http://dkandalov.github.io/code-history-mining/junit.html

Files often commit together? Is there collective code ownership?Which part of the project

have more attention?

When commit happens? How frequent commit happens?


http://dkandalov.github.io/code-history-mining/junit.html

http://dkandalov.github.io/code-history-mining/junit.html

Thank you for spending time with me this evening.More feedback can be sent to:

43

Odd-e Hong Kong Ltd.Steven Mak 麥天志Agile CoachHong KongEmail: [email protected]: www.odd-e.comTwitter: stevenmak






Technology

Quality comes free with open source testing tools