QA Lab: тестирование ПО. Яков Крамаренко: "KISS Automation"

KISS Automation

@yashakaitlabs.net.ua

AfterwordsThere are good practices in context,

but there are no best practices.

(c) Cem Kaner, James Bach

Afterwords PrefaceThere are good practices in context,

but there are no best practices.

(c) Cem Kaner, James Bach

KISS?

Keep It Simple Stupid!

Web UI Automation…

Selenium vs Wrappers

Wait for text

public class ExamplePage{ @FindBy(css = "#element") WebElement element;}

...

WebDriver driver = new FirefoxDriver();ExamplePage page = PageFactory.initElements(driver, ExamplePage.class);(new WebDriverWait(driver, 6)).until(textToBePresentInElement(page.element, ":-*"));

Wait for text

public class ExamplePage{ @FindBy(css = "#element") WebElement element;}

...

WebDriver driver = new FirefoxDriver();ExamplePage page = PageFactory.initElements(driver, ExamplePage.class);(new WebDriverWait(driver, 6)).until(textToBePresentInElement(page.element, ":-*"));

Wait for text

public class ExamplePage{ @FindBy(css = "#element") WebElement element;}

...

WebDriver driver = new FirefoxDriver();ExamplePage page = PageFactory.initElements( driver, ExamplePage.class);(new WebDriverWait(driver, 6)).until( textToBePresentInElement(page.element, ":-*"));

public class ExamplePage{ @FindBy(css = "#element") WebElement element;}

...

WebDriver driver = new FirefoxDriver();ExamplePage page = PageFactory.initElements( driver, ExamplePage.class);(new WebDriverWait(driver, 6)).until( textToBePresentInElement(page.element, ":-*"));

Wait for text

Wait for text

SelenideElement element = $(“#element"); ...element.shouldHave(text(":-*"));

Not mandatory, still possible…

Not mandatory, still possible…public class ExamplePage{ WebElement element = $(“#element");} ... ExamplePage page = new ExamplePage(); page.element.shouldHave(text(":-*"));

WrappersSelenium

public class ExamplePage{ @FindBy(css = "#element") WebElement element;} ... WebDriver driver = new FirefoxDriver();ExamplePage page = PageFactory.initElements( driver, ExamplePage.class);(new WebDriverWait(driver, 6)).until( textToBePresentInElement(page.element, ":-*"));

SelenideElement element = $(“#element"); ...element.shouldHave(text(":-*"));vs

WrappersSelenium

public class ExamplePage{ @FindBy(css = "#element") WebElement element;} ... WebDriver driver = new FirefoxDriver();ExamplePage page = PageFactory.initElements( driver, ExamplePage.class);(new WebDriverWait(driver, 6)).until( textToBePresentInElement(page.element, ":-*"));

SelenideElement element = $(“#element"); ...element.shouldHave(text(":-*"));vs

WrappersSelenium

Test logicTest logic messed up with tech details

public class ExamplePage{ @FindBy(css = "#element") WebElement element;} ... WebDriver driver = new FirefoxDriver();ExamplePage page = PageFactory.initElements( driver, ExamplePage.class);(new WebDriverWait(driver, 6)).until( textToBePresentInElement(page.element, ":-*"));

SelenideElement element = $(“#element"); ...element.shouldHave(text(":-*"));vs

WrappersSelenium

Test logicTest logic messed up with tech details

public class ExamplePage{ @FindBy(css = "#element") WebElement element;} ... WebDriver driver = new FirefoxDriver();ExamplePage page = PageFactory.initElements( driver, ExamplePage.class);(new WebDriverWait(driver, 6)).until( textToBePresentInElement(page.element, ":-*"));

SelenideElement element = $(“#element"); ...element.shouldHave(text(":-*"));vs

WrappersSelenium

may be SlowerFast

public class ExamplePage{ @FindBy(css = "#element") WebElement element;} ... WebDriver driver = new FirefoxDriver();ExamplePage page = PageFactory.initElements( driver, ExamplePage.class);(new WebDriverWait(driver, 6)).until( textToBePresentInElement(page.element, ":-*"));

SelenideElement element = $(“#element"); ...element.shouldHave(text(":-*"));vs

WrappersSelenium

may be SlowerFast

End to End vs “Unit”

def test_task_life_cycle(): given_at_todomvc() add("a") edit("a", "a edited") toggle("a edited") filter_active() assert_no_tasks() filter_completed() delete("a edited") assert_no_tasks() #...

End to End Scenario

def test_task_life_cycle(): given_at_todomvc() add("a") edit("a", "a edited") toggle("a edited") filter_active() assert_no_tasks() filter_completed() delete("a edited") assert_no_tasks() #...

End to End Scenario

implicit checks

“Unit” / “1-feature-per-test” style

def test_add(): given_at_todomvc() add("a", "b") assert_tasks("a", "b") assert_items_left(2)

“Unit” / “1-feature-per-test” style

def test_add(): given_at_todomvc() add("a", "b") assert_tasks("a", "b") assert_items_left(2)

“Unit” / “1-feature-per-test” styledef test_delete(): given_at_todomvc("a", "b", "c") delete("b") assert_tasks("a", "c") assert_items_left(2)

def test_add(): given_at_todomvc() add("a", "b") assert_tasks("a", "b") assert_items_left(2)

“Unit” / “1-feature-per-test” styledef test_delete(): given_at_todomvc("a", "b", "c") delete("b") assert_tasks("a", "c") assert_items_left(2)

def test_edit(): given_at_todomvc("a", "b", "c") edit("c", "c edited") assert_tasks("a", "b", "c edited") assert_items_left(3)

def test_add(): given_at_todomvc() add("a", "b") assert_tasks("a", "b") assert_items_left(2)

“Unit” / “1-feature-per-test” styledef test_delete(): given_at_todomvc("a", "b", "c") delete("b") assert_tasks("a", "c") assert_items_left(2)

def test_edit(): given_at_todomvc("a", "b", "c") edit("c", "c edited") assert_tasks("a", "b", "c edited") assert_items_left(3)

def test_toggle(): #...

def test_add(): given_at_todomvc() add("a", "b") assert_tasks("a", "b") assert_items_left(2)

“Unit” / “1-feature-per-test” styledef test_delete(): given_at_todomvc("a", "b", "c") delete("b") assert_tasks("a", "c") assert_items_left(2)

def test_edit(): given_at_todomvc("a", "b", "c") edit("c", "c edited") assert_tasks("a", "b", "c edited") assert_items_left(3)

def test_toggle(): #...

#...

def test_add(): given_at_todomvc() add("a", "b") assert_tasks("a", "b") assert_items_left(2)

“Unit” / “1-feature-per-test” styledef test_delete(): given_at_todomvc("a", "b", "c") delete("b") assert_tasks("a", "c") assert_items_left(2)

def test_edit(): given_at_todomvc("a", "b", "c") edit("c", "c edited") assert_tasks("a", "b", "c edited") assert_items_left(3)

def test_toggle(): #...

#...

“Unit”/“Feature”E2E

vs

“Unit”/“Feature”E2Edef test_task_life_cycle(): given_at_todomvc() add("a") edit("a", "a edited") toggle("a edited") filter_active() assert_no_tasks() filter_completed() delete("a edited") assert_no_tasks() #...

def test_add(): given_at_todomvc() add("a", "b") assert_tasks("a", "b") assert_items_left(2)

def test_delete(): #...

#...

vs

“Unit”/“Feature”E2E

SimpleEasy

def test_task_life_cycle(): given_at_todomvc() add("a") edit("a", "a edited") toggle("a edited") filter_active() assert_no_tasks() filter_completed() delete("a edited") assert_no_tasks() #...

def test_add(): given_at_todomvc() add("a", "b") assert_tasks("a", "b") assert_items_left(2)

def test_delete(): #...

#...

Simple is not Easy

–Rich Hickey

“Simple Made Easy”

“Unit”/“Feature”E2E

vs

“Unit”/“Feature”E2E

+ more coverage

+ in less time

+ in case of bugs, gives more complete report

+ easier to identify reason from the report

=>

+ less time and efforts in support

+ with less efforts during POC implementation

+ integration coverage

vs

“Unit”/“Feature”E2E

SimpleEasy

+ more coverage

+ in less time

+ in case of bugs, gives more complete report

+ easier to identify reason from the report

=>

+ less time and efforts in support

+ with less efforts during POC implementation

+ integration coverage

vs

“Unit”/“Feature”E2E

SimpleEasy

+ more coverage

+ in less time

+ in case of bugs, gives more complete report

+ easier to identify reason from the report

=>

+ less time and efforts in support

+ with less efforts during POC implementation

+ integration coverage

vs

“Unit”/“Feature”E2E

SimpleEasy

+ more coverage

+ in less time

+ in case of bugs, gives more complete report

+ easier to identify reason from the report

=>

+ less time and efforts in support

+ with less efforts during POC implementation

+ integration coverage

vs Simple“Pretty”

Reports

public class TodoMVCTest { @Test public void testTaskLifeCycle(){ givenAtTodoMVC(); add("a"); toggle("a"); filterActive(); assertNoTasks(); filterCompleted(); edit("a", "a edited"); toggle("a edited"); assertNoTasks(); //... }

@Test public void testAddTasks(){ givenAtTodoMVC(); add("a", "b", "c"); assertTasks("a", "b", "c"); assertItemsLeft(3); } @Test public void testDeleteTask(){ givenAtTodoMVC("a", "b", "c"); delete("b"); assertTasks("a", "c"); assertItemsLeft(2); } //...}

Simple

<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.18.1</version> <configuration> <argLine> -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar" </argLine> <properties> <property> <name>listener</name> <value>ru.yandex.qatools.allure.junit.AllureRunListener</value> </property> </properties> </configuration> <dependencies> <dependency> <groupId>org.aspectj</groupId> <artifactId>aspectjweaver</artifactId> <version>${aspectj.version}</version> </dependency> </dependencies> </plugin>

Allure reporting

<reporting> <excludeDefaults>true</excludeDefaults> <plugins> <plugin> <groupId>ru.yandex.qatools.allure</groupId> <artifactId>allure-maven-plugin</artifactId> <version>2.0</version> </plugin> </plugins></reporting>

@Steppublic static void add(String... taskTexts) { for(String text: taskTexts){ newTodo.setValue(text).pressEnter(); }}@Steppublic static void filterActive(){ $(By.linkText("Active")).click();}

//...

Allure reporting

“Pretty” with Allure

vs

SimplePretty

SimpleMore configs & annotations

Reporting

vs

SimplePretty

Enough for “Unit”Good for E2E

Reporting

BDD vs xUnit

BDD

BDD

xUnitBDD

vs

xUnitBDD1. Write scenarios with steps

2. Define steps

2.1 Introduce state

3. Map steps

4. Configure Runner

5. Run

6. Get pretty reports

1. Write tests with steps

2. Define steps

3. Run

4. Get reports

vs

xUnitBDD1. Write scenarios with steps

2. Define steps

2.1 Introduce state

3. Map steps

4. Configure Runner

5. Run

6. Get pretty reports

1. Write tests with steps

2. Define steps

3. Run

4. Get reportsstill pretty with Allure

vs

xUnitBDD

Simple & Easy? :)

1. Write scenarios with steps

2. Define steps

2.1 Introduce state

3. Map steps

4. Configure Runner

5. Run

6. Get pretty reports

1. Write tests with steps

2. Define steps

3. Run

4. Get reportsstill pretty with Allure

vs

xUnitBDD

Simple & Easy? :)

1. Write scenarios with steps

2. Define steps

2.1 Introduce state

3. Map steps

4. Configure Runner

5. Run

6. Get pretty reports

1. Write tests with steps

2. Define steps

3. Run

4. Get reportsstill pretty with Allure

vs

xUnitBDD

Less coding, Full power of PLNo vars & return from steps

1. Write tests with steps

2. Define steps

3. Run

4. Get reportsstill pretty with Allure

vs

xUnitBDD

Less coding, Full power of PLNo vars & return from steps

1. Write tests with steps

2. Define steps

3. Run

4. Get reportsstill pretty with Allure

vs

xUnitBDD

Less coding, Full power of PLMonkeys-friendly

1. Write tests with steps

2. Define steps

3. Run

4. Get reportsstill pretty with Allure

vs

xUnitBDD

Less coding, Full power of PLMonkeys-friendly

1. Write tests with steps

2. Define steps

3. Run

4. Get reportsstill pretty with Allure

PageModules

vs

PageObjects

PageModules

vsModularOOP

PageObjects

from pages import tasks

def test_filter_tasks():

tasks.visit() tasks.add("a", "b", "c") tasks.should_be("a", "b", "c") ...

from pages.tasks import TasksPage

def test_filter_tasks(): tasks = TasksPage() tasks.visit() tasks.add("a", "b", "c") tasks.should_be("a", "b", "c") ...

PageModulesPageObjects

ModularOOP

class TasksPage: def __init__(self): self.tasks = ss("#todo-list>li") def visit(self): tools.visit('https://todomvc4tasj.herokuapp.com/') ... def should_be(self, *task_texts): self.tasks.filterBy(visible).should_have(exact_texts(*task_texts)) ...

PageObjects

OOP

#tasks.py

tasks = ss("#todo-list>li")def visit(): tools.visit('https://todomvc4tasj.herokuapp.com/')

...

def should_be(*task_texts): tasks.filterBy(visible).should_have(exact_texts(*task_texts))

...

PageModules

Modular

PageModules

vs

ModularOOP

PageObjects

PageModules

vs

ModularOOP

PageObjects

+ simple

+ easy

+ “newbies” friendly

PageModules

vs

ModularOOP

PageObjects

+ ? + simple

+ easy

+ “newbies” friendly

PageModules

vs

ModularOOP

PageObjects

+ ? + simple

+ easy

+ “newbies” friendly+ parallelised tests

PageModules

vs

ModularOOP

PageObjects

+ ? + simple

+ easy

+ “newbies” friendly+ parallelised tests

(where there is no automatic driver management per

thread)

Pages of Widgets

vs

Plain Pages

Pages of Widgets

vs

OOP Modular/Procedural

Plain Pages

OOP

Pages of “plural” Widgets: Usagedef test_nested_custom_selements(): given_active("a", "b") main = TodoMVC("#todoapp") main.tasks.find(text("b")).toggle() main.clear_completed() main.tasks.assure(texts("a"))

OOP

Pages of “plural” Widgets: Usagedef test_nested_custom_selements(): given_active("a", "b") main = TodoMVC("#todoapp") main.tasks.find(text("b")).toggle() main.clear_completed() main.tasks.assure(texts("a"))

OOP

Pages of “plural” Widgets: Usagedef test_nested_custom_selements(): given_active("a", "b") main = TodoMVC("#todoapp") main.tasks.find(text("b")).toggle() main.clear_completed() main.tasks.assure(texts("a"))

OOP

Pages as Widgets: Usagedef test_nested_custom_selements(): given_active("a", "b") main = TodoMVC("#todoapp") main.tasks.find(text("b")).toggle() main.clear_completed() main.tasks.assure(texts("a"))

Pages (as widgets) of “plural” Widgets: Implementation

OOP

class TodoMVC(SElement): def init(self): self.tasks = ss("#todo-list>li").of(self.Task) def clear_completed(self): s(“#clear-completed").click() return self class Task(SElement): def toggle(self): self.s(".toggle").click() return self

Pages (as widgets) of “plural” Widgets: Implementation

OOP

class TodoMVC(SElement): def init(self): self.tasks = ss("#todo-list>li").of(self.Task) def clear_completed(self): s(“#clear-completed").click() return self class Task(SElement): def toggle(self): self.s(".toggle").click() return self

Pages (as widgets) of “plural” Widgets: Implementation

OOP

class TodoMVC(SElement): def init(self): self.tasks = ss("#todo-list>li").of(self.Task) def clear_completed(self): s(“#clear-completed").click() return self class Task(SElement): def toggle(self): self.s(".toggle").click() return self

Pages (as widgets) of “plural” Widgets: Implementation

OOP

class TodoMVC(SElement): def init(self): self.tasks = ss("#todo-list>li").of(self.Task) def clear_completed(self): s(“#clear-completed").click() return self class Task(SElement): def toggle(self): self.s(".toggle").click() return self

Pages (as widgets) of “plural” Widgets: Implementation

OOP

class TodoMVC(SElement): def init(self): self.tasks = ss("#todo-list>li").of(self.Task) def clear_completed(self): s(“#clear-completed").click() return self class Task(SElement): def toggle(self): self.s(".toggle").click() return self

Pages (as widgets) of “plural” Widgets: Implementation

OOP

class TodoMVC(SElement): def init(self): self.tasks = ss("#todo-list>li").of(self.Task) def clear_completed(self): s(“#clear-completed").click() return self class Task(SElement): def toggle(self): self.s(".toggle").click() return self

Pages (as widgets) of “plural” Widgets: Implementationclass TodoMVC(SElement): def init(self): self.tasks = ss("#todo-list>li").of(self.Task) def clear_completed(self): s(“#clear-completed").click() return self class Task(SElement): def toggle(self): self.s(".toggle").click() return self OOP

Plain Pages: Usageimport tasks

def test_nested_custom_selements(): given_active("a", "b") tasks.toggle("b") tasks.clear_completed() tasks.shouldBe("a")

Procedural

Plain Pages: Usageimport tasks

def test_nested_custom_selements(): given_active("a", "b") tasks.toggle("b") tasks.clear_completed() tasks.shouldBe("a")

Procedural

Plain Pages: Implementation#tasks.py

tasks = ss("#todo-list>li")def toggle(task_text): tasks.findBy(exact_text(task_text).s(".toggle").click()

...

def should_be(*task_texts): tasks.filterBy(visible).should_have(exact_texts(*task_texts))

... Procedural

OOP

Pages of “singular” Widgets: Usagedef test_nested_custom_selements(): given_active("a", "b") main = TodoMVC("#todoapp") main.tasks.find(text("b")).toggle() main.clear_completed() main.tasks.assure(texts("a")) main.footer.assure_items_left(1)

OOP

Pages of “singular” Widgets: Usagedef test_nested_custom_selements(): given_active("a", "b") main = TodoMVC("#todoapp") main.tasks.find(text("b")).toggle() main.clear_completed() main.tasks.assure(texts("a")) main.footer.assure_items_left(1)

OOP

Pages of “singular” Widgets: Usagedef test_nested_custom_selements(): given_active("a", "b") main = TodoMVC("#todoapp") main.tasks.find(text("b")).toggle() main.clear_completed() main.tasks.assure(texts("a")) main.footer.assure_items_left(1)

OOP

Pages of “singular” Widgets: Implementationclass TodoMVC(SElement): def init(self): self.tasks = ss("#todo-list>li").of(self.Task) self.footer = self.Footer("#footer") ... class Footer(SElement): def init(self): self.clear_completed = self.s("#clear-completed") def assure_items_left(self, number_of_active_tasks): self.s("#todo-count>strong").assure(exact_text(str(number_of_active_tasks)))

OOP

Pages of “singular” Widgets: Implementationclass TodoMVC(SElement): def init(self): self.tasks = ss("#todo-list>li").of(self.Task)

self.footer = self.Footer("#footer") ... class Footer(SElement): def init(self): self.clear_completed = self.s("#clear-completed") def assure_items_left(self, number_of_active_tasks): self.s("#todo-count>strong").assure(exact_text(str(number_of_active_tasks)))

Pages of “singular” Widgets: Implementationclass TodoMVC(SElement): def init(self): self.tasks = ss("#todo-list>li").of(self.Task)

self.footer = self.Footer("#footer") ... class Footer(SElement): def init(self): self.clear_completed = self.s("#clear-completed") def assure_items_left(self, number_of_active_tasks): self.s("#todo-count>strong").assure(exact_text(str(number_of_active_tasks)))

Plain Pages: Usagedef test_nested_custom_selements(): given_active("a", "b") tasks.toggle("b") tasks.clear_completed() tasks.shouldBe(“a") footer.assert_items_left(1)

Modular

Plain Pages: Implementation#tasks.py tasks = ss("#todo-list>li")def toggle(task_text): tasks.findBy(exact_text(task_text).s(".toggle").click() ...

Modular

#footer.py def assure_items_left(self, number_of_active_tasks): s("#todo-count>strong").assure(exact_text(str(number_of_active_tasks))) ...

Plain Pages

vs

Procedural/ModularOOP

Pages with Widgets

Plain Pages

vs

Procedural/ModularOOP

Pages with Widgets

+ simple

+ easy

+ “newbies” friendly

+ less code

+ visible

Plain Pages

vs

Procedural/ModularOOP

Pages with Widgets

+ ?

+ simple

+ easy

+ “newbies” friendly

+ less code

+ visible

Plain Pages

vs

Procedural/ModularOOP

Pages with Widgets

+ ?

+ simple

+ easy

+ “newbies” friendly

+ less code

+ visible

+ for complex UI with many “plural” widgets

AfterwordsThere are good practices in context,

but there are no best practices.

(c) Cem Kaner, James Bach

Q&A

Thank you!

github.com/yashaka slideshare.net/yashaka

gitter.im/yashaka/better-selenium youtube.com/c/ItlabsNetUa

@yashakaitlabs.net.ua