GettingStarted

WithTesting

Topics:

� Bugs

� Tests

� Unit Tests

� Testability

� Isolation

� Common Mistakes

BUGS

WhatBugsYou?

Logical

Bugs

Wiring

Bugs

UI

Bugs

ABug’sLife

1 X

500 X

DEV PRODUCTION QA DEV TEST LAB

The cost of a bug can increase 500 times from

development to the production release.

DESIGN

0.01

X

5 X

50 X

IneffectivePrevention

� We should code fewer bugs

� We need a tool that finds all bugs

� We should hire more QA people

� We should have (manual) test plans

� We should refactor (without tests)

� There is nothing to do about it

Oops…

EffectivePrevention

Good Design (complexity & coupling)

High-Level technical documentation

Code Reviews – Fagan, Over The Shoulder, Remote

Pair Programming

Use logs, error logs, e-mail notification, etc.

Write Tests (especially Unit Tests)

TESTS

TestTypes

TheTestPyramid

Black Box

White Box

White Box

END TO

END

INTEGRATION

TESTS

UNIT TESTS

UNITTESTS

BenefitsofwritingUnitTests

1. Bugs are found (much) earlier.

2. Safer changes/refactoring (regression).

3. Up-to-date technical documentation and

examples.

4. Better low level architecture.

Idonotwritetestsbecause…

� I do not know how.

� It is not my job. The QA people are paid to do that.

� It is not my job. Let the customer figure what’s wrong.

� I work with untestable legacy code.

� I only develop UI.

� It makes it difficult to refactor the code.

� Requirements are not clear.

� I can test everything manually.

� There is no time for that.

AnatomyofaUnitTest

Set up the needed

dependencies/para

Select a story (class

method + scenario)

Instantiate the class

Exercise/Stimulus

Run the method

Assert the expected

behavior

[TestClass]

public class CalculatorTest {

[TestMethod]

public void ShouldAddTwoPositiveIntegerNumbers() {

int addendum1 = 1;

int addendum2 = 2;

Calculator calc = new Calculator();

int sum = calc.Add(addendum1, addendum2);

Assert.AreEqual(sum, 3);

}

}

On-FieldMission:

TestingTheOrderProcessor

Your Mission, should you choose to accept

it, is to unit test ValidateOrderTotal.

INPUT:

A customer id

OUTPUT:

True is the customer order for next delivery exists and the total

amount is > 0. False otherwise.

OrderProcessor processor = OrderProcessor.GetInstance();

bool isValidOrderTotal = processor

.ValidateOrderTotal("00000345-00000-000056");

MissionImpossible?

Mission log:

Attempt # Problem Solution

1 File Not Found Need configuration file

2 Invalid Database Set valid connection string

3 Invalid Ross Customer Id Set valid Ross Customer ID

4 Folder not found The export folder must exists and have

permissions

5 SMPT error Update SMPT configuration with valid server,

credential and set notification email

6 Order Processor Stuck Exception Delete marker file

7 Customer does not have order for next del.

date

Create order in DB for customer

8 What about the following delivery? …

… $#%k! …

… … …$#%k! $#%k!

[TestMethod]

public void CheckCustomerWithValidOrderForNextDeliveryDate() {

OrderProcessor processor = OrderProcessor.GetInstance();

bool IsValidOrderTotal = processor.ValidateOrderTotal("00000345-00000-000056");

Assert.IsTrue(IsValidOrderTotal, "...some error message..."); }

Bro!Seriously?!

There should be only one reason for a unit test to fail…

Do you have the configuration file?

What is the database connection

string?

Did you set up the export folder?

Did you set up a valid SMTP server?

Did you delete the marker file?

Will the test work next week? Did you set up the Ross Customer

ID correctly?

Does the customer key exist?

Does the customer have an order

for the next delivery?

Does the customer have an order

for the next delivery?

Did you set up a customer with a

valid order total?

Did you set up a customer with an

invalid order total?

Did you set up a customer with a

valid order total?

Did you set up the maximum

waiting time?

Corpus

Delicti

public class OrderProcessor {

private static OrderProcessor _instance = new OrderProcessor();

private bool _isMarkerFilePresent = false; private int _rossCustomerId = 0; private string _exportPath = string.Empty;

private OrderProcessor() { _exportPath = ConfigurationSettings.AppSettings["OrderExportPath"];

_isMarkerFilePresent = IOHelper.CheckMarkerFile(this._exportPath); _rossCustomerId = Convert.ToInt32(ConfigurationSettings.AppSettings["RossCustomerID"]);

int maxSecs = Convert.ToInt32(ConfigurationSettings.AppSettings["MaxSecsToWait"]); int secsWaited = 0; while (_isMarkerFilePresent) {

Thread.Sleep(1000); secsWaited += 1000; if (secsWaited > maxSecs) {

string email = DBService.GetRossCustomerEmail(_rossCustomerId); EmailManager.SendEmail(email, "Order Processor is stuck! Check the marker file!"); throw new Exception("Order Processor is stuck! Check the marker file!");

} }

IOUtility.CreateMarkerFile(_exportPath, "OrderProcessor.MRK"); }

public static OrderProcessor GetInstance() { return _instance; }

public void ExportOrder(string customerId) { … }

public bool ValidateOrderTotal(string customerId) { decimal orderTotal = 0; Order order = DBService.GetOrder(customerId, DateTime.Now);

if (order != null) { foreach (OrderDetail orderDetail in order.Details) { orderTotal += orderDetail.TotalAmount;

} }

return orderTotal>0; }

}

TESTABILITY

TheThreeFactors

Easy-to-write

meaningful

automated tests

Consistent

behavior across

multiple

deployments

Introspection

Testablevs.Untestable

Dependency Injection/Transparency

Interfaces/Factories/IOC

Law Of Demeter

Seams

Single Responsibility/Layered Architecture

Composition

Simple Constructors

Idempotence

Commutativity

Implicit Dependencies

New Operator

Service Locators

Static

Mixed Concerns

Deep Inheritance

Complex Constructors

Singletons, DateTime.Now/Random

Implicit Order (hints: Init, Pre, Post, Cleanup, Load)

ABetterPointofView

Writing tests requires the code to be

testable.

Refactoring code for testability

requires time, skills and experience.

Before writing code, answer this question:

How would you like to test it?

TestFirst

Tests First = testability + small, well-isolated components

Focus on the API first and implementation later

Where did the database go?

Where is the marker file?

What happened to the configuration?

[TestMethod, TestCategory("Order Validation")]

public void OrderWithTotalAmountGreaterThanZeroShouldBeValid() {

Order order = CreateFakeOrder(totalAmount: 10.0);

IOrderValidation orderValidation = new OrderValidation(order);

bool isValidOrderTotal = orderValidation.IsValidTotal();

Assert.IsTrue(isValidOrderTotal,

"Total order amount > 0 should be valid");

}

[TestMethod, TestCategory("Order Validation")]

public void OrderWithTotalAmountZeroShouldBeInvalid() {

Order order = CreateFakeOrder(totalAmount: 0.0);

IOrderValidation orderValidation = new OrderValidation(order);

bool isValidOrderTotal = orderValidation.IsValidTotal();

Assert.IsFalse(isValidOrderTotal,

"Total order amount = 0 should be invalid");

}

[TestMethod, TestCategory("Order Validation")]

public void EmptyOrderShouldBeInvalid() {

Order order = Order.Empty;

IOrderValidation orderValidation = new OrderValidation(order);

bool isValidOrderTotal = orderValidation.IsValidTotal();

Assert.IsFalse(isValidOrderTotal,

"Empty Order should be invalid");

}

public class OrderValidation : IOrderValidation {

Order _order = Order.Empty;

public OrderValidation(Order order) {

_order = order;

}

public bool IsValidTotal() {

decimal orderTotal = 0M;

if (!_order.IsEmpty()) {

foreach (OrderDetail orderDetail in _order.Details) {

orderTotal += orderDetail.DetailAmount;

}

}

return orderTotal > 0;

}

}

ISOLATION

Controllingthe

TestEnvironment

A good unit test is like a good scientific

experiment: it isolates as many variables as

possible (these are called control variables)

and then validates or rejects a specific

hypothesis about what happens when the one variable (the

independent variable) changes.

Test Code System Under

Test (SUT)

Dep. 1

Dep. 2

Dep. N ISOLATED TEST

SEAMS

SystemUnderTest

SUT COLLABORATOR

PILOT PLANE

TestDoubles

Do you need a real plane?

� We do not have a real plane available yet

� It takes time and it is expensive to prepare a real

plane (checkup, gas, etc.)

� Too dangerous, the plane may crash

� If there is an issue it is hard to tell if the problem

was the pilot or the plane

Pickyourdouble

Fake Mock

Dummy Stub/Spy

Dummy

Dummy objects are passed around but never

actually used. Usually they are just used to fill

parameter lists.

public class DummyPlane implements IPlane {

public DummyPlane() {

// Real simple; nothing to initialize!

}

public int getAltitude() {

throw new RuntimeException("This should never be called!");

}

public int getSpeed() {

throw new RuntimeException("This should never be called!");

}

}

Stub/Spy

Stubs provide canned answers to calls

made during the test, usually not

responding at all to anything outside what's programmed in for

the test. Spies can record some information based on how

they were called

public class PlaneStub implements IPlane {

public PlaneStub () {

// Real simple; nothing to initialize!

}

public int getAltitude () {

log.Write(“getAltitude was called”);

return 300;

}

public int getSpeed () {

throw new RuntimeException("This should never be called!");

}

}

Fake

Fake objects actually have working

implementations, but usually take some shortcut which makes

them not suitable for production (an In Memory Database is a

good example).

public class FakePlane implements IPlane {

…

public FakePlane (FlightStatus status) {

_flightStatus = status;

}

public int getAltitude () {

return CalculateAltitude(_flightStatus);

}

public int getSpeed () {

return CalculateSpeed(_flightStatus);

}

}

Mock

Mocks are pre-programmed with expectations

which form a specification of the calls they are

expected to receive. They can throw an exception if they

receive a call they don't expect and are checked during

verification to ensure they got all the calls they were

expecting.

planeControl = MockControl.createControl(Plane.class);

planeMock = (IPlane) planeControl.getMock();

public void testPilotCheckAltitude() {

Pilot pilot = new Pilot();

planeMock.getAltitude();

planeMock.setReturnValue(300);

planeMock.replay();

pilot.Flight((IPlane) planeMock);

planeControl.verify();

}

MICROSOFTFAKES

Whatisit?

Microsoft Fakes is a code isolation framework that can help you isolate code for

testing by replacing other parts of the application with stubs or shims. It allows

you to test parts of your solution even if other parts of your app haven’t been

implemented or aren’t working yet.

Stubs

[TestMethod]

public void

StubAccountOpenPostsInitialBalanceCreditTransaction()

{

// Arrange:

int callCount = 0;

StubITransactionManager stubTM = new StubITransactionManager

{

PostTransactionDecimalTransactionType = (amount, type) =>

{

if (amount == 10m && type == TransactionType.Credit)

{

callCount++;

}

}

};

Account cut = new Account(stubTM);

// Act

cut.Open(10m);

// Assert

Assert.AreEqual<int>(1, callCount);

}

Shims(MonkeyPatching)

Shims are a feature of Microsoft Fakes that allows creating

unit tests for code that would otherwise not be testable in

isolation. In contrast to Stubs, Shims do not require the

code to be tested be designed in a specific way.

[TestMethod]

public void DateTimeTes()

{

using (ShimsContext.Create())

{

// Arrange:

System.Fakes.ShimDateTime.NowGet = () => { return new

DateTime(2016, 2, 29); };

// Act

int result = MyCode.DoSomethingSpecialOnALeapYear();

// Assert

Assert.AreEqual(100, result);

}

}

COMMONMISTAKES

All-In-One

This is by far the most common mistake among beginners.

It comes in two flavors:

1. Unrelated features tested in the same test class/method

2. Single feature tested across multiple layers Consequences: when the test fails, it is hard to point out at

what has failed

LongParking

Tests are written correctly but they are executed rarely

and not maintained Consequences: obsolete tests do not help preventing

bugs. They could mislead developers who read them to

try to understand the program

Whereisthenegative?

Other common mistake, tests are checking only what

the application should do (positive cases) and not

what it should not do (negative cases). Edge cases

and expected exception are also typically ignored by

beginners. Consequences: poor coverage due to untested scenarios

BadNames

Tests methods have names that are too

generic or meaningless. E.g. “reportTest”

or “authenticationTest” Consequences: when the test fails, it is

not immediately obvious what has failed.

They could mislead developers who read

them to try to understand the program

DonotsuccumbtotheDarkSide

WriteUnitTests!

Further Reading

Guide: Writing Testable Code

Exploring The Continuum of Test Doubles

Better Unit Testing with Microsoft Fakes

Videos Introduction to Unit Testing in .NET

http://dotnet.dzone.com/articles/introduction-unit-testing-net

The Clean Code Talks -- Unit Testing

http://www.youtube.com/watch?v=wEhu57pih5w

GTAC 2010: Lessons Learned from Testability Failures

http://www.youtube.com/watch?v=4CFj5thxYQA