Javascript Unit Testing with xUnit.js Building cleaner APIs, faster!

John Buchanan, salesforce.com, inc.Lead Member of Technical Staff

Safe harborSafe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services. The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site. Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.

Challenges:

1. What are some of the problems with testing javascript?

Challenges:

1. What are some of the problems with testing javascript?

• Boundary Control (Web requests, DOM access)

Challenges:

1. What are some of the problems with testing javascript?

• Boundary Control (Web requests, DOM access)

• Automation / Continuous Integration

Challenges:

1. What are some of the problems with testing javascript?

• Boundary Control (Web requests, DOM access)

• Automation / Continuous Integration

• Design Dependencies, Encapsulation

Challenges:

2. What are the differences between unit and integration tests?

Challenges:

2. What are the differences between unit and integration tests?

• Single module or component vs. interactions between modules

Challenges:

2. What are the differences between unit and integration tests?

• Single module or component vs. interactions between modules

• Boundary Control (Web requests, DOM access)

Challenges:

2. What are the differences between unit and integration tests?

• Single module or component vs. interactions between modules

• Boundary Control (Web requests, DOM access)

• Cost to exercise in automation: Unit tests are cheap

Challenges:

2. What are the differences between unit and integration tests?

• Single module or component vs. interactions between modules

• Boundary Control (Web requests, DOM access)

• Cost to exercise in automation: Unit tests are cheap

• Dependencies are abstracted in units, included in integrations

xUnit.js

xUnit.js?So what is

xUnit.jsA Host-Agnostic Test Engine with an

Attribution Based Console Runner

lolwut.

xUnit.jsA brief history of

xUnit.js:

• Started in 2006, open source project hosted on CodePlex

• Frustrated with typical test registration systems

• Environmental requirements were expensive (Server, browser)

• Loved the simplicity and speed of the WSH (cscript.exe)

• Starting to understand using unit tests as a design tool

Unit Tests:

• Should be easily organizable

• Must to be simple to add, minimizing “dev bounce”

• Need to be easy to run, but also run fast

• Should help enforce dependency isolation

• Could allow for custom types of decoration (Ignore, Test Category, etc.)

Host Agnostic:

• Should support common hosts out of the box: •Google Chrome (V8)•Mozilla Firefox (SpiderMonkey)•Microsoft Internet Explorer 4-11 (JScript and Chakra, via cscript.exe)•Mozilla Rhino

• Must also be able to easily add new and special use hosts:•JSDB•NodeJS•PhantomJS

Realization:

• We don’t need a web server

• We don’t even need a browser

• All we need is the engine, the test file, and the file under test.*

* Drastic simplification – wait for it… ☺

xUnit.jsA Host-Agnostic Test Engine with an

Attribution Based Console Runner

• Organizable:

• Easy To Add:

• Let’s call tests “Facts”, to encourage single aspect-testing• We’ll group Facts into “Fixtures”, based on “like behavior”• E.g. several tests exercising a single method

• Want to maintain registration with the test• C# style attributes are a great [Example] of this behavior• The syntax is valid javascript, and will be ignored!

Missed Mandates:

[Fixture]function foo(){

[Fact]function ReturnsFoo(){

var expected=“foo”;

var actual=new FooClass().foo();

Assert.Equal(expected, actual);}

}

function FooClass(){

this.foo=function(){return “foo”;

};

}

Test Code

Source Code

Successful Run:

Failures, Errors, Script Errors:[Fixture]function foo(){

[Fact]function FailsEqualityAssertion(){

var expected=true;

var actual=false;

Assert.Equal(expected, actual);}

} Failure

Failures, Errors, Script Errors:

Failures, Errors, Script Errors:[Fixture]function foo(){

[Fact]function ErrorsOnUnknownMember(){

var expected=true;

var actual={value:true}.slice(0,1);

Assert.Equal(expected, actual);}

}

Error

Failures, Errors, Script Errors:

Failures, Errors, Script Errors:[Fixture]function foo(){

[Fact]function ThrowsSyntaxErrorOnLoad(){

var expected=if(true));

var actual=false;

Assert.Equal(expected, actual);}

}

Script Error

Failures, Errors, Script Errors:

Mocks and Stubs:[Fact]function MocksWindowReferenceOnHost(){

var expected="expected";var mockWindow=Mocks.GetMock(Object.Global(), "window", expected);var actual=null;

mockWindow(function(){actual=window;

});

Assert.Equal(expected, actual);}

Mocks and Stubs:[Fact]function ReturnsResultWindowHandle(){

var expected="expected";var windowStub=Stubs.GetObject(

{open:{returnValue:expected}}, {location:"known/url"}

);

var actual=openResults(windowStub);

Assert.Equal(expected, actual);}

function openResults(targetWindow

){var resultsWindow =

targetWindow.open("results/url"

);return resultsWindow;

}

Data-Driven Tests:[Fact, Data(1,2,3,4)]function VerifiesDataIsInRange(data){

var expectedLow = 1;var expectedHigh = 3;

var actual=data;

Assert.InRange(actual,expectedLow,expectedHigh);}

Usage:

[…]

Benefits of Use Driven Development:

• Lightweight code host improves development cycle

• No need to wait for browser to manually test most code

• Waste less time understanding emergent side effects

• Maintain confidence while refactoring

• Use unit tests to architect cleaner, more usable APIsThe Big One:

xUnit.js+ =

Challenges:

• Multiple build automation and CI systems

• OS interoperability requirements

• Large browser (javascript engine) support matrix

• Many different teams and modules, with different behaviors

• Existing tests required full system and browser automation

• Selenium tests were becoming increasingly slow and unwieldy

Solutions:

• xUnit.js integrates easily as a command line or processor task

• Out-of-box support for native file I/O strategies and host environments

• Pluggable design allows for simple override and customization patterns

• Custom output strategies allow for report and issue tracker integration

• Shell execution and boundary control obviate end-to-end setup needs

• Selenium tests can be retargeted as integration and smoke tests

Custom Enhancements:

• JSCover Support (http://tntim96.github.io/JSCover/)

• Automatic issue assignment based on changelist author

• Test health history report integration

xUnit.js+ =

Preparation:

• Determine your test matrix

• Build or acquire necessary host engines

• Identify your build system’s executable target structure

• Identify your target test directories and files

Integration:

• Create CI target, .bat file, or .sh script

• Customize your installation (Optional)•Attributes (Default file import locations, etc.)•Output strategies (Reporting, Custom xml formats)

• Integrate additional tools (Optional, but highly recommended)•JSCover (http://tntim96.github.io/JSCover/)

cscript ./Tools/xUnit.js.Console.js ../../Tests

d8 – ./Tools/xUnit.js.Console.js -- ../../Tests

js ./Tools/xUnit.js.Console.js ../../Tests

Happy Coding!

John Buchanan

LMTS, salesforce.com, inc.

xUnit.jshttp://xunitjs.codeplex.com/