T E S T I N G T H E N E X T G E N E R AT I O N
M I K E H A R R I S
fscheck
specflow
is testing oftraits
specflow
is testing ofproperties
fscheck
B E I N G
B E I N G
B E I N G T H I N G S
B E I N G T H I N G S
A S M A R T I N H E I D E G G E R S P E A K S O F T H E M I N
“ T H E O R I G I N O F T H E W O R K O F A R T "
I N T E R P R E TAT I O N S O F T H I N G S
• substances with properties or as bearers of traits
• sense perceptions
• formed stuff
S U B S TA N C E S W I T H P R O P E R T I E S O R A S B E A R E R S O F T R A I T S
fscheck
specflow
is testing oftraits
specflow
is testing ofproperties
fscheck
S U B S TA N C E S W I T H P R O P E R T I E S O R A S B E A R E R S O F T R A I T S
Testing of Traits
is testing oftraits
specflow
I N E E D A F I Z Z B U Z Z T R A N S L AT O R
C H E C K P O I N T 1 ) N O B M I ' D I V I S I B L E P O N G 3 F I Z Z J I C H E G H ' O H . N O B M I ' D I V I S I B L E P O N G 5 B U Z Z J I C H E G H ' O H . L AT L H S O ' M E H J I C H E G H ‘ O H . !2 )⎕ I O← 0 ( L , ' F I Z Z ' ' B U Z Z ' ' F I Z Z B U Z Z ' ) [ ¯ 1 +( L×W = 0 ) + W← ( 1 0 0×~ 0 = W ) + W←⊃ + / 1 2× 0 = 3 5 |
⊂ L← 1 +⍳ 1 0 0 ]
S A M E PA G E !
C H E C K P O I N T 2 - > 2 3 - > F I Z Z 4 - > 4 5 - > B U Z Z 6 - > F I Z Z
M A K E I T S O .
F I Z Z B U Z Z
FizzBuzz(1 ) => “1” FizzBuzz(2 ) => “2” FizzBuzz(3 ) => “Fizz” FizzBuzz(4 ) => “4” FizzBuzz(5 ) => “Buzz” FizzBuzz(6 ) => “Fizz” … FizzBuzz(15) => “FizzBuzz”
namespace FizzBuzz { public class FizzBuzzer { public string Translate(int value) { var result = string.Empty; if (value % 3 == 0) result += "Fizz"; if (value % 5 == 0) result += "Buzz"; return string.IsNullOrEmpty(result) ? value.ToString() : result; } } }
T E S T I N G T R A I T S
T H E N U N I T W AY
[TestCase( 2, Result = "2")] [TestCase( 3, Result = "Fizz")] [TestCase( 4, Result = "4")] [TestCase( 5, Result = "Buzz")] [TestCase( 6, Result = "Fizz")] [TestCase(10, Result = "Buzz")] [TestCase(15, Result = "FizzBuzz")] [TestCase(45, Result = "FizzBuzz")] public string FizzBuzz_Tests(int value) { return fizzbuzzer.Translate(value); }
S P E C F L O W W AY
Feature: FizzBuzzerSpec In order to be able to communicate with the FizzBuzzons As a Starfleet Captain I want to be able to translate numbers to their FizzBuzz value
Scenario: 2 must translate to the string 2 Given a FizzBuzzer And the value of '2' When translate is invoked Then the result should be '2'
Scenario: 3 must translate to the string Fizz Given a FizzBuzzer And the value of '3' When translate is invoked Then the result should be 'Fizz'
Scenario: 5 must translate to the string Buzz Given a FizzBuzzer And the value of '5' When translate is invoked Then the result should be 'Buzz'
Scenario: 15 must translate to the string FizzBuzz Given a FizzBuzzer And the value of '15' When translate is invoked Then the result should be 'FizzBuzz'
Feature: FizzBuzzerSpec In order to be able to communicate with the FizzBuzzons As a Starfleet Captain I want to be able to translate numbers to their FizzBuzz value Scenario: 2 must translate to the string 2 Given a FizzBuzzer And the value of '2' When translate is invoked Then the result should be '2' Scenario: 3 must translate to the string Fizz Given a FizzBuzzer And the value of '3' When translate is invoked Then the result should be 'Fizz' Scenario: 5 must translate to the string Buzz Given a FizzBuzzer And the value of '5' When translate is invoked Then the result should be 'Buzz' Scenario: 15 must translate to the string FizzBuzz Given a FizzBuzzer And the value of '15' When translate is invoked Then the result should be 'FizzBuzz'
Feature: FizzBuzzerSpec In order to be able to communicate with the FizzBuzzons As a Starfleet Captain I want to be able to translate numbers to their FizzBuzz value
Scenario: 2 must translate to the string 2 Given a FizzBuzzer And the value of '2' When translate is invoked Then the result should be '2'
using FizzBuzz; using TechTalk.SpecFlow; using NUnit.Framework; namespace FizzBuzzSpec { [Binding] public class FizzBuzzerSpecSteps { [Given(@"a FizzBuzzer")] public void GivenAFizzBuzzer() { ScenarioContext.Current.Add("FizzBuzzer", new FizzBuzzer()); } [Given(@"the value of '(.*)'")] public void GivenTheValueOf(int value) { ScenarioContext.Current.Add("value", value); } [When(@"translate is invoked")] public void WhenTranslateIsInvoked() { var fizzbuzzer = ScenarioContext.Current.Get<FizzBuzzer>("FizzBuzzer"); var value = ScenarioContext.Current.Get<int>("value"); ScenarioContext.Current.Add("actual", fizzbuzzer.Translate(value)); } [Then(@"the result should be '(.*)'")] public void ThenTheResultShouldBe(string expected) { var actual = ScenarioContext.Current.Get<string>("actual"); Assert.That(expected, Is.EqualTo(actual)); } } }
Given a FizzBuzzer !
[Given(@"a FizzBuzzer")] public void GivenAFizzBuzzer() { ScenarioContext.Current.Add( "FizzBuzzer", new FizzBuzzer()); }
And the value of '2' !
[Given(@"the value of '(.*)'")] public void GivenTheValueOf(int value) { ScenarioContext.Current.Add("value", value); }
When translate is invoked [When(@"translate is invoked")] public void WhenTranslateIsInvoked() { var fizzbuzzer = ScenarioContext .Current.Get<FizzBuzzer>("FizzBuzzer"); var value = ScenarioContext .Current.Get<int>("value"); ScenarioContext .Current.Add( "actual", fizzbuzzer.Translate(value)); }
Then the result should be '2' [Then(@"the result should be '(.*)'")] public void ThenTheResultShouldBe( string expected) { var actual = ScenarioContext .Current.Get<string>("actual"); Assert.That(expected, Is.EqualTo(actual)); }
Scenario: 2 must translate to the string 2 Given a FizzBuzzer And the value of '2' When translate is invoked Then the result should be '2'
Scenario: 5 must translate to the string Buzz Given a FizzBuzzer And the value of '5' When translate is invoked Then the result should be 'Buzz'
Given a FizzBuzzer !
[Given(@"a FizzBuzzer")] public void GivenAFizzBuzzer() { ScenarioContext.Current.Add( "FizzBuzzer", new FizzBuzzer()); }
And the value of '5' !
[Given(@"the value of '(.*)'")] public void GivenTheValueOf(int value) { ScenarioContext.Current.Add("value", value); }
When translate is invoked [When(@"translate is invoked")] public void WhenTranslateIsInvoked() { var fizzbuzzer = ScenarioContext .Current.Get<FizzBuzzer>("FizzBuzzer"); var value = ScenarioContext .Current.Get<int>("value"); ScenarioContext .Current.Add( "actual", fizzbuzzer.Translate(value)); }
Then the result should be 'Buzz' [Then(@"the result should be '(.*)'")] public void ThenTheResultShouldBe( string expected) { var actual = ScenarioContext .Current.Get<string>("actual"); Assert.That(expected, Is.EqualTo(actual)); }
Scenario: 2 must translate to the string 2 Given a FizzBuzzer And the value of '2' When translate is invoked Then the result should be '2'
[NUnit.Framework.TestAttribute()] [NUnit.Framework.DescriptionAttribute("2 must translate to the string 2")] public virtual void _2MustTranslateToTheString2() { TechTalk.SpecFlow.ScenarioInfo scenarioInfo = new TechTalk.SpecFlow.ScenarioInfo( "2 must translate to the string 2", ((string[])(null))); #line 6 this.ScenarioSetup(scenarioInfo); #line 7 testRunner.Given("a FizzBuzzer", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Given "); #line 8 testRunner.And("the value of \'2\'", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "And "); #line 9 testRunner.When("translate is invoked", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "When "); #line 10 testRunner.Then("the result should be \'2\'", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Then "); #line hidden this.ScenarioCleanup(); }
G O O D , B U T N O T G R E AT.
Examples: | value | expected value | | 2 | 2 | | 4 | 4 | | 3 | Fizz | | 9 | Fizz | | 5 | Buzz | | 25 | Buzz | | 15 | FizzBuzz | | 30 | FizzBuzz |
Scenario Outline: FizzBuzz translation Given a FizzBuzzer And the value of '<value>' When translate is invoked Then the result should be '<expected value>' Examples: | value | expected value | | 2 | 2 | | 4 | 4 | | 3 | Fizz | | 9 | Fizz | | 5 | Buzz | | 25 | Buzz | | 15 | FizzBuzz | | 30 | FizzBuzz |
[Given(@"a FizzBuzzer")] public void GivenAFizzBuzzer() { ScenarioContext.Current.Add( "FizzBuzzer", new FizzBuzzer()); }
[Given(@"the value of '(.*)'")] public void GivenTheValueOf(int value) { ScenarioContext.Current.Add("value", value); }
[When(@"translate is invoked")] public void WhenTranslateIsInvoked() { var fizzbuzzer = ScenarioContext .Current.Get<FizzBuzzer>("FizzBuzzer"); var value = ScenarioContext .Current.Get<int>("value"); ScenarioContext .Current.Add( "actual", fizzbuzzer.Translate(value)); }
[Then(@"the result should be '(.*)'")] public void ThenTheResultShouldBe( string expected) { var actual = ScenarioContext .Current.Get<string>("actual"); Assert.That(expected, Is.EqualTo(actual)); }
Examples: | value | expected value | | 2 | 2 | | 4 | 4 | | 3 | Fizz | | 9 | Fizz | | 5 | Buzz | | 25 | Buzz | | 15 | FizzBuzz | | 30 | FizzBuzz |
[TestCase( 2, Result = "2")] [TestCase( 3, Result = "Fizz")] [TestCase( 4, Result = "4")] [TestCase( 5, Result = "Buzz")] [TestCase( 6, Result = "Fizz")] [TestCase(10, Result = "Buzz")] [TestCase(15, Result = "FizzBuzz")] [TestCase(45, Result = "FizzBuzz")] public string FizzBuzz_Tests(int value) { return fizzbuzzer.Translate(value); }
Examples: | value | expected value | | 2 | 2 | | 4 | 4 | | 3 | Fizz | | 9 | Fizz | | 5 | Buzz | | 25 | Buzz | | 15 | FizzBuzz | | 30 | FizzBuzz |
G R E AT.
"FizzBuzzSpec\FizzBuzzSpec\FizzBuzzerSpecFeature\_15MustTranslateToTheStringFizzBuzz","Passed","00:00:00.183","(local)" Given a FizzBuzzer -> done: FizzBuzzerSpecSteps.GivenAFizzBuzzer() (0.0s) And the value of '15' -> done: FizzBuzzerSpecSteps.GivenTheValueOf(15) (0.0s) When translate is invoked -> done: FizzBuzzerSpecSteps.WhenTranslateIsInvoked() (0.0s) Then the result should be 'FizzBuzz' -> done: FizzBuzzerSpecSteps.ThenTheResultShouldBe("FizzBuzz") (0.0s)
D O E S T H I S R E A L LY W O R K ? ! ?
W E C O U L D H AV E T H E S Y S T E M G E N E R AT E D ATA F O R U S
T O I L L U S T R AT E T H E A D H E R E N C E O F P R O P E R T I E S
M A K E I T S O .
Testing of Properties
is testing ofproperties
fscheck
T E S T I N G P R O P E R T I E S
T H E N U N I T W AY
[Test] public void Divisible_By_3_Is_Fizz( [Range(3, 300, 3)] int value) { var removeBuzz = (value % 5 == 0) ? 3 : value; Assert.That(FizzBuzz(removeBuzz), Is.EqualTo("Fizz")); }
[Test] public void Divisible_By_5_Is_Buzz( [Random(1, 10000, 100)] int value) { var removeFizz = (value % 3 == 0) ? 5 : value * 5; Assert.That(FizzBuzz(removeFizz), Is.EqualTo("Buzz")); }
F S C H E C K W AY
[TestCase] public void Number_Divisible_By_3_Will_Return_Fizz() { Spec.ForAny<int>(value => _fizzBuzzer.Translate(value).Equals("Fizz")) .When(value => value % 3 == 0 && value % 5 != 0) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
[TestCase] public void Number_Divisible_By_5_Will_Return_Buzz() { Spec.ForAny<int>(value => _fizzBuzzer.Translate(value).Equals("Buzz")) .When(value => value % 3 != 0 && value % 5 == 0) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
[TestCase] public void Number_Divisible_By_15_Returns_FizzBuzz() { Spec.ForAny<int>(value => _fizzBuzzer.Translate(value).Equals("FizzBuzz")) .Or(value => value % 15 != 0) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
H M M M M
W E C O U L D C R E AT E A M O D E L T O D E M O N S T R AT E T H E
R O B U S T N E S S O F T H E S O L U T I O N
M A K E I T S O .
T E S T I N G A G A I N S T A M O D E L
F S C H E C K W AY
class FizzBuzzModel { public static string Translate(int value) { if (value % 15 == 0) return "FizzBuzz"; if (value % 3 == 0) return "Fizz"; if (value % 5 == 0) return "Buzz"; return value.ToString(CultureInfo.InvariantCulture); } } [TestCase] public void Will_Match_Model() { Spec.ForAny<int>(value => FizzBuzzModel.Translate(value).Equals(_fizzBuzzer.Translate(value))) .QuickCheckThrowOnFailure(); }
class FizzBuzzModel { public static string Translate(int value) { if (value % 15 == 0) return "FizzBuzz"; if (value % 3 == 0) return "Fizz"; if (value % 5 == 0) return "Buzz"; return value.ToString(CultureInfo.InvariantCulture); } }
[TestCase] public void Will_Match_Model() { Spec.ForAny<int>(value => FizzBuzzModel.Translate(value) .Equals(_fizzBuzzer.Translate(value))) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
H M M M M
W E C O U L D U S E A S Y S T E M T O G E N E R AT E
D E G E N E R AT I V E T E S T D ATA T O D I S P L AY C O R R E C T N E S S
M A K E I T S O .
T E S T I N G N E G AT I O N
F S C H E C K W AY
[TestCase] public void Numbers_Not_Divisible_By_3_Do_Not_Fizz() { var noThrees = from number in Any.OfType<int>() where number%3 != 0 select number; Spec.For(noThrees, x => _fizzBuzzer.Translate(x) .Contains("Fizz") == false) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
[TestCase] public void Numbers_Not_Divisible_By_5_Do_Not_Buzz() { var noFives = from number in Any.OfType<int>() where number%5 != 0 select number; Spec.For(noFives, x => _fizzBuzzer.Translate(x) .Contains("Buzz") == false) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
H M M M M
[TestCase] public void Numbers_Divisible_By_15_Must_FizzBuzz() { var onlyFifteens = from number in Any.OfType<int>() where number%15 == 0 select number; Spec.For(onlyFifteens, x => _fizzBuzzer.Translate(x) .Equals("FizzBuzz")) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
H M M M M
[TestCase] public void Classify_Values() { Spec.ForAny<int>(x => true) .Classify( x => _fizzBuzzer.Translate(x).Equals("Fizz"), "Fizz") .Classify( x => _fizzBuzzer.Translate(x).Equals("Buzz"), "Buzz") .Classify(x => _fizzBuzzer.Translate(x).Equals("FizzBuzz"), "FizzBuzz") .Classify(x => _fizzBuzzer.Translate(x) != string.Empty, "Other") .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests. 55% Other. 22% Fizz, Other. 13% FizzBuzz, Other. 9% Buzz, Other.
H M M M M
[TestCase] public void Collect_Values_For_Negatives() { Spec.ForAny<int>(x => true) .When(x => x < 0) .Collect(x => _fizzBuzzer.Translate(x)) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
G R E AT !
T H A N K Y O U
M I K E H A R R I S @ M I K E M K H H T T P : / / C O M P - P H I L . B L O G S P O T. C O M /
Recommended
