Unit-Testing Bad-Practicesby Example
Benjamin Eberlei
direkt effekt GmbH
IPC 09, Karlsruhe
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 1 / 47
About Me
I Benjamin Eberlei
I direkt effekt GmBH
(digital marketing)
I Open Source contributor
(Zend Framework and Doctrine 2)
I Twitter @beberlei
I Blog: www.whitewashing.de
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 2 / 47
And You?
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 3 / 47
Why Test Quality Matters
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 4 / 47
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 5 / 47
“We spent 90% of the time
modifying existing tests to
acommodate for a relatively
minor change.“
(G. Meszaros, xUnit Test Patterns)
Safety Net vs Dead Weight
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 6 / 47
Test Smells
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 7 / 47
“Smell you later!”
(Nelson, The Simpsons)
Code Duplication
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 8 / 47
ZF Controller Action
public function testInitView (){
Zend_Controller_Front :: getInstance ()->setControllerDirectory(’/_files ’);
require_once ’/_files/ViewController.php’;$controller = new ViewController(
new Zend_Controller_Request_Http (),new Zend_Controller_Response_Cli ()
);$view = $controller ->initView ();$this ->assertTrue($view instanceof Zend_View);$scriptPath = $view ->getScriptPaths ();$this ->assertTrue(is_array($scriptPath));$this ->assertEquals(’/views/scripts/’, $scriptPath [0])
;}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 9 / 47
ZF Controller Action 2
public function testRenderByName (){
$request = new Zend_Controller_Request_Http ();$request ->setControllerName(’view’)
->setActionName(’test’);$response = new Zend_Controller_Response_Cli ();Zend_Controller_Front :: getInstance ()->
setControllerDirectory(’/_files ’);require_once ’/_files/ViewController.php’;$controller = new ViewController($request , $response);
$controller ->testAction ();$this ->assertContains(’In the index action view’,
$response ->getBody ());}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 10 / 47
ZF Controller Refactoring
Extract Test Utility Method:public function createViewController($controllerName=null ,
$actionName=null){
$request = new Zend_Controller_Request_Http ();if($controllerName !== null) {
$request ->setControllerName($controllerName);}if($actionName !== null) {
$request ->setActionName($actionName);}$response = new Zend_Controller_Response_Cli ();
Zend_Controller_Front :: getInstance ()->setControllerDirectory(’/_files ’);
require_once ’/_files/ViewController.php’;
return new ViewController($request , $response);}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 11 / 47
ZF Controller Refactoring 2
public function testInitViewRefactored (){
// fixture setup$controller = $this ->createViewController ();
// execution$view = $controller ->initView ();$scriptPath = $view ->getScriptPaths ();
// assertions$this ->assertTrue($view instanceof Zend_View);$this ->assertTrue(is_array($scriptPath));$this ->assertEquals(
’/views/scripts/’, $scriptPath [0]);
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 12 / 47
ZF Controller Refactoring 3
public function testRenderByNameRefactored (){
// fixture setup$controller =
$this ->createViewController(’view’, ’test’);
// execution$controller ->testAction ();
// assertions$this ->assertContains(
’In the index action view’,$response ->getBody ()
);}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 13 / 47
Assertion Roulette
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 14 / 47
Doctrine ResultSetMapping
public function testBasicResultSetMapping (){
// Fixture Setup$rsm = new ResultSetMapping ();$rsm ->addEntityResult(
’Doctrine\Tests\Models\CMS\CmsUser ’,’u’
);$rsm ->addFieldResult(’u’, ’id’, ’id’);$rsm ->addFieldResult(’u’, ’status ’, ’status ’);$rsm ->addFieldResult(’u’, ’user’, ’user’);$rsm ->addFieldResult(’u’, ’name’, ’name’);// [..]
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 15 / 47
Doctrine ResultSetMapping 2
public function testBasicResultSetMapping (){
// [..]$this ->assertFalse($rsm ->isScalarResult(’id’));$this ->assertFalse($rsm ->isScalarResult(’status ’));$this ->assertFalse($rsm ->isScalarResult(’user’));$this ->assertFalse($rsm ->isScalarResult(’name’));
$this ->assertTrue($rsm ->getClass(’u’) ==’Doctrine\Tests\Models\CMS\CmsUser ’
);$class = $rsm ->getOwningClass(’id’);$this ->assertTrue(
$class == ’Doctrine\Tests\Models\CMS\CmsUser ’);
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 16 / 47
Doctrine ResultSetMapping 3
public function testBasicResultSetMapping (){
// [..]$this ->assertEquals(’u’, $rsm ->getAlias(’id’));$this ->assertEquals(’u’, $rsm ->getAlias(’status ’));$this ->assertEquals(’u’, $rsm ->getAlias(’user’));$this ->assertEquals(’u’, $rsm ->getAlias(’name’));
$this ->assertEquals(’id’, $rsm ->getField(’id’));$this ->assertEquals(’status ’, $rsm ->getField(’status ’));$this ->assertEquals(’username ’, $rsm ->getField(’user’));$this ->assertEquals(’name’, $rsm ->getField(’name’));
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 17 / 47
Eager Test
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 18 / 47
ezcUrl Test
public function testRemoveOrderedParameter (){
$urlCfg = new ezcUrlConfiguration ();$urlCfg ->addOrderedParameter( ’section ’ );$urlCfg ->addOrderedParameter( ’module ’ );$urlCfg ->addOrderedParameter( ’view’ );
$u = ’http :// www.example.com/doc/components ’;$url = new ezcUrl($u, $urlCfg);
//[..]}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 19 / 47
ezcUrl Test 2
public function testRemoveOrderedParameter (){
// [..]
// functionality tested in other tests before$this ->assertEquals(
array(’section ’ => 0, ’module ’ => 1, ’view’ => 2),$url ->configuration ->orderedParameters
);$this ->assertEquals(’doc’, $url ->getParam(’section ’));$this ->assertEquals(
’components ’, $url ->getParam(’module ’));
// [..]}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 20 / 47
ezcUrl Test 3
public function testRemoveOrderedParameter (){
// [..]
// Primary Assertion according to test method name$url ->configuration ->removeOrderedParameter(’view’);$this ->assertEquals(
array( ’section ’ => 0, ’module ’ => 1 ),$url ->configuration ->orderedParameters
);
// [..]?}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 21 / 47
ezcUrl Test 4
public function testRemoveOrderedParameter (){
// [..]
try{
$this ->assertEquals(null , $url ->getParam(’view’));$this ->fail(’Expected exception was not thrown.’);
} catch ( ezcUrlInvalidParameterException $e ) {$expected = "...";$this ->assertEquals($expected , $e->getMessage ());
}
// [..]?}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 22 / 47
ezcUrl Test 5
public function testRemoveOrderedParameter (){
// [..]
// try removing again - nothing bad should happen$url ->configuration ->removeOrderedParameter(’view’);try{
$this ->assertEquals(null , $url ->getParam(’view’));$this ->fail(’Expected exception was not thrown.’);
} catch ( ezcUrlInvalidParameterException $e ) {$expected = "...";$this ->assertEquals($expected , $e->getMessage ());
}}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 23 / 47
Fragile Test
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 24 / 47
Zend SOAP Wsdl Testfunction testAddBinding () {$wsdl = new Zend_Soap_Wsdl(
’MyService ’, ’http :// localhost/MyService.php’);$wsdl ->addPortType(’myPortType ’);$wsdl ->addBinding(’MyServiceBinding ’, ’myPortType ’);
$this ->assertEquals($wsdl ->toXml ()),’<?xml version ="1.0"? >’ .’<definitions xmlns ="http :// schemas.xmlsoap.org/wsdl/" ’. ’xmlns:tns="http :// localhost/MyService.php" ’. ’xmlns:soap="http :// schemas.xmlsoap.org/wsdl/soap/" ’. ’xmlns:xsd="http ://www.w3.org /2001/ XMLSchema" ’. ’xmlns:soap -enc="http :// schemas.xmlsoap.org/soap/
encoding /" ’. ’xmlns:wsdl="http :// schemas.xmlsoap.org/wsdl/" ’. ’name=" MyService" targetNamespace ="http :// localhost/
MyService.php">’. ’<portType name=" myPortType "/>’. ’<binding name=" MyServiceBinding" type=" myPortType "/>’
. ’</definitions >’ );}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 25 / 47
Obscure Tests
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 26 / 47
Global State: Zend Framework
// Zend_Controller_Action_Helper_ViewRendererTestprotected function setUp(){
$this ->request = new Zend_Controller_Request_Http ();$this ->response = new Zend_Controller_Response_Http ();$this ->front = Zend_Controller_Front :: getInstance ()
;$this ->front ->resetInstance ();$this ->front ->addModuleDirectory(’/_files/modules ’)
->setRequest($this ->request)->setResponse($this ->response);
$this ->helper = newZend_Controller_Action_Helper_ViewRenderer ();
Zend_Controller_Action_HelperBroker :: addHelper($this ->helper
);}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 27 / 47
Indirect Tests: ezcMvcfunction testInternalRedirect () {
$config = new simpleConfiguration ();$config ->route = ’IRController ’;$dispatcher = new ezcMvcConfigurableDispatcher(
$config );$dispatcher ->run();self:: assertEquals("BODY: Name: name , ".
"Vars: array ([CR] ’nonRedirVar ’ => 4,"."[CR] ’ReqRedirVar ’ => 4,[CR])", $config ->store);
}
function testExternalRedirect () {$config = new simpleConfiguration ();$config ->route = ’IRController ’;$dispatcher = new ezcMvcConfigurableDispatcher(
$config );$dispatcher ->run();self:: assertEquals("BODY: Name: name , ".
"Vars: array ([CR] ’nonRedirVar ’ => 4,"."[CR] ’ReqRedirVar ’ => 4,[CR])", $config ->store);
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 28 / 47
Test-Names: FLOW3 MVC
I dispatchCallsTheControllersProcessRequestMethodUntilTheIsDispatchedFlagInTheRequestObjectIsSet()
I dispatchThrowsAnInfiniteLoopExceptionIfTheRequestCouldNotBeDispachedAfter99Iterations()
I resolveControllerReturnsTheNotFoundControllerDefinedInTheFLOW3SettingsAndInjectsCorrectExceptionIfTheResolvedControllerDoesNotExist()
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 29 / 47
Test-Names: eZ Workflow
I testProperties()
I testProperties2()
I testProperties3()
I testProperties4()
I testProperties5()
I testProperties6()
I testProperties7()
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 30 / 47
Test-Names: eZ Workflow
I testProperties()
I testProperties2()
I testProperties3()
I testProperties4()
I testProperties5()
I testProperties6()
I testProperties7()
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 30 / 47
Slow Tests
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 31 / 47
Zend Service Amazon
public function setUp (){
$this ->_amazon = new Zend_Service_Amazon ();$this ->_query = new Zend_Service_Amazon_Query ()
$this ->_httpClient =new Zend_Http_Client_Adapter_Socket ();
$this ->_amazon ->getRestClient ()->getHttpClient ()->setAdapter($this ->_httpClient);
// terms of use compliance:// no more than one query per secondsleep (1);
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 32 / 47
Zend Service Amazon 2
public function testItemSearchMusicMozart (){
$resultSet = $this ->_amazon ->itemSearch(array(’SearchIndex ’ => ’Music ’,’Keywords ’ => ’Mozart ’,’ResponseGroup ’ => ’Small ,Tracks ,Offers ’
));
foreach ($resultSet as $item) {$this ->assertTrue(
$item instanceof Zend_Service_Amazon_Item);
}}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 33 / 47
Zend Amazon Refactored
public function setUpRefactored (){
$this ->_amazon = new Zend_Service_Amazon ();
$this ->_httpClient =new Zend_Http_Client_Adapter_Test ();
$this ->_amazon ->getRestClient ()->getHttpClient ()->setAdapter($this ->_httpClient);
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 34 / 47
Zend Amazon Refactored 2
public function testItemSearchMusicMozartRefactored (){
$this ->_httpClient ->setResponse(file_get_contents("ExpectedTestResponse.txt")
);
$resultSet = $this ->_amazon ->itemSearch(array(’SearchIndex ’ => ’Music ’,’Keywords ’ => ’Mozart ’,’ResponseGroup ’ => ’Small ,Tracks ,Offers ’
));
foreach ($resultSet as $item) {$this ->assertTrue(
$item instanceof Zend_Service_Amazon_Item);// Assert some relevant stuff now!
}}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 35 / 47
Conditional Logic
“Everyone knows that debugging is
twice as hard as writing a program in
the first place. So if you’re as clever as
you can be when you write it, how will
you ever debug it?” (Brian Kernighan)
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 36 / 47
FLOW3 Cache Frontend
public function theConstructorAcceptsValidIdentifiers () {$mockBackend = $this ->createMockBackend ();
$identifiers = array(’x’, ’someValue ’, ’123 fivesixseveneight ’,’some&’, ’ab_cd%’,rawurlencode(’package :// some/ $ &% sadf’),str_repeat(’x’, 250)
);
foreach ($identifiers as $identifier) {$abstractCache = $this ->getMock(
’F3\FLOW3\Cache\Frontend\StringFrontend ’,array(),array($identifier , $mockBackend)
);}
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 37 / 47
FLOW3 Cache Refactored/*** @dataProvider dataAcceptValidIdentifier*/
public function constructorAcceptsValidIdentifier($id) {$mockBackend = $this ->createMockBackend ();
$abstractCache = $this ->getMock(’F3\FLOW3\Cache\Frontend\StringFrontend ’,array(),array($id , $mockBackend)
);}static public function dataAcceptValidIdentifier () {
return array(array(’x’), array(’someValue ’),array(’123 fivesixseveneight ’),array(’some&’), array(’ab_cd%’),array(
rawurlencode(’package :// some/ $ &% sadf’)),array(str_repeat(’x’, 250))
);}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 38 / 47
Zend Server ReflectionClass
public function testGetMethods (){
$r = new Zend_Server_Reflection_Class(new ReflectionClass(’Zend_Server_Reflection ’)
);
$methods = $r->getMethods ();$this ->assertTrue(is_array($methods));foreach ($methods as $m) {
$this ->assertTrue($m instanceof Zend_Server_Reflection_Method
);}
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 39 / 47
A working implementation
class Zend_Server_Reflection_Class{
public function getMethods (){
return array ();}
}
Great, all tests pass!
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 40 / 47
Zend Server ReflectionClassTest Refactoring
public function testGetMethodsRefactored (){
$r = new Zend_Server_Reflection_Class(new ReflectionClass(’Zend_Server_Reflection ’)
);
$methods = $r->getMethods ();$this ->assertTrue(is_array($methods));$this ->assertEquals (3, count($methods)); // (!!)foreach ($methods as $m) {
$this ->assertTrue($m instanceof Zend_Server_Reflection_Method
);}
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 41 / 47
Zend Server ReflectionClassTest Refactoring 2
public function assertReflMethods($methods , $expected){
$this ->assertTye(’array ’, $methods);$this ->assertEquals($expected , count($methods));foreach ($methods as $m) {
$this ->assertTrue($m instanceof Zend_Server_Reflection_Method
);}
}public function testGetMethodsRefactored (){
$r = new Zend_Server_Reflection_Class(new ReflectionClass(’Zend_Server_Reflection ’)
);$this ->assertReflMethods($r->getMethods (), 3);
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 42 / 47
Mock-Overkill
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 43 / 47
FLOW3 MVC Dispatcher
public function testDispatch () {$mockRequest = $this ->getMock(’F3\FLOW3\MVC\
RequestInterface ’);$mockRequest ->expects($this ->at(0))
->method(’isDispatched ’)->will($this ->returnValue(FALSE));
$mockRequest ->expects($this ->at(1))->method(’isDispatched ’)->will($this ->returnValue(FALSE));
$mockRequest ->expects($this ->at(2))->method(’isDispatched ’)->will($this ->returnValue(TRUE));
$mockResponse = $this ->getMock(’F3\FLOW3\MVC\ResponseInterface ’
);
// [..]}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 44 / 47
FLOW3 MVC Dispatcher 2public function testDispatch () {
// [..]$mockController = $this ->getMock(
’F3\FLOW3\MVC\Controller\ControllerInterface ’,array(’processRequest ’, ’canProcessRequest ’)
);$mockController ->expects($this ->exactly (2))
->method(’processRequest ’)->with($mockRequest , $mockResponse);
$dispatcher = $this ->getMock(’F3\FLOW3\MVC\Dispatcher ’, array(’
resolveController ’),array(), ’’, FALSE
);$dispatcher ->expects($this ->any())
->method(’resolveController ’)->will($this ->returnValue($mockController))
;$dispatcher ->dispatch($mockRequest , $mockResponse);
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 45 / 47
Further Readings
I xUnit Test Patterns
Book by Gerald Meszaros, http://xunitpatterns.com/Test%20Smells.html
I TDD Anti Patterns
http://blog.james-carr.org/2006/11/03/tdd-anti-patterns/
I Misko Hevery’s Blog
http://misko.hevery.com/
I Google Testing Blog
http://googletesting.blogspot.com/
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 46 / 47
Thank You!
E-Mail: [email protected]
Twitter: beberlei
Blog: http://www.whitewashing.de
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices IPC 09, Karlsruhe 47 / 47