Upload
icsm-2010
View
838
Download
1
Tags:
Embed Size (px)
Citation preview
Automatically Repairing Test Cases
for Evolving Method DeclarationsMehdi MirzaaghaeiFabrizio Pastore
Mauro Pezzè
Universitàdella Svizzeraitaliana
http://swiss-landmarks.ch/panos/Lugano9.jpg
Software evolves
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance };}
Software evolves
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance };}
Software evolves
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance };}
Software evolves
public class .....
public class AccountUtil { private double dailyInterestRate = 0.00005; private int interestTerm=365; public double interest(BankAccount account return account.getBalance() *dailyInterestRate*interestTerm; } ...
public class AccountFactory {
public static BankAccount create( AccountContext ctx, boolean special ){ BankAccount account = new BankAcccount( ctx.amount ); if ( special ){ account.setInterestRate(0.0001); } ...
public class .....
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance };}
Software evolves
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance };}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; };}
Original classes are modified
public class .....
public class .....
Test cases need maintenance
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance };}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; };}
Compilation error:Method deposit(Money) in the type BankAccount is not applicable for the arguments (int)
changes in method declarations: 23% changes
that impact on compilation
changes in method declarations: 23% changes
that impact on compilation
> 80% in maintenance releases
changes in method declarations: 23% changes
that impact on compilation
> 80% in maintenance releases
Automatic Repair to Save Effort
Automatic Test Repair
[Memon et. al. 2008, Grechanik 2009 ]Repair GUI tests
Well suited only for GUI tests
Automatic Test Repair
[Memon et. al. 2008, Grechanik 2009 ]Repair GUI tests Repair Oracles
[B. Daniel et al 2010]
testInterest(){ ... assertEquals( 50, result );} FAILURE: expected 50, found: 40
assertEquals( 40, result );
Well suited only for GUI tests Focus on oracles only
Automatic Test Repair
[Memon et. al. 2008, Grechanik 2009 ]Repair GUI tests
Refactoring Techniques [ReBa, Eclipse]
Repair Oracles[B. Daniel et al 2010]
testInterest(){ ... assertEquals( 50, result );} FAILURE: expected 50, found: 40
assertEquals( 40, result );
//calculate one year interestresult = account.interest();
result = account.interest( 0 );
Well suited only for GUI tests Focus on oracles only
Prevent only some compilation errors
Automatic Test Repair
TestCareAssistant repairs test case compilation errors
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes
TestCareAssistant repairs test case compilation errors
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes
TestCareAssistant repairs test case compilation errors
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes
TestCareAssistant repairs test case compilation errors
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes
TestCareAssistant repairs test case compilation errors
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes
TestCareAssistant repairs test case compilation errors
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes
TestCareAssistant repairs test case compilation errors
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes
Repair parameter type change
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.CentsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
Which parameter to initialize?
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.CentsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
Which parameter to initialize?
Use code diff to identify the parameter1
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.CentsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How to initialize a complex object?
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How to initialize a complex object?
Find first uses of parameter fields in Version12
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How to determine fields values?
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How to determine fields values?
Diff to identify corresponding variable in Version 03
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
What was the original value?
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.CentsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
What was the original value?
Analyze the def-use chain of the variable back to the definition in the test4
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.CentsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
What was the original value?
Analyze the def-use chain of the variable back to the definition in the test4
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.CentsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How can we repair the test?
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How can we repair the test?
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How can we repair the test?
Instantiate the new object, use original values to initialize fields5
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
How can we repair the test?
Instantiate the new object, use original values to initialize fields5
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
public class BankAccount { private int balance;
public void deposit(int cents){ balance += cents; }
public int getBalance(){ return balance; }}
public class BankAccount { private int balance;
public void deposit(Money money){ balance += money.centsValue; }
public int getBalance(){ return balance; }}
testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}
Version 0 Version 1
To what extent TestCareAssistant can fix test cases automatically?
To what extent TestCareAssistant can fix test cases automatically?
• Repair 22 test cases of 6 open source systems
• Continuum, Geronimo, xml-security, PMD, POI, Shindig
• 24 compilation errors caused by different changes
• 9 parameter types changed, 8 parameters added, 3 parameters removed, 4 return types changed
To what extent TestCareAssistant can fix test cases automatically?
Results Total Repaired %Test Cases 22 16 72
Compilation Errors 24 18 75
Values Initialized 36 29 80
• Repair 22 test cases of 6 open source systems
• Continuum, Geronimo, xml-security, PMD, POI, Shindig
• 24 compilation errors caused by different changes
• 9 parameter types changed, 8 parameters added, 3 parameters removed, 4 return types changed
To what extent TestCareAssistant can fix test cases automatically?
Results Total Repaired %Test Cases 22 16 72
Compilation Errors 24 18 75
Values Initialized 36 29 80
• Repair 22 test cases of 6 open source systems
• Continuum, Geronimo, xml-security, PMD, POI, Shindig
• 24 compilation errors caused by different changes
• 9 parameter types changed, 8 parameters added, 3 parameters removed, 4 return types changed
To what extent TestCareAssistant can fix test cases automatically?
• Static data flow analysis not always effective
• Use of complex data structures, e.g. hash tables
• Changes in method logic
• Changes in interfaces
Results Total Repaired %Test Cases 22 16 72
Compilation Errors 24 18 75
Conclusionspublic class BankAccount { public void deposit(int cents){ balance += cents; }}
public class BankAccount { public void deposit(Money money){ balance += money.CentsValue; }}
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
Conclusionspublic class BankAccount { public void deposit(int cents){ balance += cents; }}
public class BankAccount { public void deposit(Money money){ balance += money.CentsValue; }}
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());
Conclusionspublic class BankAccount { public void deposit(int cents){ balance += cents; }}
public class BankAccount { public void deposit(Money money){ balance += money.CentsValue; }}
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());
TestCareAssistant
Future Work
CopticChronology EthiopicChronology
Chronology + withUTC()
TestCopticUTC(){ .... assertEquals("AM",
copticChronology.withUTC() );}
Future Work
CopticChronology EthiopicChronology
Chronology + withUTC()
TestCopticUTC(){ .... assertEquals("AM",
copticChronology.withUTC() );}
TestEthiopicUTC(){ .... assertEquals("EE",
ethiopicChronology.withUTC() );}
CopticChronology EthiopicChronology
Chronology + withUTC()
Future Work
CopticChronology EthiopicChronology
Chronology + withUTC()
TestCopticUTC(){ .... assertEquals("AM",
copticChronology.withUTC() );}
TestEthiopicUTC(){ .... assertEquals("EE",
ethiopicChronology.withUTC() );}
CopticChronology EthiopicChronology
Chronology + withUTC()
Future Work
CopticChronology EthiopicChronology
Chronology + withUTC()
TestCopticUTC(){ .... assertEquals("AM",
copticChronology.withUTC() );}
TestEthiopicUTC(){ .... assertEquals("EE",
ethiopicChronology.withUTC() );}
CopticChronology EthiopicChronology
Chronology + withUTC()
Automate Test Generation by Identifying and Implementing Test Adaptation Patterns
Questions?public class BankAccount { public void deposit(int cents){ balance += cents; }}
public class BankAccount { public void deposit(Money money){ balance += money.CentsValue; }}
Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);
testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());
testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());
TestCareAssistant