Research seminar
1. Dr. S. Vilkomir, Research directions in software testing.
2. Brandi Amstutz, Combinatorial Approach to Testing Mobile Applications: An Experimental Investigation.
Monday, October 28 2013
Room 232 S&T Bld. 1
Research directions
in software testing
Dr. S. Vilkomir
1. Effectiveness of combinatorial testing for logical expressions
2. Effectiveness of MC/DC
3. Testing mobile applications on different devices
1. Effectiveness of combinatorial testing for logical expressions
Combinatorial testing:
• Each choice
• Base Choice
• Pair-wise
• t-way
Pair-wise testing
–Dest: London, Paris, Chicago, NY
–M: 1000, 2000, 3000
–Direct: Y, N
Dest M Direct
t1 L 1 Y
t2 L 2 N
t3 L 3 -
t4 P 1 -
t5 P 2 Y
t6 P 3 N
t7 C 1 N
t8 C 2 -
t9 C 3 Y
t10 N 1 Y
t11 N 2 N
t12 N 3 -
Pair-WisePair-Wise: A value of each level for each factor must be combined : A value of each level for each factor must be combined with a value from every levels for each other factor.with a value from every levels for each other factor.
Our results
• Fault Evaluator tool
http://core.ecu.edu/STRG/tools.html
• Four research papers:
• PSAM 10, Seattle, Washington, USA
• PIC-2010, Shanghai, China
• ICST 2012, Montreal, Canada
• ICSTW 2013,Luxembourg, Luxembourg
http://core.ecu.edu/STRG/publications.html
New book
Introduction to Combinatorial Testing
by
D. Richard Kuhn,
Raghu N. Kacker, and
Yu Lei
2. Effectiveness of MC/DC
• “Independent affect” is the main idea of Modified Condition/Decision Coverage (MC/DC)
• Suggested in 1992-1994 for avionic software
• DO-178B. Software Considerations in Airborne Systems and Equipment Certification. USA, 1992.
• Chilenski, J. and Miller, S. Software Engineering Journal, 1994
Modified Condition/Decision Coverage
• Every point of entry and exit in the program has been invoked at least once, every condition in a decision in the program has taken on all possible outcomes at least once, every decision in the program has taken all possible outcomes at least once, and each condition in a decision has been shown to affect the decision's outcome independently.
A condition is shown to affect a decision's outcome independently by varying just that condition while holding fixed all other possible conditions.
From DO-178B.
9
3
1
2
• Do these test cases satisfy MC/DC?– t1: <1, 0, 1> d= 1
– t2: <0, 1, 1> d= 1
– t3: <0, 1, 0> d= 0
– t4: <0, 0, 1> d= 0
• Holding conditions – OK• Affect the decision's outcome – OK• Answer: YES
d = A (B C)
Plans – to continue this research direction
3. Testing mobile applications on different devices
Practical questions
• No scientific approaches
• No justification
• Optimal set of devices?
• Our approach – next presentation