SW Testing -part of a better Process
CERN-BE-BI-SW Training DayM.Andersen
Testing late vs. early
Why is software late & buggy?
1. Rushed timetables ->Hope is lost 2. Lousy design -> Hope is there 3. Inadequate testing -> Hope is there
Jack Ganslle – Embedded SW Guru
Unlike HW which still can break under load, a tested SW stays always perfect!
How do we test?
cout<<“value=”<val<<endl;
System.out.println(“value=”+val);
Main(){MyNewClass a = new MyNewClass();a.doMethod(); a.doMethod();cout<< a.data()<<endl;
}
When to test?• When left to the end of the
project, which is running late... testing is the last thing on your mind.
• Final functional testing only covers 20% of the code!
• Only 20% is new code writing so you won’t escape, 80% is bugfixing and testing anyway!
• XP: Write software tests software, before or at least at the same time with coding features Output
Input
Test Driven Development 1
• Writing tests clarifies the purpose of the code.
• Goes in hand with specs• Reduces time in rework• Less time spent
debugging• Code simplification
1. You only do what is needed to test
2. You think in small modules to be tested
Test Driven Development 2
3. Makes code less coupled- thus reusable
Improves quality and reduces bugs by:
4. Making you think in terms of purpose and specs.
5. Simple≠Complexity↔Bugs
3. Maintainable
http://agilepainrelief.com/notesfromatooluser/2008/10/advantages-of-tdd.html
Does it work?
• "measuring a over 20 projects: if you have a large number of unit tests your code will be an order of magnitude (x10) less complex.“
• Controlled study results:1. “..quality increased linearly with the number
of programmer tests...”2. “..test-first students on average wrote more
tests and, in turn, students who wrote more tests tended to be more productive...”
http://agilepainrelief.com/notesfromatooluser/2008/11/misconceptions-with-test-driven-development.htmlhttp://collaboration.csc.ncsu.edu/laurie/Papers/TDDpaperv8.pdf
Types of SW test
Unit Test
Functional Test
Regression Test
Acceptance Test
Integration Test
Load Test
Good to be aware which one we are doing
Acceptance (Functional) Testing
Requirements documentFeature A
descriptionFeature B
descriptionFeature C
description
Acceptance Test documentTest Feature A:
test case atest case b
Test Feature Btest case dtest case etest case f
Test Feature Ctest case h
Example Functional specifications example
In some companies done by QA section and often outsourced
Unit Testing
• Individual components, modules, classes not the final application.
• Independent of context• Makes you think and
design modular code• Requires extra code• Easy in Java & Eclipse
using JUnit
JUnit Step 1Step 2
JUnit
Step 4
Step 3
assertTrue(d4.hashCode()==d5.hashCode()
);
Testing GUI in Java
Challenge: How to automate test for a GUI?• You need human to press buttons..• You need human to read out fields, graphs..
UISpec4J framework
• .jar library to put in a class pass.• Uses JUnit• Offers a large set of wrappers for Java Swing
graphical components• Allows users to write Unit and Full system
tests for Panels or whole GUI Apps
UISpec4J mechanism
JFrame
Window
JPanel
JMenuB
ar
Panel
MenuBar
getMenuBar()
getPanel()
JButton
JLabel
TextBoxgetText()
getButton(“Set”)
Button
click()
getSubMenu(“Open”)
MenuItem
getTextBox()
JMenu
UISpec4J Code
Integration Testing with Bamboo
• Build server runs on cs-ccr-builds (BE-CO)• Java projects are rebuild with their
dependencies• Executes Java JUnit tests at 2:00 at night• Provides a detailed status report from the test
runs.• Sends an email with report on success or fail.• http://builds.cern.ch/browse/LHCFB-OFSUTES
TS
Example: OFSU test on Bamboo
C++ testing
• BE-CO Fesa team started using Google test framework
• Profound experience. Direct quote: “ ..The problem with testing is that you discover how bad your code is!...”
• You see bugs in code you thought was OK• You see bad design & architecture (tight
coupled vs. Loose coupled code, complex dependencies...)
Google Test
C++ Testable code
1000+ lines of code inside RT actionSmall loosely coupled objects invoked from RT action
Execute() Execute()
Obj A
Obj B
Obj C
References
• http://collaboration.csc.ncsu.edu/laurie/Papers/TDDpaperv8.pdf
• http://www.uispec4j.org/getting-started• http://www.vogella.de/articles/JUnit/article.html• http://www.softwaretestinghelp.com/types-of-software
-testing/
• http://searchsoftwarequality.techtarget.com/resources/Software-Testing-Best-Practices
• http://www.robthefiddler.com/java/using-uispec4j/• http://java.sun.com/developer/Books/javaprogrammin
g/ant/ant_chap04.pdf