Upload
david-tzemach
View
379
Download
1
Embed Size (px)
Citation preview
THE COMPLETE GUIDE FOR SOFTWARE
INTEGRATION TESTING
DAVID TZEMACHWWW.MACHTESTED.COM
MAY 12 2017
AGENDA• WHAT IS INTEGRATION TESTING?
• THE INTEGRATION TESTING PROCESS
• WHEN SHOULD WE START INTEGRATION TESTS?
• WHY SHOULD WE USE INTEGRATION TESTS?
• INTEGRATION TESTS TECHNIQUES
• ENTRY AND EXIT CRITERIA
• BEST PRACTICES
WHAT IS INTEGRATION TESTING?
OVERVIEWINTEGRATION TESTING IS A TESTING APPROACH THAT WE USE TO TEST THE INTEGRATION AMONG TWO OR MORE
OBJECTS THAT SHOULD WORK AND INTERACT TOGETHER. DURING THE SOFTWARE DEVELOPMENT LIFE CYCLE, WE
WILL SEE THAT INTEGRATION TESTS ARE DONE ON DIFFERENT OBJECTS THAT ARE NOT DIRECTLY RELATED TO THE
SYSTEM COMPONENTS.
EXAMPLES:
• INTEGRATION TESTS AMONG THE SYSTEM MODULES (TESTING THE INTERFACE AND COMMUNICATION
AMONG MODULES).
• SYSTEM INTEGRATION TESTING (INTERFACES WITH EXTERNAL ORGANIZATIONS, INTEGRATION OF SYSTEM
PACKAGES, ETC.).
• INTEGRATION TESTS BETWEEN SYSTEMS AND OS.
INTEGRATION TESTING PROCESS
THE SEVEN PHASES OF INTEGRATION TESTING THERE IS A BASIC PROCESS THAT YOU CAN USE IN AN INTEGRATION TESTING PROJECT, THE PROCESS CONTAINS A FEW BASIC
PHASES THAT ARE RELEVANT TO ALMOST ANY OTHER TESTING PROJECT.
1. ANALYZE THE REQUIREMENTS AND SPECIFICATION OF EACH MODULE.
2. MAKE SURE THAT THE PERSON/TEAM THAT RUN THE TESTS ARE FAMILIAR WITH THE TECHNICAL DETAILS OF EACH MODULE
AND THE OVERALL SYSTEM ARCHITECTURE.
3. NOW THAT YOU ACQUIRE THE TECHNICAL DATA, REVIEW THE RISKS AND DETERMINE THE APPROACH TO REMOVE THEM.
4. DETERMINE THE TESTING STRATEGY WITH A CORRESPONDING TEST PLAN (TEST SCENARIOS, TEST CASES, EXPECTED
RESULTS, TEST PRIORITIES ETC.).
5. TEST EXECUTION (EXECUTING THE TESTS AND OPEN THE RELEVANT DEFECTS).
6. REVIEW OF THE TEST EXECUTION RESULTS AND RE-TEST IF NECESSARY.
7. REVIEW AND APPROVAL OF THE TESTING PHASE BEFORE PRECEDING TO THE NEXT TESTING LEVEL (SYSTEM TESTS).
WHEN TO START INTEGRATION TESTING?
IT’S TIME TO INTEGRATE!
INTEGRATION TESTING CAN BE PERFORMED ONCE WE HAVE COMPLETED AT LEASE TWO
COMPONENTS THAT WE CAN USE THROUGHOUT THE INTEGRATION TESTS.
[UNIT TESTING] -> [COMPONENT TESTING] -> [INTEGRATION TESTING] -> [SYSTEM TESTING]
WHY DO WE NEED TO USE INTEGRATION TESTS?
SO WHY DO WE NEED TO RUN INTEGRATION TESTS..?
THERE ARE MANY REASONS THAT CAN EXPLAIN THE NEED FOR THIS TYPE OF TESTS, LET’S EXAMINE
THREE OF THEM:
• ALTHOUGH EACH COMPONENT IS TESTED AND APPROVED (PRIOR TO THE INTEGRATION PHASE), THERE IS ALWAYS A CHANCE THAT THE INTERFACE THAT ALLOWING THE COMMUNICATION
BETWEEN THE MODULES CONTAIN DEFECTS THAT MAY TRIGGER FURTHER DEFECTS IN THE
INTEGRATED MODULES.
• INTEGRATION TESTS CAN LEAD TO MANY SITUATIONS THAT CAN TRIGGER DEFECTS THAT WILL
APPEAR ONLY IN SPECIFIC SITUATIONS SUCH AS CONFLICTS AND DEPENDENCIES BETWEEN TWO
MODULES.
EXAMINE THE BASIC SCENARIO
MODULE A SHOULD BE INTEGRATED WITH MODULE B, BOTH MODULES ARE TESTED AND READY TO INTERACT ONE
WITH EACH OTHER, NOW LET’S ASSUME THAT THE INTERFACE THAT SHOULD PASS THAT OUTPUT FROM MODULE A INTO MODULE B HAS A DEFECT THAT DOES NOT ALLOW HIM TO ENCRYPT/DECRYPT THE DATA.
AS A RESULT, THE TWO MODULES WILL FAIL TO INTERACT AND SOMETIMES, MODULE B WILL RECEIVE A CORRUPTED
AND UNEXPECTED DATA THAT WILL LEAD TO A MASSIVE FAILURE
INTEGRATION TEST TECHNIQUES
THE BIG BANG TECHNIQUETHIS TECHNIQUE IS ACTUALLY VERY SIMPLE…. BEFORE PERFORMING THE TESTS, DEVELOPMENT
TEAM WILL INTEGRATE ALL COMPONENTS/MODULES AT ONCE (SIMULTANEOUSLY) AND THEN
TESTED AS ONE BIG SYSTEM, ONCE DONE, THE INTEGRATION TESTS WILL START FOR SPECIFIC
COMPONENTS/MODULES.
THE BIG BANG TECHNIQUE (PROS VS CONS)
Advantages DisadvantagesMake sense when you need to test small systems Consumes the integration testing time (the integration tests can start only when
whole system is tested)
All integrations are done before starting the
integration tests
This technique is not relevant to large and complex systems.
Very difficult to trace the root cause of failures (Interfaces are not isolated during
the tests)
Increases the chance to miss the small interfaces that need to be tested (All
are integrated and tested at the same time)
THE INCREMENTAL TESTING TECHNIQUE
IN THIS TESTING TECHNIQUE, THE TESTING IS DONE WHEN MODULES THAT ARE LOGICALLY RELATED ARE INTEGRATED
ONE BY ONE AND TESTED, THIS APPROACH WILL CONTINUE UNTIL THE LAST MODULE IS INTEGRATED AND TESTED.
TO USE THIS TEST TECHNIQUE, WE WILL NEED TO USE STUBS AND DRIVERS THAT WILL HELP US TO RUN THE TESTS
WITHOUT THE NEED TO IMPLEMENT THE ENTIRE MODULE (SIMULATE THE COMMUNICATION AND DATA TRANSFERRING
BETWEEN THE MODULES).
WHEN USING THE INCREMENTAL APPROACH, WE CAN USE TWO DIFFERENT METHODS CALLED “TOP-DOWN” AND
“BOTTOM-UP”.
TOP-DOWN INTEGRATION TESTINGIN THIS TESTING TECHNIQUE, THE TESTING IS DONE WHEN MODULES THAT ARE LOGICALLY RELATED ARE INTEGRATED ONE BY ONE
AND TESTED, THIS APPROACH WILL CONTINUE UNTIL THE LAST MODULE IS INTEGRATED AND TESTED.
DUE TO THE NATURE OF THIS TECHNIQUE, WE WILL ALWAYS HAVE A TOP MODULE THAT CAN INTERACT WITH A LOWER MODULE, THE PROBLEM BEGINS WHEN THE LOWER MODULE IS STILL NOT DEVELOPED OR JUST NOT READY FOR INTEGRATION, AS A RESULT WE
CANNOT RUN THE TESTS WHICH AFFECT THE PROJECT TIMELINES.
TO OVERCOME THIS PROBLEM AND REDUCE THE WASTE OF TIME WHILE WAITING TO THE READINESS OF THE LOWER MODULE, WE
WILL USE “STUBS”, STUB IS A TERM THAT REPRESENTS A CODE SNIPPET THAT CAN BE USED TO ACCEPT THE REQUEST FROM THE TOP
MODULE AND RETURN A RESULT SIMILAR TO A REAL MODULE.
TOP-DOWN INTEGRATION TESTING(PROS VS CONS)
Advantages DisadvantagesIt’s easier to find the Root cause of defects (A Single integration
tested each time)
Although stubs are similar to the real module, we will still need to make
further tests once the module is ready, which consume more time.
This approach will allow the team to rank the major flows first, as
a result, the most important defects will be found and removed
before other defects that are less important.
In complex systems that built with many modules, we will need to develop
multiple stubs
Comparing to the “Big-bang” approach, we can save time
we do not need to wait for all modules to be developed
In real world testing scenarios that involve complex design logic, you may
have trouble to predicting how the stubs will react during the tests.
often, you will find that the development of stubs will cost more than the
actual module development
BOTTOM-UP INTEGRATION TESTING
IN THIS TECHNIQUE, WE WILL START THE TESTS FROM THE LOWEST MODULE OF THE APPLICATION AND GRADUALLY PROGRESSES
TOWARDS THE TOP MODULES OF THE APPLICATION UNTIL ALL MODULES ARE INTEGRATED AND VALIDATED AS A WHOLE SYSTEM.
THE PROBLEM WITH THIS TECHNIQUE IS SIMILAR TO THE PROBLEM THAT WE HAD IN THE “TOP-DOWN” TECHNIQUE WHEN YOU
START TO TEST THE INTEGRATION BETWEEN TWO MODULES THAT ONE OF THEM IS STILL NOT READY TO BE INTEGRATED AND TESTED
(THE HIGHER MODULE IN OUR CASE).
TO OVERCOME THIS PROBLEM IN THE “TOP-DOWN” TECHNIQUE, WE USED “STUBS” THAT ARE NOT RELEVANT TO OUR APPROACH
WERE YOU NEED TO USE ANOTHER SIMULATOR CALLED “DRIVER” THAT WILL ALLOW YOU TO USE THE DRIVER TO TRIGGER/CALL THE
FUNCTIONS FROM THE LOWEST MODULE AND TO SIMULATE THE TEST INPUTS TO THE TESTED MODULE.
BOTTOM-UP INTEGRATION TESTING(PROS VS CONS)
Advantages DisadvantagesDefects that are found in the lowest modules are easier to detect
The whole system is not built until the creation of the last Top module
It’s easier to find the Root cause of defects (A Single integration is tested each time)
The critical design flows are tested at the end which, increases the risk to find critical defects in this flow
Comparing to the “Big-bang” approach, we can save time because we do not need to wait for all modules to be developed
ENTRY AND EXIT CRITERIA FOR INTEGRATION
TESTING
ENTRY CRITERIA FOR INTEGRATION TESTING • THERE IS A TECHNICAL DOCUMENT THAT DESCRIBES THE EXPECTED BEHAVIOR OF EACH INTERFACE.
• ALL MAJOR/CRITICAL DEFECTS FROM THE EARLY TESTING PHASES ARE CLOSED AND VERIFIED.
• THE TEST ENVIRONMENT IS READY TO SUPPORT THE TEST SCENARIOS.
• THE MODULES ARE INTEGRATED TOGETHER AND READY FOR TESTING.
• THERE IS A DOCUMENT THAT CONTAINS THE TEST STRATEGY.
• EACH COMPONENT IS TESTED AND APPROVED.
• EACH COMPONENT HAS A LIST OF UNIT TESTS.
• ALL RISKS ARE ANALYZED AND REMOVED.
• ALL MODULES ARE IDENTIFIED.
EXIT CRITERIA FOR INTEGRATION TESTING
• ALL TEST CASES FROM THE TEST PLAN ARE EXECUTED AND APPROVED.
• ALL MAJOR/CRITICAL BUGS ARE RE-TESTED, FIXED AND VERIFIED.
• THE TEST PLAN IS SIGNED –OFF.
• A TEST REVIEW IS DONE.
BEST PRACTICESFOR INTEGRATION TESTING
HOW CAN WE ACHIEVE MORE FROM OUR TESTS? • IN ANY CASE THAT IT’S POSSIBLE, AUTOMATE YOUR TESTS, OTHERWISE, YOU
WILL RUN EACH TEST MANUALLY PER THE INTEGRATION OF EACH MODULE.
• MAKE SURE THAT YOU FAMILIAR WITH THE MODULES TO BE INTEGRATED
(ESPECIALLY THE DESIGN AND ARCHITECTURE OF THE MODULES).
• DETERMINE THE INTEGRATION TEST STRATEGY, THIS STRATEGY SHOULD ALLOW
YOU ACCOMPLISH THE TESTING GOALS.
• MAKE SURE THAT EACH COMPONENT IS TESTED AND APPROVED BEFORE
EXECUTING THE INTEGRATION TESTS.
• MAKE SURE THAT YOU HAVE A DETAILED TECHNICAL DOCUMENT THAT
EXPLAINS HOW THE INTERACTIONS BETWEEN EACH TWO MODULES
(INTERFACES AND HOW THEY SHOULD WORK).
FOR ADDITIONAL KB’S PLEASE VISIT MY BLOG
WWW.MACHTESTED.COM