Embed Size (px)
Citation preview
Cost & quality effective management of your test environments
Dennis JanssenTestFrame Research Centre, CMG
18 April 2023
2
Agenda
•What can go wrong with a test environment and why does it happen so often?
•Situation in practice
•Technical solutions and results of its implementation
•Orginazational solutions and results of its implementation
3
What can go wrong?
•Errors due to environment instead of “real” software failure
•False sense of security (testing against the wrong environment)
•Delays in testing (and therefore the whole project) due to environmental problems
•Loads o’ money spent!!!!
4
Why does it go wrong?
• It’s very complex to build in todays complex architectures
• Most are project based, so…• … focus on 1 IT system instead of business processes
• … for the duration of the project
• … no core business for developers
• Testdata problems, no dedicated environments
• No maintenance
• De-centralized, no co-ordination
5
The situation we found…
6
The technical environment
• Tandem
• IBM OS/390
• IBM RS6000
• PC-Lan
• Web Architecture
• Siemens Nixdorf
• Philips
• DECC/ Alpha
• Sun Solaris
• Windows 95/ 98/ NT
• Websphere
• CICS
• MQ-Series
• Netweave
• Generic Service Layer
• To name but a few…
7
The organisation
• Very de-centralized
• Every project operates almost alone, loads of hardware
• Lots of “shopping” needed to build a test environment (> 10 contacts is not unusual)
• Test environment = numbre 1 problem for developers and testers alike (survey)
• No “owner” of the test environment problem
• Test environment maintenance? Never heard of!!
8
Problem Analysis
• Technical nature • Organisational nature
9
Technical Solution
• Insourcing of key information systems in a central test environment
• Master – copy concept
• Develop distribution procedures to create copies (software and data)
• Create dedicated environments
• Version control & configuration management is a must (master & copies)
• Consistent testdata (not frozen)
• Synchronizing with third party test environments
10
A
C
B
D
Master environment
BA
BA
C
Copy environments
C D
I
II
III
Basics of technical environment management
11
Results of technical solution
• 90% less duration when building atest environment (quicker work)
• 65 % reduction in costs of building a testenvironment (less work)
• Dedicated testenviroments per project (1:1)
• Almost no bugs (down by 95%) during testing caused by a mistake in the testenvironment (less wasted time and frustration during test execution phase)
• Synchronized testdata (also with third parties) so end-to-end tests are possible
12
Orginazational solution
• Seperate department for building AND maintaining test environments
• Testenvironments = core business
• Divided in tasks
• Generalists
• Start early in the project
• Single Point Of Contact (SPOC)
• Planning office
13
Order 1
TechnicalCo-ordinator A
Order 2
TechnicalCo-ordinator B
Order 3
TechnicalCo-ordinator C
Order 4
TechnicalCo-ordinator D
•Internal co-ordination•Account Management
•Accounting/ reporting•Planning•Copy procedures•Distribution procedures
•Update Master•Update Copies
•Create copy environments•Make tailor made solutions•Support during test execution phase
14
Results of organisational solution
• Always the right environment
• Always up to date testenvironments
• 95% of needed testenvironments is deliverd in time
• Support for the project is there when needed
• 50% less hardware needed because of explicite planning
• Less pressure on projecmanager, we take care of the environmental business (spoc)
15
Implementation challenges
• Challenges when implementing:
• De-centalized nature of organization
• “Not invented here”
• How to counter challenges:• Communicate top down and
bottom up
• Make small steps
• Make quick steps
• Give added value
• Give them a “good nights sleep”!
16
Summary
• Developing and maintaining test environments is complex work
• Challenges both in technique and organisation, solving only one is like solving nothing at all
• Create a dedicated unit to focus on test environments
• Be a SPOC for your customers (tread them as such!)
• Duration and cost will go down, quality will go up!!!!
