Upload
perforce
View
925
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Perforce is central to Spirent's global automated continuous change management system of their software products. Software changes and test case changes, both managed by Perforce, are continuously verified using an extensive integrated automated testing system. In this presentation, see how the system code management and test management systems are integrated, the metrics used to measure system performance, and the dramatic improvements to development and release efficiency that have been realized.
Citation preview
#
Marc HornbeekSr. Solutions ArchitectSpirent Communications
Experiences of Automated Testing in Continuous Change Management
#
Marc HornbeekSenior Solutions ArchitectSpirent Communications
Marc Hornbeek is Sr. Solutions Architect of automated test management solutions at Spirent Communications, Infrastructure Test Optimization (ITO) BU. He recently managed SCM and DevOps at Spirent. As primary architect of test automation tools and champion of test automation for firms ranging from start-ups to large multi-national companies he published more than 30 articles and has been a speaker at numerous international conferences and user forums.
#
• Network test systems and solutions• Global engineering teams• Transitions:
– Perforce to GitFusion/DVCS– SCM to DevOps– Large Waterfall to Agile Modular Releases– Train Model: “Freight train” > “Rapid Transit” > “Movers”
Spirent Communications
#
Continuous Change Management
Continuous Testing provides continuouscontrol for DevOps CCM
CI CT CD
#
Changes drive builds and tests
Continuous Test System
Test results drive changes
Change Management
System
Report and TrackingSystem
Build Systems
(Real or Virtual) Test Systems(Real or Virtual)
Artifacts Repository
SUT Systems(Real or Virtual)
Process Automation Tool
Poll
Start
Checkout
StartRun
PullPush
Reports And Logs
PullPull
Change Request
Run
#
Agile Train Model CTThis presentation describes Spirent’s journey of testing solutions and tools for faster and more modular trains.
#
Train Model Release Velocity
Large infrequent releases
Monthlysegment releases
Continuouspackagereleases
Separatetest setups
Networked DevOps
Central DevOps
Freight Train Rapid Transit Modular Mover
#
Continuous ControlEntiresystem
Segmentspecific
Modularpackages
Systemdecisions
Modularreverts
Integration stage tests and reverts
#
Build / Test System Cycle VelocityLarge infrequent releases
Monthlysegment releases
Continuouspackagereleases
Full build& regress96 hour
Incr. build & regress1 to 8 hours
Incr. build full regress 32 hours
#
Lower Test Cost, Higher CoveragePhysical labs
Virtuallabs
Fixed multiple ofsingle lab
Shared resource
Cloudlabs
On-demandresource
#
Test Version ManagementSeparate agents
Centralagent
Distributedagents
Singledepot
GitFusion / DVCS
Databasecontrolled
#
Test SuitesEntire SUT
SUT Segment
Modularpackages
Fixedsuites
Modular sequencedsuites
Taggedsuites
#
Test AnalysisEntire SUT
Segmentspecific
Modularpackages
Fixedreport
Analysis network
Segmentanalysis
#
CT Admin / % AvailabilitySeparate agents
Centralagent
Distributedagents
Various admin tools<90% Avail.
Central admin tools> 99% Avail
Central admin tools > 95% Avail
#
Engineering Process and CultureSeparate Processes
CommonProcess
DistributedProcess
Distinctcultures
Culture alignments
Cultureadjustments
#
• Culture changes • Solution architecture• Modular systems• Change-driven
automation• ROI driven• Incremental budget
Pitfalls and Lessons Learned
#
• Change-driven configuration• Scalability desktop to global lab• Mixed virtual and real SUT • KPI Dashboard• Novice to expert user interfaces• Services for test creation (Create), test set-up
(Connect), test execution (Play) and test results analysis.
CT Toolkit Checklist
#
• Seamless integration, rich APIs and plug-ins.• Pre-packaged tests• Variable skill level test IDE• Automated results
CT Toolkit Checklist (continued)
#
• CT automation journey is challenging but ROI is high: velocity, stability, coverage
• Each CT capability level achieved enables the next so keep the train moving!
• Choose CT automation solutions and tools with change-driven control capabilities
Summary
##
Thank you!Marc [email protected]
#
RESOURCESCD Report: info.perforce.com/cd-report
White Paper: perforce.com/white-paper-link
Template: perforce.com/template