Embed Size (px)
Citation preview
SQE Architecture – Have you lost your mind! Erik Stensland
Architecture The art and science of designing and erecting buildings and other physical structures.
It has also been adopted to describe the activity of designing any kind of system, and is commonly used in describing information technology.
Software Architecture The term software architecture denotes the set of practices used to select, define or design a software architecture.
SQE Architecture A set of practices; used to select, define and/or design the methodologies, approaches and tools for different types of software testing.
Motivation
Provides a basis for analysis of the testability of software systems’ before the system has been built.
Architectural strategies and decisions, can be re-used across multiple systems who require similar quality attributes or functionality.
Supports early design decisions that impact a system's development, testability, deployment, and maintenance life which can prevent schedule and budget overruns and mitigate risks.
Goal
Aventador Lamborghini LP 700-4 – Top Speed 217
Produce high quality software that is reliable, available, and performant.
How
! Be involved in the early software and system architecture phases, to ensure that testability is a consideration during design.
! To understand the software and systems from an architecture level by being able to link integration points, so they can guide development and testing strategies across boundaries.
SQE Architect duties Help drive down the overall cost of testing:
! Pulling teams together and facilitate technical alignment. ! Analyzing product lines and recognize testing gaps across multiple
teams. ! Knowing and understanding the customer. ! Collaborate and help facilitate diverse teams including Dev, Sys
Archs, Operations, PMO and Product Management. ! Ensure consistency of process and tools in a test organization. ! Bring about visibility into the product and help facilitate a
complete high-quality solution. ! Provide strategic technical leadership and direction for SQE
management.
The Case Study: Code Name - Manhattan
What is Manhattan
The “Discussion”
Culture
• “DevOps is not something you build, it's something you do. It is a methodology and a practice.”
• “DevOps is the set of methods in which developers and operations communicate and collaborate to deliver software more rapidly, more reliably, and with higher quality.”
• “DevOps, as a whole, encompasses the entire team, including QA, PMO and Product.”
• “Most importantly, DevOps is not exclusionary. All the participants in creating a product or system should collaborate from the beginning - business stakeholders, developers, operations, QA, security, network, and whoever else needed. “
People
YESTERDAY • Any Automation
experience • 5+ years of software testing
experience • Warm body
TODAY • BS in Computer Science • Software development experience • Experience with Java • Experience with REST services • Experience with Cloud technologies • Experience with CI • The “Right” Attitude
Methodology Risk Based Testing is a methodology that can be described as “prioritization” of various software and/or system changes on the basis of the risk they carry and create / execute tests according to the derived priority.
Challenges to Solve ! Overall cost of testing.
! Huge business requirements
! Time to market
! Limited resources
! Unstable environments
! Testing focuses on most essential functions first.
! Automation must be a priority.
Advantages ! Testing concentrates on the most important
matters first.
! Focus lies on risks to the product instead of just the functionality of the software changes.
! A large set of automated regression tests, testing is no longer an obstacle to releasing an update.
! Exploratory and Manual testing is only required for the new features or un-automatable tests.
! When there is a report of a bug, the DevOps team can improve or fix the system quickly.
Automation Strategy
Approach Continuous Testing Model using Risk Based Testing Methodology
Process Integrate application components frequently through automatically executing suites of functional, integration and performance tests during the build and deployment process
Mindset “Walk Away Automation”
Automation writing SO well and self contained it requires little to no maintenance.
Continuously add new automated Functional, Integration and Performance tests.
Technology Stack Web Services
Ziggy Java Framework Internally developed Java tool Used by both Developers and SQE’s Execution is fast and automated
User Interface – Unified Java Selenium Framework
Internally developed Java tool Mobile –
Calabash (iOS and Android) frameworks. Continuous automation
Jenkins Standardized method of executing automated tests either for the Continuous Integration or stand alone.
Defect Tracking Jira
Standardized method of reporting defects
LoadRunner Cloud generated load Rich Internet Applications Run isolated tests on functionality that may be in early development/service-based tests Easy script maintenance
Jmeter Used by both Developers and SQE’s Can be executed as part of the Continuous Integration process.
Test Case Management
Testlink Integration with automated test tools. Standardized way for all teams to create/execute test cases. Standardized reporting metrics
Software Engineering • Few Builds a week.
• ~1 Deployment every week or two. • Limited Unit testing • Manually decide to Build and Deploy
Software Quality Engineering • Very little architecture review for testability. • Heavy Manual testing.
• (weeks of regression testing) • Limited UI automated tests, little web services automated tests. • Automated test manually kicked off. • Some performance testing.
• Trending analysis, etc
No Metrics
Yesterday
Today Software Engineering
• 100’s of Builds a day / Developer. • Large amounts of Unit tests • Multiple Deploys a day
• Only if automated builds are Successful. • Auto Builds and Auto Deploys to all pre-prod environments.
Software Quality Engineering • Involved early in review of Software and Systems Architecture. • Impact Analysis for all software changes. • Code reviews and BOM reviews as needed. • Targeted Automated tests
• (Unit, Service, UI, Performance) • New feature Exploratory testing only (manual)
• Day(s) of manual testing instead of weeks • Automated Tests in All Pre-Prod environments as part of the Auto Builds, Auto
Deployment process • Continuously running Integrated tests and daily automated Acceptance tests in
Integrated environments. Dashboards capturing different types quality metrics
In Summary Software Quality Architecture plays a key role in any organization that is looking to release high quality, reliable and stable software.
Software Quality Architecture Key to driving down the cost of testing across the organization.
SQE Architects Ensure that testability is a consideration during design and helps drive down the overall cost of testing.
Risk Based Test Methodology Most effective testing methodology for DevOps teams.
Continuous Testing Model Increases productivity and decreases cost of testing.
SQE Tech Stack SQE is inline with the Software Development teams and provides a collaborative environment.
