Embed Size (px)
Citation preview
www.kjross.com.au
1300 854 063
EVENT PROCEEDINGS
Most notable this year was the high number of attendees that are seriously involved with some form of agile software development methodology compared to previous years. Rather than the majority either “thinking about”, or “just starting” with agile, a significant number of delegates were managing testers within agile development teams and seeking to refine their approach.
Andrew Prentice from Atlassian led an engaging workshop session on the evolution of agile testing. The major shift at Atlassian was from Quality Assurance to Quality Assistance. This revolves around testers engaging with developers earlier in the lifecycle to provide static reviews and input on proactive test design as part of the development lifecycle. Through this more proactive approach, Atlassian has been able to reduce the ratio of testers to developers from 1:4 to 1:10.
Within this framework, code literacy and the ability to automate tests is an essential skill for testers. There was wide agreement that all mature agile testing practices have automation at the core. The close collaboration between testers and developers is driving adoption of common ALM tools, such as Microsoft Visual Studio / TFS or Jira / Zephyr. Behaviour Driven Development and a focus on defining clear, automated acceptance tests early in the development lifecycle is emerging as a common practice.
Catherine Keeffe shared extensively on CGU’s migration to an agile approach, noting that the change required careful management of organisational culture, with an emphasis on hiring people who are willing to embrace change. The shift of silo’d toolsets is frequently painful, and many organisations have opted to either migrate to a common toolset only for greenfield projects, or else archive historical test assets and start afresh.
MANAGING THE EVOLUTION OF AGILE TESTING
KEY TRENDS IMPACTING TESTING
The Cloud Migration Risk Mitigation workshop featured input from Chris Birmele from Microsoft and Andrew Phillips from Amazon Web services.
One key observation was that migration to the cloud is presenting a compelling business case for most organisations, and migration of at least some if not all business processes to a cloud-based IT infrastructure is being driven largely by business stakeholders, rather than IT departments, and is now seen as inevitable in many organisations. The availability of on-shore data centres has opened up the option of cloud-based infrastructure for more Australian organisations. Notable examples of large scale migration include NAB and Suncorp.
Cloud migration is driving a number of changes in testing.
Where test environments are hosted in the cloud, VERTICALISATION
MANAGEDSERVICES
KNOWLEDGE
TRANSFER
RESOURCE
PLACEMENT
CONSULTING
“Shifting from Quality
Assurance to Quality
Assistance”
MIGRATION TO THE CLOUD
www.kjross.com.au
1300 854 063
K. J. Ross & AssociatesSoftware Testing & ICT Risk Mitigation
EVENT PROCEEDINGS
leading organisations are using technologies like Chef or Puppet to provision test environments on demand, although these tools are not without their own challenges. Testing in an elastic cloud environment enables functional test environments to be temporarily scaled up to production configuration levels for performance testing, which, if the production environment is also in the cloud, can be as close to production equivalence as you need.
Risks evaluation and mitigation is a key conversation for test managers to be engaging in with business and IT stakeholders. Microsoft has produced a broad Cloud Risk Assessment Framework for organisations based on ISO 31000, and KJ Ross have produced a draft extension to that framework using ISO/IEC 25010 Software Quality
Characteristics, both of which may be used to drive risk assessment and test strategy workshops.
In most aspects, the risks being faced in a Cloud migration are similar to any technology change, however the risk profile for some areas may shift compared to on-premises systems. For example, the likelihood of a physical security breach is generally significantly lower for cloud hosted systems compared to on-premises, whereas the likelihood of a legacy application’s architecture failing to scale appropriately in an elastic cloud environment is higher compared to fixed capacity on-premises platforms.
MOBILE APPLICATION TESTING
There is no doubt that mobile platforms have become a high priority for many business systems, and are increasingly the primary delivery platform for user interaction. With this focus on mobile has come a raft of compatibility testing challenges that harken back to the bad old days of the browser wars. Michael Palotas shared his insights from leading the mobile test automation practice at eBay international, and many other insights were shared by workshop participants involved in rolling out mobile apps within their own organisations.
Key points that emerged from this discussion included:
» Aim for achieving automated test coverage of common Create Read Update Delete functions using device emulators (hosted on virtual machines) and complement this with manual testing on real
devices to cover off user experience and non-functional factors.
» Field testing is critical, as real-life application functional behaviour and performance issues can often only be discovered in field settings where network connectivity, noise levels and visibility vary significantly compared to lab environments.
» Analysis of your users’ actual behaviour is critical for identifying the priority devices that need to be covered – it may not be what you expect. There is never enough devices available, and Bring Your Own Device is common strategy to get coverage of both bleeding edge and superseded mobile hardware.
Context and device changes are common, and testers need to consider multi-device, multi-context scenarios.
www.kjross.com.au
1300 854 063
EVENT PROCEEDINGS
In this session we looked at the typical challenges test managers face around test data management, as well as the changing landscape of data quality, particularly in terms of Big Data and Business Intelligence systems.
Changes in the Privacy Act have already impacted test data management. Non-compliance with the stricter privacy act risks fines in the order of $1.7M for organisations, and over $300,000 for individuals. For many participants, this stronger regulatory framework around the use of production data has created the opportunity to properly address test data management issues within their organisation, enabling the adoption of formal test data management tools and helping drive the creation of dedicated test management roles. The problems of managing the scale and variety of data, as well as the challenges around appropriately aging data, have highlighted the need to form interdisciplinary teams which include representatives from architecture, infrastructure and development as well as testing, and leading organisations have adopted this tiger team approach.
Addressing quality issues in Big Data has been pushing organisations away from more traditional Extract Transform Load (ETL) testing, where parallel runs and other stepwise verification tests can be applied to provide assurance of data quality, toward a production monitoring model. The risk with Big Data business intelligence systems that are being used to drive real-time business decisions is that it is frequently hard to know whether the data analysis is correct, because businesses often don’t really know what to expect from that data.
One case study raised within the workshop highlighted the need to apply a continually evolving data quality monitoring process in order to detect data quality issues that can emerge over time as the content and structure of aggregated data changes. Sanity checking, using a common sense rules engine, is often a good first step in establishing a baseline, which can then be refined further if specific data quality issues emerge later.
BIG DATA / TEST DATA
KEY TRENDS IMPACTING TESTING
The forum also focused on process issues impacting Test Management. Many of us now work with distributed teams, either across the country or onshore and offshore, and pace of change in resourcing models for testing services has certainly stepped up a notch, with a larger number of private sector organisations as well as government organisations now embracing some form of outsourcing or managed service offering.
Forum participants engaged in some robust workshop discussion on the topic, with a number of key points emerging:
» Building trust with your outsourcing partner is critical to success, and that means sharing IP and
VERTICALISATION
MANAGEDSERVICES
KNOWLEDGE
TRANSFER
RESOURCE
PLACEMENT
CONSULTING
MANAGING DISTRIBUTED TEAMS
www.kjross.com.au
1300 854 063
EVENT PROCEEDINGS
MANAGING UPWARD
Shifting the focus toward managing upwards and outwards, in terms of Project Governance we had a number of experienced panellists share their journeys into senior test management roles, where their focus had to shift from managing the testing team to managing the team’s customers. The key issues discussed revolved around building influence within the broader organisation so that the testing team’s message would reach the right audience and be understood and acted upon appropriately.
For some people this meant learning to delegate the technical details, including the delivery of test summary reports, to members of their team, while they focused on understanding the broader business perspectives and helping other stakeholders interpret the test team’s output in business terms. For others the key challenges revolved around building a stronger culture of responsibility within both the test team itself and
the broader organisation through building clear lines of communication and establishing the responsibilities and limits of control for each role.
Regardless of the specific approach, courage is required to step outside of comfortable technical roles we may have had into organisational leadership roles where we may be the new kid on the block. As a relatively younger specialisation within the IT industry, the challenge for testing is that fewer people in IT leadership come from a testing background, and within our own organisations, we may need to be the pioneers in stepping into more senior organisational forums. Test Managers have much to offer in bringing a software quality perspective to organisational decision making. We also have a lot to bring back into our teams in order to ensure that this same focus on quality continues to grow at the grass roots level.
trusting individuals by including them as part of the team.
» Finding ways to build personal connections and mutual investment in each other helps to overcome the challenges of distance, culture and language.
» Make sure that outsourcing is solving your core business problem. If your core business is achieved through innovation and agility, outsourcing may not be what you need. If what you need is scale and efficiency for a well-defined process, then outsourcing could very well be your edge.
K. J. Ross & AssociatesSoftware Testing & ICT Risk Mitigation
“TEST MANAGERS HAVE MUCH TO OFFER IN BRINGING A SOFTWARE
QUALITY PERSPECTIVE TO ORGANISATIONAL DECISION MAKING”
