6th Annual Event – Development Lifecycle Management For ... · 6th Annual Event – Development Lifecycle Management For Medical Device Summit orcanos 2013. From Generation Based

6th Annual Event –

Development Lifecycle

Management For Medical

Device Summit

orcanos 2013

From Generation Based Product

to a Brand New One

Accelerating Scrum through advanced

methodologies, testing & collaboration

Boaz Ben Nahum

Senior Software

Engineer

Hagai Livni

Head of Software

Validation

May 2013

− 3 −

Agenda

• Given Imaging Products and the PLCM process

• What Agile process was used in the Generation based

product

• Moving Forward to a new product using SCRUM

methodology “by the book”

• User stories

• Agile Planning

• Continuous integration

• Test planning and executing

• Tools in use

• How do we move forward in future projects

(opportunities for improvement)

− 4 −

Agenda



product



• User stories

• Agile Planning



• Tools in use



− 5 −

Given Imaging Products

• Established in 1998, and started selling our products in

2001

• Given develops, produces and distributes medical

systems for the gastrointestinal community, which

helps diagnose pathologies through visualization of the

GI tract as well as manometry measuring products

• Given products are distributed in more than 75

countries

• Just released our 8th generation of the software

• This project was developed for the first time in an

incremental approach (SCRUM)

• Started a completely new big project

− 6 −

Given Imaging Products

� PillCam COLON 2

� Safe, minimally invasive,

sedation-free, patient-

friendly modality to

visualize the colon and

rectum

� PillCam ESO 2

� Capsule endoscopy

procedure for visualization

of the esophagus

� The only capsule-

based pH test for

GERD

� Ambulatory pH testing

gold standard

� High-Resolution

Manometry with

physiology

visualization and data

analysis software for

identification of

motility disorders

Fully-integrated GI solutions

� Ingestible, wireless

motility capsule using

sensor technology to

measure pH, pressure

and temperature from

within the entire GI

tract

� Complete reflux

diagnostic solution for

both capsule and

catheter-based testing

4

� PillCam SB 2

� Most Widely used capsule

endoscpe for visualization

of the entire small bowel

� The standard of care in

numerous conditions

− 7 −

Given Imaging Workflow

Define features & effort

PM

MKT

Clinical analysis

Feasibility &

Fine-tuning features

SW R&D

Algorithms\HW

Clinical analysis

PM

MKT

SQA

CS

Extensive QA &

Final fine-tuning

SW R&D

V&V= SQA + Clinical

analysis (Alpha)

PM

MKT

3rd level support &

Field questions

CS

PM

MKT

− 8 −

The “Traditional” Process

Implementation – R&D

Verification - SQA

Internal Val.

(clinical staff)

Clinical Trials

‘last feature in’

full testing cycle

vs. regression

when to start? when to end?

− 9 −

The Agile Work Flow

Implementation + Verification

R&D, PM, and SQA

Bug fixing + Verification

R&D, PM, and SQA

Clinical Trials

‘last feature in’

full testing cycle

vs. regression

Internal Val.

(clinical staff)

− 10 −

Agenda



product



• User stories

• Agile Planning



• Tools in use



− 11 −

Previous* Status of SW at Given Imaging

• The 8th generation of our main software product,

“RAPID for PillCam®”, was developed, for the first

time, using an iterative approach, adopting Scrum

methodology

• 3-week long sprints were executed, practicing:

• Sprint planning using developer tasks

• Bi-Weekly stand-up meetings

• Task-oriented test execution using exploratory testing

method

• Sprint “retrospective” meetings

• Releasing sprint SW version to internal clients (marketing),

using Citrix XenApp

But…

* Previous = End of 2011

− 12 −

What can be improved?

• Plans & Specifications

Development Plans

• The planning unit is a

“Development Task”

• Development Tasks are

detailed using MS Project

and GANTT charts

• Tasks are “engineering-

centric” and can be hard to

test

SW Specifications

• The specification unit is a

“Software Requirement”

• SW requirements are detailed in

a monolithic SRS MS-Word

document

• SRS written by a single person

and available too late in the

process

The two types of entities are not connected!

− 13 −

Agenda



product



• User stories

• Agile Planning



• Tools in use



− 14 −

Moving Forward to a New Product

• Using SCRUM methodology “by the book”:

• User stories

• Agile planning

• Continuous Integration

• Test Planning and Execution

• Tools

• ALM – Application Lifecycle Management system

• Project Portal

• The most significant change was the introduction of

user stories into the process.

− 15 −

Why User Stories?

• User stories

• Emphasize the user’s goals, not the system’s attributes

• Can be verified by SQA

• Can be demoed to stake holders

• Bridge the communication gap between PM, Dev & SQA

• Support and encourage iterative development

• Provide means for flexible and efficient planning

• Each user story satisfies two goals at the same time:

• It describes the software requirement in detail

• It also describes a piece of development work for planning

purposes

• User stories bridge the gap between plans and

specifications!

− 16 −

How User Stories Are Applied?

• They are written collaboratively & incrementally by entire team

• Initial form may include a single sentence.

• That sentence may be refined during sprint planning

• Details emerge during the sprint by discussions between

developers, SQA & PM, and are written inside the User Story as

“Conditions of Satisfaction”

• The detailed user story serves as a first class Software

Requirement within the ALM system

• The list of “Done” user stories forms the software specification

As a <role> I want to <goal> so that <benefit>

Conditions of Satisfaction

1. …

2. …

3. …

− 17 −

Agile Planning Phases

• Backlog grooming

• Sprint planning

− 18 −

Backlog Grooming

• User stories are added to the backlog periodically by

individuals (developers, team leader, product

manager), in “New” status.

• We hold regular “backlog grooming” meetings where

we:

• Review new stories and either approve or reject them.

• Questionable stories are moved to “Pending Decision” status

for later discussion with PM

• Estimate user stories using story

points – playing "Planning Poker".

• Participants: SW team leader,

SW Director, Scrum Master

− 19 −

Backlog Grooming - How to Estimate Stories?

• User stories are estimated relatively using abstract

measures: story points

• Story point have no meaning per se, only relatively to other

stories – a 20 point story represents twice amount of work than

a 10 point story

• Sprint velocity

• Measured empirically (using points) for all past sprints

• It is then used to determine sprint capacity during planning

(with some adjustments)

• Estimation may be inaccurate for single story & single

developer• But tend to be more accurate for a team of developers over a

sprint

− 20 −

Sprint Planning

• Sprint Planning Meeting is held before each sprint

Participants: SW team leader, SW Director, Scrum Master, Product

Manager

Agenda:

• Decide on sprint goals.

• Set the number of story points for the coming sprint (based on

velocity, team’s availability, vacations, etc.).

• Allocate stories for the coming sprint.

• Assign stories to developers.

− 21 −

User Story Workflow

New

Open

In Work

Implemented

Done

Review

& Approve

Allocate

& Assign

Implement

& Test

Verify

− 22 −

Continuous Integration - Daily Process

• Daily automated process includes:

• Building the latest version of the software

• Deploying the built software on a dedicated server

• Executing the following test sets:

• Unit level tests

• Component level tests

• System level tests

• Notifying all relevant parties with the results

• The daily updated server is used for exploratory testing

and verification of user stories during the sprint

− 23 −

Continuous Integration – On Demand

• Team City System – by JetBrains• For every source code commit, performs full build and runs unit tests

• Average duration: 4 minutes

• If any unit test fails, developer is instantly emailed, and can fix the

problem even before it reaches daily build.

• Allows also monitoring build & tests stability over time

• Visual Studio add-in allows “remote run”: sending code changes for

build & test on the server, before committing to source control

− 24 −

Agenda



product



• User stories

• Agile Planning



• Tools in use



− 25 −

Test Planning and Execution

• Test Planning and Designing

• Exploratory testing > design within execution

• What to execute per sprint / where to focus

• Automation testing implementation

• Automation testing advantages

• Constrains and limitations

• Identifying the tests to be automated

• When to start automate (cost effective)

• Creating different test sets for different purposes

• Using KDT test design methodology

− 26 −

Test Planning and Designing

• Exploratory testing > design within execution during the

sprint

• Specific module testing (Exploratory testing - linked to the user

stories) on the daily build

• SQA-development interaction during iterations

• What to execute per sprint released version / where to

focus

• Pre-determined Baseline testing for the sprint’s release version

• Risk based tests execution

• Execution of all other tests in a FIFO method (executing the

earliest previous test first)

• Executing a full test cycle (all test suite) in pre determined

milestones within project

− 27 −

Automation Testing - Advantages & Limitations

• Automation testing advantages

• Exposes defects in the unit level at early stages

• Reduces testing cycles duration

• Overnight test run

• Ideal for having a ‘green light’ indication to enter a full testing

cycle

• ‘Clears’ the manual testers time to do more complex tests

• Constraints and limitations

• In the unit testing level – time allocation within the

development process

• In the components testing level – prof. resources are needed

• In the system testing level – execute the exact scenario vs. the

randomness which exists in manual tests

− 28 −

Automation Testing – What to Automate

• Identifying the tests to be automated

• Good test cases for automation are ones that are running

frequently and require large amounts of data to perform the

same action

• Test cases that require multiple data sets

• Frequently used functionality that introduces high risk

conditions

• Test cases that are impossible (or almost) to be performed

manually

• Test cases that are exposed in their execution to human error

• Test cases that take a lot of effort and time when executed

manually

− 29 −

Automation Testing –When Do We Automate

• Unit testing level

• As part of the development

• Unit tests can be created based on found and treated defects

• Components testing level

• Design in parallel to the components development

• Execute once the component was implemented AND test data is

available

• System testing level – once a GUI objects' properties

has been finalized

− 30 −

Automation Testing Implementation

• Creating different test sets for different purposes

• Unit level

• Creating the tests as part of the implementation of the relevant

function (part of the developer’s tasks)

• Using Nunit ,unit-testing framework, enabling execution of unit

tests in two modes: GUI Runner, and Console Runner

• Running the tests as a task on the build machine and generating a

report with email

• Component level

• Executed a set of the tests on the daily build

• Executing the full test suite at the end of the sprint

• System level

• Test Objects’ Repository qualifying

• Functional - Sanity test set

• Functional - Full test suite

− 31 −

Automation Testing Implementation - cont.

• Using KDT test design methodology

• QTP with KDT - KDT (Keyword Driven Testing) tool, running a

dedicated engine within the QTP which call other QTP

functions

• Bridging between the ‘tool experts’ and the ‘domain experts’

• Helping in design and execute tests sequences and test

scenarios

• Enabling ‘manual testers’ with basic QTP understanding to

design automatic tests

• QTP manage the tests’ infrastructure - object and function level

• KDT manage the tests’ design - sequence and test scenario level

− 32 −

Tools - ALM

Planning• User stories

• Developer’s Tasks

• Design Specifications

• Test cases

• Tests Sets for execution

Managing• Tasks and Stories ‘burn down’

status

• Tests Planning status (workflow)

• Product requirement’s coverage

Risks vs. SR

SR vs. SDD

SR vs. TC

• Test execution

• Recording automation test

execution results

• Defects / change requests

Records generating for the DHF

− 33 −

Tools – Project Portal

• A SharePoint-based portal is used for collaboration of

team members

• Allows anyone to contribute using Wiki-style web pages

• Includes:

• Educational materials & Best Practices

• Description of software architecture

• Deployment guide

• Description of Scrum Process & Ceremonies

• Primary usage: maintain sprint information

• Sprint Schedule

• Sprint Goals

• Sprint Retrospective Feedback

− 34 −

Tools – Project Portal

− 35 −

Agenda



product



• User stories

• Agile Planning



• Tools in use



− 36 −

How do we move forward

• Enforcing Done, Done approach

• Fully implemented

• Tests approved (after R&D review), executed and passed

• All defects are closed

� Shaping the process of test cases review by Dev. Team

� Improved feedback throughout the process by

� End Users

� Marketing

• Level of test coverage of a feature / function – where do

we stop?

• Building up automation test within the test design by

manual testers (“domain experts”) using the KDT

platform

− 37 −

How do we move forward – cont.

� Using automatic tools to measure code coverage (unit

and system level testing)

� Static code analysis tools use

� Adopting Test driven development approach

• Using the BI analysis tool for better planning,

controlling and reflecting progress to management

� Integration between the source control system and the

ALM system

• Shortening formal SQA phase (stabilization period) –

time from feature freeze to product release

Questions?

THANK YOU!!!

[email protected]

[email protected]

Documents

6th Annual Event – Development Lifecycle Management For ... · 6th Annual Event – Development Lifecycle Management For Medical Device Summit orcanos 2013. From Generation Based