20
BPM-X GmbH | www.bpm-x.com | Public 1 Whitepaper Model-Based-Testing for ERP Model-Based-Testing for ERP An adaptive software solution generating test cases based on Blueprint business process models Whitepaper June 2011, by Dr. Harald Goebel Andrew Kashulin Heinz-Jürgen Scherer Dr. Boris Zinchenko

Model-based-testing for integration and regression test of ERP solutions

Embed Size (px)

DESCRIPTION

Business process models from the blueprinting phase of implementing SAP Solutions can easily be leveraged by the BPM-X Test Case Generator software for generating end-to-end test cases.

Citation preview

Page 1: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

1 Whitepaper – Model-Based-Testing for ERP

Model-Based-Testing for ERP

An adaptive software solution generating test cases based on Bl ueprint business

process models

Whitepaper

June 2011, by

Dr. Harald Goebel

Andrew Kashulin

Heinz-Jürgen Scherer

Dr. Boris Zinchenko

Page 2: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

2 Whitepaper – Model-Based-Testing for ERP

Table of Contents

Abstract 3

Motivation 3

Blueprinting an ERP Solution 4

Requirements for Testing 5

Approach for Model-Based Testing 6

Test Case Generation Algorithm 7

Execution Strategies for Test Cases 10

Adaptive Test Case Generation 11

Important Considerations 11

Interfaces and IT Systems 11

Work Products and Deliverables 12

Organizational and Resource Assignments 12

Costs Optimization of Test Case Analysis 12

End-to-End Process Execution Probability 12

Effort Weighted Testing 12

Methodology for Test Case Generation 13

ERP Solution Blueprinting 13

Modeling Method 14

Model and Diagram Data Interchange 14

Generation of Test Cases 15

BPMN Method Internally Used 15

Test Case Generation Based on the BPMN Method 16

Linking with Testing Tools 18

Summary 18

Bibliography 20

Page 3: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

3 Whitepaper – Model-Based-Testing for ERP

Abstract For the requirements definition of Enterprise Resource Planning (ERP) software, the management

discipline and methodology of Business Process Management (BPM) has an important role in the

Blueprint project phase of ERP implementation. Using BPM methodology, the business requirements are

documented and analyzed. Process improvement is achieved via better support of business activities by

ERP software. The Blueprint consists of functional, organizational and system process models developed

by a Business Process Analysis (BPA) tool prior to the realization phase of customizing the ERP software.

Because of the complexity of ERP software, the integration testing of ERP supported processes is a very

essential task to lower business impacts of badly tested software. At the same time, testing is a resource-

consuming task in critical project phases. This whitepaper discusses an adaptive software solution for

automatically transforming Blueprint business process models into test cases for a methodical and tool-

based integration testing.

Motivation ERP software is standard software from leading software vendors like SAP® or Oracle®. It comprises a

comprehensive support of business processes, and is generically developed for the enterprise market.

As implied by the definition of standard software, it is maintained by updates of the vendor and can be

updated. It is in the nature of things that ERP standard software tends to be a complex construct

consisting of many software components and configurations. ERP software needs to be customized to

implement requirements and support business processes of an organization. Customization is a standard

process in an ERP implementation, and is done by parameterization and change of the software

components like the programs and configuration originally supplied by the software vendors. Beside

functional acceptance, the software quality is influenced by errors in originally delivered software and

customization.

To achieve a qualified support of business processes, extensive software testing is mandatory not only

on the software development level, e.g. using unit tests. The maturity of customized software is achieved

by a continuous testing approach with functional use-case testing and integration testing. This is equally

true for traditional transaction-based ERP systems and Business Process Management Systems (BPMS)

with execution and integration of processes. Nowadays, SAP ERP is still a predominant transactional ERP

system where SAP transactions like forms support business activities.

Implementation of an ERP system is done through several project phases. SAP uses the Accelerated SAP

(ASAP) project methodology where Blueprinting is the design phase of the ERP solution. In the realization

phase, the ERP solution is customized based on the requirements specified in the Blueprint.

For the methodical approach of testing, the definition of test plans based on test cases is a challenging

task. To attain it, different members of the project team with various skills such as business key users,

business analysts, quality assurance (QA) and software implementation team members must work

together. It is even more complex because team members talk with different business or technical

languages and are distributed across different locations including the employment of System Integrator

Page 4: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

4 Whitepaper – Model-Based-Testing for ERP

offshore development teams. Primarily, we can consider three challenging aspects in

ensuring the quality of business processes ERP support:

Costs

The task of manually creating test cases is accomplished with high project efforts for the design of many

end-to-end business scenarios. The traditional design of test cases takes qualified and costly resources

out of the project core and testing teams. The business impact of insufficient integration testing reduces

production efficiency, and increase software maintenance costs.

Risks

Even if skilled project team members design the test cases, there is a high risk of not capturing all

relevant end-to-end scenarios to be executed as test cases, or having mostly incomplete test cases with

business impacts in the production environment. In productive operation, a process may fail because the

executed scenario was not tested or interfaces between different components were not identified for

appropriate interface integration tests. If a business process has changed in the context of risk

management like Sarbanes-Oxley Act (SOX), it is critical to miss the re-testing of process risk controls.

Quality

Bad test coverage leads to low software quality with high business impact. This negative impact is

increased by an incomplete matching of the test data and insufficient reviews of work products and

deliverables.

All in all, the business impact of insufficiently tested end-to-end scenarios results in low quality business

process support and higher software maintenance costs.

Blueprinting an ERP Solution For many years the Blueprint of an ERP solution was traditionally created via workshops to capture

business requirements from business key users and process owners. This process called “Blueprinting the

ERP solution” commonly uses tools from Microsoft Office to describe the organizational and functional

aspects with Word documents and Visio process flow diagrams.

The Business Process Management (BPM) concept is a management discipline that offers a more

comprehensive view of organizational, functional, data-centric, performance and process aspects of an

organization. BPA tools are used to describe the business architecture structured by management, core

and support process models in a single repository. The BPM approach for Blueprinting is supported by

software vendors like SAP® with Solution Manager and BPM or Oracle® with the Application Integration

Architecture (AIA) and the BPM Fusion Middleware.

As the use of Enterprise Architecture (EA) and BPM methodologies is becoming more and more popular

for large organizations, business processes are described by process models and their visual

Page 5: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

5 Whitepaper – Model-Based-Testing for ERP

representation which can show the process logic and flow with different levels of

abstraction. On a functional level, business processes are described by a model with a sequence of

activities depicting the business logic.

Figure 1 Business Process “Sales Order Processing“ supported by SAP ERP

Process models map business logic that describes end-to-end scenarios as sequences of business

activities.

The motivation for developing the solution described below is to use available Blueprint process models

to automatically generate test cases that can be used by test management tools to lower project costs

and reduce the risk of software quality impacts on IT supported business processes.

Requirements for Testing As ERP software is usually customized for organizational and business requirements, there is the

absolute need for test management with applied testing concepts. Similar to concepts like software unit

testing which are very important for software development on the coding level, testing procedures like

functional use-case testing and integration testing are crucial to ensure the quality of ERP support on the

business process level.

To achieve the goal of improving quality and minimizing risks, a certain level of completeness of the test

cases is needed. Testing efforts can be reduced by avoiding unneeded test cases, removing redundancy,

Page 6: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

6 Whitepaper – Model-Based-Testing for ERP

and improving automation by setting correct priorities for frequently-operated end-

to-end scenarios. It is important to identify gaps between business processes and related test cases to

achieve completeness of test case coverage.

Because of business and IT complexity, the emergence of Change Requests (CR) is not unusual in the

project life cycle.

Using BPM methodology and a BPA tool for Blueprinting an ERP solution reduces the absolute number

and complexity of CRs because of a comprehensive understanding of the business logic and

requirements. Each CR influences the scope of the software solution, and has an impact on the

customizing. A change in a business process is managed in an ERP project by Change Management. This

should include the impact of the changes with respect to related test cases. In order to achieve lower

testing efforts, redundant and obsolete test cases have to be removed; and to ensure quality, existing

test cases must be updated to reflect the change of the business process.

Under software maintenance, test cases should be periodically reviewed to detect the impact of change

requests.

Approach for Model-Based Testing Blueprint process models created in the BPA tool and imported for the generation of test case. The test

case data is utilized by the testing tool or for a manual test procedure.

Figure 2 Approach for model based testing

A test case is an end-to-end business scenario as a sequence of activities to be executed sequentially.

The process logic is explicitly formulated by using logical rules commonly expressed with exclusive

execution (XOR), parallel execution (AND) and inclusive execution (OR) of a branch in the process flow.

The OR rule is complex to understand. It can be comprehended as a combination of a XOR and an AND

rule [OS08]. Because process activities can consists of human or system atomic tasks (atomic means

here that a task cannot be divided further into subtasks, like a SAP transaction), a sub process is defined

by several tasks or a process interface. We use statements as designation of activities in a test case. The

statements are linked by transitions1 to the logical flow of an end-to-end scenario.

1 A transition is described in a process model by a connector between activities

Page 7: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

7 Whitepaper – Model-Based-Testing for ERP

Test Case Generation Algorithm To calculate the full set of potentially possible test cases, we implemented an algorithm adapted from

[OS08] for the linearization of the model graph into test case models. While [OS08] discusses the

generation of test cases based on the Event-Process-Chains (EPC) method, the algorithm used here is

based on BPMN to internally describe process models and test cases. With regard to control flow based

testing this approach aligns with statement condition coverage [WP.2].

To calculate the combinatory of statements suitable for execution as test cases, we utilize the basic

BPMN patterns as described in the DoD Enterprise Architecture Design Patterns [MZM09]. For the

linearization of statements, in addition to the DoD design patterns we have to consider the iteration

pattern.

Elementary sequence

Split and Join Patterns

Iteration

Elementary Sequence

The sequence is the fundamental idea of the chain of discrete individual statements following each other

in a certain order. It is the common semantic interpretation or implication causality.

Figure 3 Elementary sequence of statements

Split and Join Patterns

Split gateways have exactly one incoming and more than one outgoing connection, whereas join

gateways have more than one incoming and exactly one outgoing edge. Although implicit splits and joins

are not a good practice at all, they can be replaced by explicit gateways.

Page 8: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

8 Whitepaper – Model-Based-Testing for ERP

Pattern Linearization

Parallel AND split

1: A, B, C 2: A, C, B2

Exclusive choice XOR split

1: A, B 2: A, C

Multiple choice OR split

1: A,B 2: A, C 3: A, B, C 4: A,C, B2

Event-based choice XOR split

1: A, B 2: A, C

2 Only needed for C3 condition coverage, not needed for C2 path, C1 branch or C0 statement coverage.

Page 9: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

9 Whitepaper – Model-Based-Testing for ERP

Pattern Linearization

Synchronized AND join

1: A, B, C 2: B, A, C2

Unsynchronized XOR join

1: A, C 2: B, C

Synchronized OR join

1: A, C 2: B, C 3: A, B, C 4: B, A, C2

Implicit and explicit Iteration

The iteration is the fundamental idea of the similar repetitive sequence. Iterations use only XOR

connectors to create such a pattern.

Figure 4 Implicit iteration with F1, F2 and explicit iteration with F3, F4

Table 3 shows the results of the linearization of the iteration pattern of Figure 5.

The iteration is expanded into all possible sequences of statements.

Table 1 Linearization of the iteration

Page 10: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

10 Whitepaper – Model-Based-Testing for ERP

Runs (N) Linearization event

0 (F1) so-called “Top tested loop” 1 (F1, F2) N = 2,… (F1, F2, F1, F2, …)

The linearization of the iteration is simple, but in practice it requires handling additional reasonable

assumptions to reduce the number of test runs. An additional parameter in the generation algorithm of

test cases describes how many loops have to be executed for an iteration pattern. Per default 2 loops are

generated per iteration.

Conclusion

Based on a breadth first search (BFS) algorithm handling graph theory [MAI11] the logic of linearization

of statements finds all potential end-to-end scenarios with the linear sequence of statements describing

a test case. For complex business processes with deeply nested branches, the number of some

thousands of potential possible end-to-end scenarios exceeds the practically approach of executing the

generated test cases. The generation of test cases for practical use is improved by an adaptive algorithm

for test case coverage.

Execution Strategies for Test Cases Integration testing is a kind of control flow based testing procedure. There are different concepts to

reduce the potentially enormous number of possible test cases. For an example of a complex process

model with 10 XOR gateways and 2 branches each we will have 210 = 1024 test cases feasible for

execution. Because this number of potential test cases is too costly to be executed, the number should

be reduced by different test execution strategies.

The approach of narrowing the number of suitable test cases is called statement coverage test analysis

[WP.1]. In principle, 4 different statement coverage strategies CX (X=0..3) can be considered.

Table 2 Cases for test-case statement coverage

Case Coverage of Description Pro Con

C0 Statements All statements and gateways as nodes of the control graph will be covered by the testing not possible in a logical order.

Less effort, all statements of the process flow are executed.

Not all transitions (edges of the control graph) between statements will be passed through while testing the process model. All statements are weighted equally Empty branches are not covered

C1 Branches All edges of the control graph with all nodes and edges are tested at least 1 time. This includes the C0

Empty braches are covered

Dependencies of decisions are not covered Loops are not tested sufficiently

Page 11: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

11 Whitepaper – Model-Based-Testing for ERP

Case Coverage of Description Pro Con

coverage and all functional needed test cases, but not all test cases for integration testing.

Complex branches are weakly tested

C2 Paths Each rule with possible decision is included in a test case. High efforts by given number of possible test cases.

High error detection High effort for testing Based on conditions not all paths are covered

C3 Conditions All variations of rule decisions are covered in test cases. Very high effort for testing by potential number of test cases.

High error detection Very high effort

Adaptive Test Case Generation The test case generation software uses an adaptive approach to reduce the potential number of test

cases by the use of the parameter CX=0..3. For a practical use, branch coverage C1 coverage testing

deliverers good results of test quality with reasonable costs for the test case execution.

The testing condition coverage C3 is equivalent to the test case generation algorithm described in the

previous chapter. The adaptive approach starts with the generation of possible test cases. Each

generated test case is tested against the chosen C0 or C1 coverage.

For statement coverage, for each new generated test case it is tested against the available test case

statements if all statements of the process model are covered. For branch test cases this is the same

with edges tested against the process model instead of statements. Overlapping test cases are

eliminated to reduce the effort of test case execution.

Important Considerations Although the process is in the center of test case generation, it is important to understand the context of

a statement in the end-to-end scenario. The context is described by the preceding and subsequent

statements linked by transitions, the statement is supported by IT systems, the data of work products is

linked with the help of interfaces between different technical components. Resources support the

execution of the business process.

Interfaces and IT Systems

The activities (e.g. SAP process steps and transactions) are supported by IT systems. By the knowledge of

this support it is clear if a transition links to statements executed in different IT systems. The statement

associated work products have to be serialized and transformed using an interface between the two

different IT systems. It is very important to review this interface link in the execution of test cases.

Page 12: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

12 Whitepaper – Model-Based-Testing for ERP

Work Products and Deliverables

To properly describe test data and functional use case success criteria it is important to know which work

products are processed in the end-to-end business scenario.

Organizational and Resource Assignments

Finally, the organization and resources supporting the execution of statements being responsible,

accountable, controlled and informed (RACI) have extensive business knowledge which in many cases is

not described by Blueprint documents. They know e.g. how to handle exceptions in workflows, or know

about probabilities of executing branches in complex workflows. The algorithm uses this information for

cost optimization of test case analysis.

Costs Optimization of Test Case Analysis In the case of test case coverage C2 or C3, the potential number of possible test cases can be very large.

The calculation of execution probabilities or execution costs can be used to reduce the number of test

cases to be executed, e.g. to avoid the execution of test cases belonging to quantiles with lesser

probability like the first quartile [WP.1].

End-to-End Process Execution Probability

The probability of executing an end-to-end process called test case can be calculated based either on

“as-is” facts or on “to-be” estimations. As a fact in ERP, the number of transaction execution times in a

process can be used and stored as a frequency value in a statement attribute. As an alternative, an

execution probability can be assigned to branches of gateways expressing explicit rules like exclusive

XOR, inclusive OR, and AND. For the case of methodically possible implicit rules, the value of probability

is stored in the transition.

The probability of an end-to-end scenario is the product of all existing probabilities of statement

transitions passed through. For a simple case with a number of 2 XOR-rules with 2 branches each, there

are 4 possible test cases with a overall probability of 0,25 for each test case. The probability of the test

case is calculated and stored in the Probability attribute of model describing the test case.

Effort Weighted Testing

To calculate the time effort of executing a test case, a weight factor can describe the effort to test

statement. The standard value per default is 1. For a complex statements to be executed, the weight can

be increased or decreased for simple statements. The overall effort of a test case is stored in a model

attribute Effort.

Conclusion

Depending on the complexity of the business process to be tested, an approach using C1 for branch

coverage is practically sufficient. Using C2 or C3 delivers too many test cases with high effort to test all

end-to-end variations. For very complex process models it will not be possible to generate and test all

possible test cases. For e.g. 20 XOR gateways we would have a number of more than 1M test cases. For a

standard PC with 100 test cases generated per second it takes 3h to calculate in memory all possible

Page 13: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

13 Whitepaper – Model-Based-Testing for ERP

variations for this process model. For this reason the algorithm uses an Exception

parameter to avoid the time costly generation of too many C2 or C3 test cases.

Methodology for Test Case Generation For the purpose of the automatic generation of test cases based on Blueprint business process models

an integrative methodology is developed. The methodology considers three phases to control the risk of

business impact by insufficiently tested end-to-end processes and to lower the efforts for integration

testing.

1. ERP Solution blueprinting

2. Test case generation

3. End-to-end testing

Figure 5 Solution for model based testing

ERP Solution Blueprinting As already stated, blueprinting an ERP solution using tools for business process analysis (BPA) delivers

many benefits. For e.g. SAP® ERP the project method SAP® Accelerate SAP (ASAP) defines the Blueprint

project phase before the realization phase starts that includes the testing of software. While a BPA tool

captures and documents business requirements for subsequent analyzing and business improvement,

the SAP Solution Manager (SSM) tool is used for the documentation of the Blueprint by means of a

Page 14: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

14 Whitepaper – Model-Based-Testing for ERP

structured IT view containing business scenarios, processes and process steps with

linked SAP transactions. The SSM includes a test management that is integrated with the Blueprint.

Modeling Method

It is a characteristic of the BPM market that BPA tools use many different modeling methods (commonly

nicknamed as a “modeling language”). The latter use a set of specific rules to describe a business process

control flow and its relationships between processes, activities, resources, work products and the

organization. The methods use various notations with symbols to visually express the organizational or

process models by diagrams. In general, methods can be distinguished into legacy or vendor specific

methods, and standard methods like BPMN developed and controlled by the Object Management Group

(OMG).

Today, what is mainly used are variants of flowcharting methods, Event-Process-Chain (EPC) variants and

variations of BPMN-based methods. Even if a vendor claims that the promoted BPA tool uses a standard

method, in most cases the tool implements a specific method version, or a method subset or extension

as compared to the standard method.

For model data exchange, BPA tools use mostly vendor specific XML formats. To interchange model data

between tools of different vendors one can use standard formats like XMI (OMG) or XPDL (Workflow

Management Coalition - WfMC).

However because of variations in three dimensions

1. Implementation of modeling standard

2. Implementation of exchange format standard

3. BPM vendor commercial interests

in many cases unexpected results occur when importing process models from a different BPA tool into

the tool used. The US Department of Defense (DoD) made a review using standards BPMN and XPDL for

model data interchange [MZM09 page 10]. No tool was capable of importing error-free models from

another tool. Sometimes the tools failed to import its own (!) created export format.

Model and Diagram Data Interchange

To overcome the known problem of interoperability between tools and modeling standards, the BPM-

Xchange® (BPM-X) middleware for the interchange of model, master and meta data is available. This

software solution establishes interoperability in the enterprise management systems landscape of an

organization. This approach eliminates existing silos of enterprise meta-data information and avoids

efforts of manually re-keying or re-mapping existing model data or visual diagrams. The automatic

conversion model and diagram improves the quality of exchanged data and reduces needed human

interactivity. The BPM-X middleware is an Enterprise Integration Application (EAI) framework for the

execution of transformation sequences. The framework uses inbound and outbound adapters to read

and write tool or standard-specific data formats. For standard formats like XPDL, BPM-X offers support

Page 15: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

15 Whitepaper – Model-Based-Testing for ERP

for different versions and so called flavors to support vendor specific

implementations of exchange standards. To eliminate vendor specific (standard) methods, BPM-X uses

pattern-based rules for transformations of model and diagram data described by tool specific methods.

The automated process using BPM-X mainly consists of five steps, namely to

1. Export data from the supplier tool, database or repository

2. Import data into a BPM-X repository neutral model representation using an inbound adapter

3. Optionally improve model consistency

4. Execute a sequence of model transformations from supplier into consumer methods

5. Export into consumer tool format using an outbound adapter and import into the consumer tool

In the case of model-based testing, the BPA tool is the supplier tool exporting the business process

models. The consumer tools are testing tools like HP Quality Center (QC), SAP Solution Manager (SSM),

or Microsoft Word in case of manual testing.

Figure 6 Generic model transformation into BPMN using BPM-X middleware

Generation of Test Cases

BPMN Method Internally Used

A process model is managed by the supplier BPA tool described by a specific method. A method consists

of a meta model and a diagram notation for the visual modeling of a diagram. In addition, modeling

Page 16: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

16 Whitepaper – Model-Based-Testing for ERP

rules describe possible patterns that cannot be expressed through the meta model.

Commonly the meta model plus notation is called a modeling language. From the point of view of

generating test cases, this is an external method. The model data including the visual diagrams is loaded

into the BPM-X middleware repository still using the external method.

Figure 7 Test-case model based on BPMN

Internally, BPM-X organizes its repository with model instances allowing several logical representations

of the same model within one physical repository. To have a common initial point for the generation of

test cases, BPM-X initially transforms the models stored in one model instance (1) into a new model

instance (2). For test case generation, the internal method used is BPMN. This operation provides a

normalization of a specific model method into the BPMN method.

During this transformation of the business process, included work product objects are associated with

activities by associations. Resources such as organizational units, groups, roles or IT applications are

converted to so-called “performers”. Process activities, transactions or interfaces are converted to tasks,

sub-processes or call activities.

Test Case Generation Based on the BPMN Method

BPM-X middleware offers a flexible framework to execute transformations in a sequence of so called

atomic operations (AO). With an application programming interface (API), developers can develop own

AO’s. MBT is developed as a BPM-X AO to implement the test case generation.

Page 17: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

17 Whitepaper – Model-Based-Testing for ERP

Based on the model data stored in the instance (2) the software generates test case models and stores

them in a new instance (3). The test case models are also described by the BPMN method. The algorithm

described can be easily modified or replaced to deliver alternative generated test cases. The following

figure shows the first four test cases generated on the sample SAP business process model. Based on this

specific example including 4 XOR gateways with 2 branches each, a total of 24 equal 16 possible test

cases can be generated.

Figure 8 A set of test-case models automatically generated on business process model

Page 18: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

18 Whitepaper – Model-Based-Testing for ERP

Linking with Testing Tools Once the test case models are available within the model instance (3), BPM-X AOs can transform the test

cases into an external format for a testing tool.

Figure 9 Export to Testing Tool

These generated test case models can be converted into the model representation described by a

consumer method and data format. E.g. export is possible into XPDL format for test case processing in

HP QC, into SSM Blueprint projects or into Visio for manual test case documentation.

Table 3 Examples for Testing Tools and external formats

Tool Format Description

HP QC XPDL V2.0 or V2.1 Load test cases into HP QC based on XPDL SAP Solution Manager T-Code SOLAR02

Word-Document Use tab “test-case” with type “manual test case” using Word documents can be used to assign the test case documents to a blueprint structure

SAP Solution Manager T-Code SOLAR01

Blueprint Project Implementation project with Scenarios and uploaded Processes for each test case model and Process-Steps for activities.

Microsoft Office Word, Excel, Visio Documentation, analyzing and manual execution of test cases

Summary This whitepaper addresses the integrated solution of test management using business process models

from the ERP Blueprint as a basis of automatically generated test cases for further use in a testing tool or

manual testing. Using the BPM-X integration middleware for software interoperability builds a very

flexible software solution for the interoperability of any BPA tools to be linked to a testing tool. The

Page 19: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

19 Whitepaper – Model-Based-Testing for ERP

BPM-X framework allows the development specific operations like the discussed test

case generation. This new operation can be linked into the chain of transformations starting from the

import of business process models, through transformations of model data by different methods into

BPMN, to the export of test cases in a specific data format used by the testing tool.

This concept of testing is much more business-focused instead of isolated functional testing not fully

covering business logic and technical interfaces that supports data flow between business activities.

The approach ensures the completeness of test case coverage to improve the quality of ERP supported

business processes and lower the impact of weakly tested end-to-end scenarios.

Costs

High efforts are reduced by automatically generated test cases. By re-using business know-how from

business process models defining probabilities of process flows, the test cases can be prioritized only re-

testing an appropriate number of test cases when software changes and not the full set of test cases for

a business process that is very time consuming in execution of the test cases.

Integration of test case tools like HP QC or SSM Blueprint test management or test projects speeds up

the handling of test cases in complex ERP projects, as compared to manual analysis and description of

test cases especially when the test cases change through change requests.

By examining end-to-end business scenarios with high probabilities, the automation of tests with tools

like HP Loadrunner or SAP eCATT scripts save big amounts of time by re-running theses test during

change requests or ERP maintenance.

The overall efforts are greatly reduced for costly resources of the QA team and business key-users.

Risks

Many business executives have concerns about the quality and resulting costs for implementation and

maintenance of ERP software. By using the model based approach for integration testing, the business

impact of insufficiently tested ERP software is eliminated. By the re-use of business logic given by

process models and execution probabilities combined with systematic execution of prioritized end-to-

end scenarios, the relevant states of a business processes are reviewed. This includes identification and

explicit testing of interfaces for exchange of data between activities supported by different involved ERP

and 3rd party software components.

A more methodical and tool-supported approach by using tools like QC or SSM lowers the risk of

influence of inexperienced QA team members and insufficient project time available for testing.

For revisions of business software, automatically generated test documentation can be used to audit and

review test cases.

Quality

Page 20: Model-based-testing for integration and regression test of ERP solutions

BPM-X GmbH | www.bpm-x.com | Public

20 Whitepaper – Model-Based-Testing for ERP

Using business process models as a basis of their generation, the test cases re-use

documented process knowledge. The use of BI technology to analyze test case data and the visual

documentation of test case diagrams improves the analytical understanding of end-to-end business

scenarios. Based on test cases it is much easier to define appropriate test data for integration and for

functional use case testing.

The integrated approach addresses the impact given by business requirement changes to test cases in

the ERP project cycle or during the maintenance of ERP. The changed test cases can be re-tested again to

eliminate side effects of software changes.

At the bottom line, the described methodology of the model-based generation of test cases

complements the concept of using BPM for ERP Blueprinting. This establishes an optimal alignment

between the business and IT.

Bibliography [OS08] Zur automatischen Ermittlung von Testszenarien aus EPK-Schemata, Oliver Skroch, 2008

[WP.1] http://en.wikipedia.org/wiki/Quartile

[WP.2] http://de.wikipedia.org/wiki/Statement_Coverage

[MZM09] Enterprise Architecture based on Design Primitives and Patterns Guidelines for the Design

of Business Process Models (DoDAF OV-6c) using BPMN, Michael zur Muehlen, 2009

[MAI11] Graph Theory and Algorithms, M. Ashraf Iqbal, 2011