Testing SAP Solutions for Dummies

8/21/2019 Testing SAP Solutions for Dummies

1/76


2/76

These materials are © 2015 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.

http://ibm.biz/IBMDevOpsforSAPprojects


3/76

IBM Limited Edition

Testing SAPSolutions

by Allan Wagner, James Hunter,Nick Portalski, and Bernd Eberhardt



4/76

Testing SAP Solutions For Dummies®, IBM Limited Edition

Published byJohn Wiley & Sons, Inc. 111 River St.Hoboken, NJ 07030-5774www.wiley.com

Copyright © 2015 by John Wiley & Sons, Inc.

No part of this publication may be reproduced, stored in a retrieval system or transmitted in anyform or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise,except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without theprior written permission of the Publisher. Requests to the Publisher for permission should beaddressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions.

Trademarks: Wiley, For Dummies, the Dummies Man logo, The Dummies Way, Dummies.com, MakingEverything Easier, and related trade dress are trademarks or registered trademarks of John Wiley &Sons, Inc. and/or its affiliates in the United States and other countries, and may not be used withoutwritten permission. IBM and the IBM logo are registered trademarks of International BusinessMachines Corporation. SAP is a registered trademark of SAP SE in Germany and in several other

countries. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc.,is not associated with any product or vendor mentioned in this book.

LIMIT OF LIABILITY/DISCLAIMER OF WARRANTY: THE PUBLISHER AND THE AUTHOR MAKE NOREPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE ACCURACY OR COMPLETENESSOF THE CONTENTS OF THIS WORK AND SPECIFICALLY DISCLAIM ALL WARRANTIES, INCLUDINGWITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE. NO WARRANTYMAY BE CREATED OR EXTENDED BY SALES OR PROMOTIONAL MATERIALS. THE ADVICE ANDSTRATEGIES CONTAINED HEREIN MAY NOT BE SUITABLE FOR EVERY SITUATION. THIS WORKIS SOLD WITH THE UNDERSTANDING THAT THE PUBLISHER IS NOT ENGAGED IN RENDERINGLEGAL, ACCOUNTING, OR OTHER PROFESSIONAL SERVICES. IF PROFESSIONAL ASSISTANCE ISREQUIRED, THE SERVICES OF A COMPETENT PROFESSIONAL PERSON SHOULD BE SOUGHT.NEITHER THE PUBLISHER NOR THE AUTHOR SHALL BE LIABLE FOR DAMAGES ARISING HERE-FROM. THE FACT THAT AN ORGANIZATION OR WEBSITE IS REFERRED TO IN THIS WORK AS ACITATION AND/OR A POTENTIAL SOURCE OF FURTHER INFORMATION DOES NOT MEAN THATTHE AUTHOR OR THE PUBLISHER ENDORSES THE INFORMATION THE ORGANIZATION ORWEBSITE MAY PROVIDE OR RECOMMENDATIONS IT MAY MAKE. FURTHER, READERS SHOULDBE AWARE THAT INTERNET WEBSITES LISTED IN THIS WORK MAY HAVE CHANGED OR DISAP-PEARED BETWEEN WHEN THIS WORK WAS WRITTEN AND WHEN IT IS READ.

SAP SE is neither the author nor the publisher of this publication and is not responsible for its con-tent. SAP Group shall not be liable for errors or omissions with respect to the materials. The onlywarranties for SAP Group products and services are those that are set forth in the express warrantystatements accompanying such products or services, if any. Nothing herein should be construed asconstituting an additional warranty.

For general information on our other products and services, or how to create a custom For Dummies

book for your business or organization, please contact our Business Development Department in theU.S. at 877-409-4177, contact [email protected], or visit www.wiley.com/go/custompub. Forinformation about licensing the For Dummies brand for products or services, contactBrandedRights&[email protected].

ISBN: 978-1-119-04720-9 (pbk); ISBN: 978-1-119-04722-3 (ebk)

Manufactured in the United States of America

10 9 8 7 6 5 4 3 2 1

Publisher’s Acknowledgments

Some of the people who helped bring this book to market include the following:

Project Editor: Carrie A. Johnson

Editorial Manager: Rev Mengle

Business Development Representative: Sue Blessing

Production Coordinator: Melissa CossellAuthor’s Acknowledgments

The IBM authors would like to thank their colleagues Kay Johnson, Glyn Rhodes,Stuart Walker, Carsten Siegler, and Lynn Giles, who provided vision, content, review,and assistance to help make this book possible.


http://www.wiley.com/http://www.wiley.com/go/permissionsmailto:[email protected]://www.wiley.com/go/custompubhttp://www.wiley.com/go/custompubhttp://www.wiley.com/go/custompubmailto:BrandedRights%[email protected]:BrandedRights%[email protected]:[email protected]://www.wiley.com/go/custompubhttp://www.wiley.com/go/permissionshttp://www.wiley.com/


5/76

Table of ContentsIntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

About This Book ........................................................................ 1Icons Used in This Book ............................................................ 2

Chapter 1: Setting the Stage . . . . . . . . . . . . . . . . . . . . . . . . 3

Taking Testing Seriously ........................................................... 3Testing Software Using a DevOps Approach .......................... 4

Chapter 2: The Challenges of Testingon SAP Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

Examining Typical Approaches to Testingon SAP Projects ...................................................................... 8

Learning to Live with Multiple User Interfaces .................... 10Navigating a Changing SAP Landscape ................................. 11

Chapter 3: Developing a Testing GamePlan for SAP Projects . . . . . . . . . . . . . . . . . . . . . . . . . . .13

Knowing Which Business Scenarios to Test ........................ 13Preparing Your Test Execution and Reporting Strategy ..... 15Making Sure You Can Test Across All Environments .......... 20

Chapter 4: Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . 23

Knowing Where to Start .......................................................... 23Creating Your Own SAP Testing Center

of Excellence (CoE) .............................................................. 26

Chapter 5: Getting to Know the Componentsof Continuous Testing for SAP Projects . . . . . . . . . . .29

Testing on SAP Projects Earlier and Continuously.............. 29Testing below the Presentation Layer .................................. 30Shifting Left and Prioritization ............................................... 32

Chapter 6: Making Sure Your SAP DeploymentsPer form as Expected . . . . . . . . . . . . . . . . . . . . . . . . . . .43

Keeping Your Focus on Performance Testing ...................... 44Prioritizing What to Test ......................................................... 45



6/76

iv

Considering the Interface – GUI Clientversus Thin Client ................................................................ 46

Batch Loading the Back-End Servers..................................... 50Things to Watch Out for: The Common Pitfalls ................... 50Verifying That the Functionality Is Working As Expected ......52Putting the Pieces Together to Achieve

End-to-End Testing ............................................................... 54

Chapter 7: SAP Solution Managerin Continuous Testing . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Working with SAP Business Blueprint ................................... 56

Gauging Change Impact with BusinessProcess Change Analyzer (BPCA) ...................................... 60

Connecting with the SAP Service Desk Component .................61

Chapter 8: Ten Considerations forTesting on Your SAP Project . . . . . . . . . . . . . . . . . . . .63

Increase Confidence across Stakeholders ............................ 63Reduce Your Cost of Testing .................................................. 64

Benefit from the IBM and SAP Alliance ................................. 64Test as Early as Possible ......................................................... 64Extend the Value of SAP Solution Manager .......................... 65Realize the Value of DevOps ................................................... 65Reduce Business Risk .............................................................. 65Protect Your Investments ....................................................... 66Minimize the Impact of Testing .............................................. 66Control Your Requirements ................................................... 66

Testing SAP Solutions For Dummies, IBM Limited Edition



7/76

IntroductionW

elcome to Testing SAP Solutions For Dummies, IBMLimited Edition. We hope you find this material help-

ful as you begin or continue your work to measure the qualityof your SAP solutions implementation. With much of today’s

business relying on integrating SAP solutions with a mix ofestablished custom applications and commercially availablesolutions, the task of testing these complex applications con-tinues to become more and more challenging. To help organi-zations achieve results they desire, IBM software has the toolsand technology to achieve continuous delivery of high qualitysoftware while reducing the cost and risk of managing changein the SAP landscape. IBM calls this set of tools and technolo-gies DevOps for SAP systems (DevOps is short for develop-

ment and operations).

The main themes of IBM’s approach to DevOps is to acceler-ate software delivery while at the same time balancing speed,cost, quality, and risk and improving the client experience —no small task, as you may expect. Because testing makes up asignificant portion of the delivery life cycle, it can be a majorcontributor to the success of the business. When the testinggoes wrong and announced release dates are missed, you can

damage your brand’s reputation or (even worse) lose custom-ers. By testing efficiently and releasing quality software in atimely manner, you can grow your market share while delight-ing your customers.

About This BookIn this book, we share how IBM software and solutions canhelp an organization manage quality and begin testing SAPsolutions and the integrations they depend on earlier, continu-ously, and with greater test coverage.

SAP is a well-established leader in the packaged applicationsmarket, providing software with a rich set of composable appli-cation modules and configurable functional capabilities, deliv-ered by a comprehensive enterprise application software suite.



8/76

Testing SAP Solutions For Dummies, IBM Limited Edition2

However, organizations typically modify the packaged businessprocesses to meet their specific needs. Testing those changes

becomes even more important because many organizationsadopting SAP software remain heterogeneous enterprises —enterprises where SAP implementations are integrated witha variety of non-SAP enterprise systems, portals, messaginginfrastructures, security, systems of record, systems of engage-ment, and more. These complex implementations must workproperly — end-to-end across a wide variety of technologies.

IBM offers a complete quality management and testing solu-

tion enabling companies to continuously test and measurequality levels throughout the software development life cycleon into production. This should be done without concern forwhether their implementation is a stand-alone SAP deploy-ment or integrated with other critical business systems. Bybringing together functional, performance, and integrationtesting, combined with service virtualization — the ability tosimulate dependent yet unavailable software and systemsfor the purpose of testing — organizations are able to deliver

higher quality products and services to market rapidly withgreater efficiency, higher predictability, and reduced cost.

Icons Used in This BookThis book uses the following icons to call your attention toinformation you may find helpful in particular ways.

The information in paragraphs marked by the Remember iconis especially important, so we’d ask you to take special noteof it.

The Tip icon indicates extra-helpful information. You maydiscover how to implement an especially useful testing imple-mentation or find out crafty ways to help you save time ormoney.

This icon marks places where technical matters are dis-cussed. Sorry, it can’t be helped; plus, the information isintended to be helpful.

Paragraphs marked with the Warning icon call attention tocommon pitfalls that you may encounter.



9/76

Chapter 1

Setting the StageIn This Chapter

▶Meeting testing timelines

▶ Testing software the DevOps way

▶ Going for lean and mean

Y ou’ll often hear that testing takes as long as it takes. Todig down and determine what this statement actually

means — other than an attempt to avoid committing to atimeline — you’d have to closely examine the many factorscontributing to how much time “as long as it takes” actuallytakes. Things to look at include

✓ The complexity of the technologies being tested

✓ The availability of complete and realistic test environments

✓ The particular approach to testing: manual, automated,

or a combination of the two ✓ The number of resources you can devote to testing

✓ The testing experience of the quality professionals

✓ Knowledge of the application being tested

Taking Testing SeriouslyThis idea of “testing taking as long as it takes” could certainlyapply to organizations developing and testing SAP solutions.These projects tend to be complex and also tend to take a longtime. One could also imagine accepting a testing taking as longas it takes approach if the delivery team was in fact achievingmaximum efficiency without any distractions or interruptions.However, “maximum efficiency” isn’t the case for many teams.



10/76


Now, if your crack team is able to deliver higher quality soft-ware and meet all planned release dates with room to spare,

you’ve got little to no risk for the business, and your stakehold-ers are happy. When the day comes, however, that deliveryexpectations are missed or the release date slips, those verysame stakeholders may feel that your all-star delivery team hasended up with egg on its face.

To meet customer expectations and deliver quality software atthe speed customers or end-users demand, teams need to shiftto the more responsible side of the tactics spectrum — they

need to become more efficient in their testing practices,they need to involve testers much earlier in the process, andthey need to begin testing earlier and continuously so theycan release software of higher quality much more frequently.

Testing Software Using

a DevOps ApproachTo reduce costs and win in the market, companies are now look-ing to extend agile and lean principles — increasing efficiency,reducing waste, and embracing best practices to shorten devel-opment cycles — in a DevOps approach to software develop-ment. This desire and need to improve their capability alignsperfectly with IBM’s definition of a DevOps approach to softwaredevelopment — an enterprise capability for continuous delivery

of quality software that enables organizations to seize marketopportunities and reduce time to customer feedback by releas-ing new functionality in smaller increments more frequently.

By applying lean and agile principles across the softwaredelivery life cycle, IBM DevOps helps organizations deliver adifferentiated and engaging customer experience, helps themachieve quicker time to value, and works to increase theircapacity to innovate.

For the delivery team to meet the needs of the customer andthe employers at the speed demanded, the team needs tobecome more efficient and look for ways to reduce bottlenecks.Working in silos — with analysts, developers, testers, andoperations all working in isolation with limited collaboration —creates inefficient handoffs between the individual groups.This approach doesn’t work when time is of the essence. All



11/76

Chapter 1: Setting the Stage 5

roles should focus on working together as one delivery teamwhere all team members are contributing to the quality of soft-

ware being developed.

It is really all about time and taking less time to get to the endof your project while maintaining or increasing the level ofquality in the software being delivered. So, to deliver softwarethat is better, deliver it faster, and deliver it at a lower cost,the delivery team needs to look for ways to begin testing ear-lier and continuously so it can shift release dates to the left onyour time scale, rather than settle for always shifting release

dates to the right.

Shifting left means breaking down the silos and eliminatingthe idea of handoffs by building quality into the delivery pro-cess earlier — during blueprinting and design, code reviews,requirement approval, and so on. But again, building qualityin doesn’t mean that testers are the only ones responsible forthe quality of the software to be delivered.

In a DevOps approach to software development, everyonecontributes to quality, and everyone should be contributingto the shared goal of delivering a positive user experience.Delivery teams need to expand the effort to do more unit test-ing, increase the level of integration testing done to checkservices, and decrease the level of effort focused on testingthrough the user interface (UI). See Figure 1-1.

Figure 1-1: The layers of testing.

Unit testing refers to testing carried out in the developmentenvironment by the person who wrote the code or carriedout the configuration. Integration testing (also known as APItesting) requires the delivery team to test that all integration



12/76


points between the many components making up the com-plete application are interacting with each other properly.

Neither unit testing nor integration testing require a stabilizeduser interface — that means you can start with these types oftesting very early on in the development process. User interface (UI) testing is performed by testers interacting with theapplication’s user interface to measure whether the applica-tion functionality is working properly. This type of testing caneither be done manually or by using test automation tools oncethe user interface is available from the development team.

Having developers do more unit testing — or get that firstround of unit testing up and running if your organization hasn’tyet incorporated that aspect into the delivery life cycle —makes sense since it’s the developers who write the code.

Next would be increased testing at the service layer, con-tinuously validating the integration between components toensure the messages being exchanged are processed appro-

priately and the components are performing as expected.In situations where all the components aren’t available fortesting, service virtualization can be used to simulate what’smissing.

Leveraging automation techniques to expand the coverageof testing at the unit and service levels frees up testers todo other value-added activities such as exploratory testing,asking better questions, and improving test design — activities

that would definitely improve the overall level of test coveragefor many organizations.

By being able to test earlier and test continuously across theentire delivery life cycle, teams are able to improve efficien-cies to such an extent that they’ll no longer be tempted to fallback on the tried and true ways of just throwing more money(or more people) at the problem. We’re here to tell you thatthere is a better way to test and measure the quality of your

SAP solutions and to do it at a lower cost. Read on to find outmore about what’s available to help organizations managequality and automate testing of their SAP solutions while help-ing testers test more efficiently and effectively.



13/76

Chapter 2

The Challenges of Testingon SAP Projects

In This Chapter

▶ Examining typical approaches of testing SAP projects

▶Working with multiple user interfaces

▶ Looking to the future

T eams embarking on developing and deploying high-qualitySAP solution implementations are sure to encounter several

distinct challenges — challenges they’ll need to be overcome ifthey ever hope to succeed. The delivery team needs to be able to

✓ Consistently manage change when faced with testingboth SAP and non-SAP solutions

✓ Deliver a high level of quality on time and within budget

✓ Validate that the business processes have been checkedand are working as expected

It goes without saying that all this is supposed to be done withthe goal of minimizing errors, reducing costs, and aligning thebusiness goals with the development effort. So, in order to trulytest SAP applications effectively, you really need to align theproject blueprint — the business process hierarchy, or BPH, to

use more technical language — with the quality managementeffort. It is the BPH that defines the scope of the developmenteffort by documenting the processes to be implemented anddrives the test design for testing changes.

When properly aligned with the BPH, any SAP project test planis going to contain multiple test cases, each of which is goingto contain multiple test scripts (wheels within wheels within



14/76

8 Testing SAP Solutions For Dummies, IBM Limited Edition

wheels, as it were). Within the test script are the specific,detailed steps a test is going to execute, either manually or

through some form of automation. The test execution resultsare then linked to the test case, along with any associateddefects created as a result of a test failing. Finally, and toensure that a high level of quality is maintained, teams needto validate that the changes haven’t impacted the existingfunctionality and performance of the deployment.

Sounds like a tough row to hoe, and it can be, but approachesexist that can ease the burden to a certain extent. The rest of

this chapter takes a look at a few.

Examining Typical Approachesto Testing on SAP Projects

While there are many approaches to testing SAP implementa-

tions, the driving factor that needs to be accepted across theentire team is a constant focus on delivering quality whilebeing aware of potential roadblocks along the way. The follow-ing list, while far from exhaustive, can give you a sense of thechallenges your team will be facing in this regard:

✓ The uncertainties of testing your SAP implementationwhen it has been integrated with legacy applications anddeployed in environments made up of many differing

technologies and commercially available solutions ✓ Setbacks that may occur if some members of your team

(or maybe all members) are hampered by a lack of spe-cific testing skills

✓ The (potential) impact to a delivery schedule when per-forming a high volume of manual tests

✓ The (again, potential) creation of bottlenecks when testenvironments and test data aren’t available when teams

need them

But it’s not all doom and gloom; help is available — help toassist teams in managing quality, automating their tests toreduce test execution time, and removing testing bottleneckslike waiting for test labs to become available so testing canbegin. Read on to find out the details.



15/76

Chapter 2: The Challenges of Testing on SAP Projects 9

Manually testingIn many organizations, details of the test case and the detailedtest scripts are documented and maintained in a collection ofdocuments and spreadsheets and managed by the SAP projecttest manager or test coordinator. These test cases (detaileddescriptions of how the software will be tested) and test scripts (the step-by-step instructions spelling out how a test will becarried out) are then assigned to the tester. The tester in turnperforms the steps as documented in the test script and manu-ally records which test steps were completed, which verificationpoints passed — and which failed. Other information capturedduring the manual test execution process may include whethera defect discovered is blocking the tester’s ability to completethe testing or if any other observations were made. This is alsoa great time to capture any discrepancies or possible errors inthe test script itself. The test results are then returned to the SAPproject test manager or test coordinator.

An SAP project test manager is a member of the testing teamwho has good knowledge of quality management as well asof testing methodologies, the tools available to run testingprocedures, and of defect management — the process thatfocuses on preventing defects, catching defects as early in theprocess as possible, and minimizing the impact of defects. Inaddition, a competent SAP project test manager also displays adeep understanding of all aspects of SAP applications — fromthe various SAP components and implementation methods, tothe integration between the various application components,to an appreciation of the role SAP applications play with thebusiness domain itself.

Automating your testingAutomated testing has often been overhyped as some kindof a silver bullet that would solve any organization’s testingproblems. We’re here to state two important truths:

✓ Not all tests can be automated.

✓ Not all tests should be automated.



16/76


However, we’re saying that automated testing should play asignificant part in your testing strategy. Applying an 80/20

approach to automation is a good rule of thumb. ( Note: 80percent means 80 percent automation. ) Automated testing issure to speed up your test execution. Together, automatedtesting and service virtualization can produce tremendoustime savings and allow teams to stand up realistic test labsquickly and begin testing the application end-to-end earlier.

Learning to Live with MultipleUser Interfaces In today’s world, the same business process or applicationfunctionality may be available to the end-user through anumber of different user interfaces (UI) deployed on manydifferent platforms. Perhaps an end-user uses a rich clientinterface while at the office, accesses the same application

via a thin client (browser) while away from her desk, orneeds to do some last-minute work from a mobile device.The multiple channels still need to be tested individually,and a single test script doesn’t lend itself to test everything.

To get a sense of the lay of the land, check out the fol-lowing UIs, which are quite common in SAP solutionimplementations:

✓ SAP Graphical User Interface (GUI ): SAP GUI is the graph-ical UI installed on the end-user’s desktop to access theback-end system of an SAP R/3 deployment — the applica-tion server and database, in other words.

✓ SAP UI5: UI5 is an extensible library provided by the SAPdevelopment team that removed the need for organiza-tions to roll their own custom frameworks when buildingmobile and web interfaces for their SAP implementations.

✓ Custom UIs: Many SAP implementations have dependen-cies on other systems, such as databases or services,resulting in the integration of SAP components with non-SAP components. Such custom applications will inevita-bly need access to SAP systems to retrieve data, makingthis another channel where the interface presented tothe user requires testing.



17/76

Chapter 2: The Challenges of Testing on SAP Projects 11

In cases where the same business functionality is presented ina variety of UIs or channels, teams should consider modular-

izing the same test steps into a single test script and maximizereuse. You may know this as keyword scripting.

Navigating a Changing SAP Landscape

When working to manage quality and test your SAP solutiondeployments, it would serve you well to be aware of howSAP landscapes and deployment models may change overtime. Change is inevitable — being prepared for that change,however, is never a sure thing. You need to be in a positionto make the right decisions early on when radical changesthreaten to disrupt your business model. Staying aware towhat the future may bring will go a long way to determiningwhether such changes require a complete overhaul of your

test assets or whether the changes to the test assets areminimized.

Dealing with business changes Large organizations around the world are pursuing enterprisetransformations designed to streamline their business as wellas their IT landscapes by leveraging prepackaged enterpriseapplication suites. These transformations often replace asignificant portion of the existing IT landscape and wouldn’tbe possible to accomplish without a prepackaged solution.Understanding which legacy components will remain andwhich ones will be replaced plays a crucial role in effectivechange and quality management.

Without a solid quality management foundation offering trace-ability across requirements management, test management,and change management throughout the entire developmentlife cycle, many organizations would struggle in understand-ing what tests need to be updated, which test scripts needto be created, and which tests need to be run to protect theinterests of the business.



18/76


Dealing with technology changes As you might expect, we are of the opinion that the effortto test SAP solution deployments is both huge and oftenunderestimated.

SAP project rollouts are typically large discrete projects withmany dependencies on other subsystems deployed in hetero-geneous environments. The ability of the composite application(SAP applications plus the complex collection of dependentsystems) to maintain a system of record while accessing mul-

tiple data sources is critical. (When that ability is lacking, youcan never really ensure that your business will continue tooperate without issue.) Organizations need to ensure that theexchange of data between these disparate systems is workingas expected by executing integration tests.

In addition to testing the integration points between your SAPcomponents and non-SAP components, you may choose torun a full set of regression tests on the system as a whole. In

this context, regression tests are tests designed to ensure thatthe applications that worked before continue to work properlyafter implementation.

As always, forewarned is forearmed: Knowing the impactof change to workflows, reports, interfaces, conversions,enhancements, and forms as a result of the upgrade can go along way toward helping test teams understand what new testassets, test cases, and test scripts need to be created.

Dealing with legislative changes At times, changes required by new legislation may force orga-nizations to make modifications to their SAP landscape — andthose changes must of course be tested. Existing SAP implemen-tations must be tested for functionality, performance, security,user experience, and so on. More importantly, any change man-dated by legislative action must work, or the business could be

at risk of some type of legal action. IBM’s quality managementsolution can help the SAP project test manager manage andprioritize the test effort by delivering the information they needto understand which tests are needed to check which businessprocess or process steps. Without this insight, teams could endup in a situation where they’re running the wrong tests or creat-ing/running unnecessary tests. Such missteps could potentiallydelay a release, which could potentially result in hefty fines.



19/76

Chapter 3

Developing a Testing GamePlan for SAP Projects

In This Chapter

▶ Choosing the proper business scenarios

▶ Coming up with a test execution and reporting strategy

▶ Knowing what to do when things go wrong or plans change

▶ Ensuring the proper team spirit

▶Making sure your testing covers all the (environmental) bases

L ike many things in life, such as renewing your driver’slicense or seeing your doctor for that annual health

checkup, testing SAP solution deployments, upgrades, or newbusiness functionality tends to take longer than you mightexpect. Given that fact, the more effort you put into plan-

ning your deployments — including foreseeing all the stepsthat may slow you down — the smoother the process will be.Defining your strategy early on and determining your reportingneeds will help keep the team on task and deliver the neces-sary feedback to your stakeholders.

Knowing Which Business

Scenarios to Test By using the Business Blueprint, a feature of SAP SolutionManager that you can use to document business scenariossupported by an SAP system, you can come up with a commonstrategy of how your business processes are to be mappedinto one or more SAP applications. Your Business Blueprint(if you have one) details the scope of business scenarios,



20/76


business processes, process steps, and the requirements of anSAP project implementation.

In an SAP project, a Business Blueprint brings together the fol-lowing structural elements into an ordered hierarchy:

✓ Organizational units

✓ Master data

✓ Business scenarios

✓ Business processes

✓ Process steps

You can assign content — project documentation, for example,or business configuration sets (BC Sets) or transactions — toindividual structure elements with the help of SAP SolutionManager. For more on working with SAP Solution Manger, seeChapter 7.

If you’re using SAP Solution Manager, then specifying a BusinessBlueprint is a necessary first step for testing. It allows you tocontrol two crucial aspects of your strategy:

✓ Configuration: When you configure your business pro-cesses with reference to the Business Blueprint projectstructure, you are essentially drawing a detailed map ofhow the business expects to operate. With that informa-tion in hand, you’re in a better position to determine

what should be tested and where. ✓ Test organization: Your test plans can be generated

based on the Business Blueprint project structure.The transactions that you assign in Business Blueprintprocess steps are then put into test plans when yougenerate them, and you can run these transactions astransaction function tests.

If you use IBM Rational Quality Manager (RQM), you can importthe Business Blueprint to an RQM project area and automaticallycreate requirements, test plan, and test cases. The auto-generatedtest assets are linked back to the original Business Blueprinthierarchy, providing full traceability — achieved by creating linksand relationships between project artifacts — across businessrequirements, test cases, test results, and all the way down toany associated defects.



21/76

Chapter 3: Developing a Testing Game Plan for SAP Projects 15

Preparing Your Test Execution and Reporting Strategy

If showing up is half the battle, then showing up with a properplan of execution means you’re three-fourths of the way there.The next sections spell out how you can prepare for testingsuccess.

Test executionThe test execution effort is highly dependent on the numberof test cases that have to be executed in a complete systemtest. The recommended approach is to create one test casefor the entire business process. The test case is assigned atthe business process level of the Business Blueprint.

The advantage of this approach is that you end up with a smaller

number of test cases, since test cases are now created at thelevel of the business process, or its variants or flavors, drasti-cally reducing the test case development and maintenance effort.

An alternative approach is to create a test case for each pro-cess step of an end-to-end business. With this approach, eachtest case is assigned at the relevant step of the business pro-cess in the Business Blueprint.

The advantage of this second approach is that test results andassociated defects can be managed at a lower level with commu-nication, status reporting, and root cause analysis being muchmore transparent. However, this approach might result in anextremely large number of test cases, where test case creationand maintenance effort could potentially outweigh any benefits.

ReportingWith test status reporting in IBM Rational Quality Manager,you get a summary of the test results for the reporting period,including analysis of result information. The summary describeshow well the test execution is progressing.

IBM Rational Quality Manager comes with a dashboard thatyou can use to track your test plan and its execution progressin real-time throughout the testing cycles. These dashboards



22/76


are highly customizable and can be easily adapted in theirappearance. Predefined and customized queries help you

filter the right data from the project so you can quickly collectthe metrics required for a test status report.

The test status report identifies those activities completed inthe current reporting period and those planned for the next.Some of the measurements produced by this report includethe following:

✓ Number of defects found

✓ Severity levels of each defect

✓ Status of defects (outstanding, investigated, explained,fixed, retested, accepted, and so on)

✓ Rate of defect arrival (in other words, defects found pertest executed and/or per tested product component,organized by test period and level)

The report also includes information on planned tasks, plannedschedules, the actual status of both tasks and schedules, theforecasted outlook, issues, recommendations, and an actionplan. With this information in hand, you are in a position to

✓ Review actual status against plan

✓ Identify and address constraints, concerns, and issues

✓ Identify items requiring containment and/or escalation toresolve issues

Appropriate consolidation and filtering of data is necessaryto ensure that key information is both evaluated and assimi-lated at the respective planning meetings. You’d then be ableto use any statistically valid conclusions to predict the qual-ity level achieved by the tested application and comparethat level to the target level documented in the test plan.

Dealing with defects and change requests

In all the tests we’ve ever run, we’ve always ended up withtwo main results:



23/76


✓ Test logs: These contain the test results with all yourpass/fail information

✓ Defects: These contain the information and details abouta problem with the application that has been foundduring testing — typically a deviation from the require-ment’s expected result or outcome of a test.

You need to record defects in a defect management system.A proper defect management tool, such as IBM Rational TeamConcert, provides a workflow engine that can help projectteams track defects and manage requests. Inefficiency in man-aging defects and change requests means you’ll most likelyend up spending a lot more time than anticipated in order toresolve problems. Team members are unable to work togethereffectively, which often results in development and testingefforts being duplicated. Lacking an effective tracking tool canalso result in poor quality as important changes are lost ormissed because of a lack of prioritization.

Many organizations that run on SAP applications requireheterogeneous environments — environments that make useof both SAP components and non-SAP components — inorder to deliver the end-to-end business process. Often achange request to a business process will require differentteams to do work to deliver that change. As a result, youoften need to parcel out the responsibilities for managing andtesting a change to different parties.

This parceling out often leads to unclear ownership and a lackof visibility in the overall process. Adding to the complexity,managing and testing these application changes is frequentlyperformed using different change control tools, resulting inquality management that’s often unstructured and dispersed,with insufficient sign-off procedures. Additionally, due to alack of sufficient automated testing, whether that be func-tional, integration, or performance testing, the testers areunable to keep up.

To make matters worse, reporting across all the various con-current activities can be really difficult. All this is meant tosay is that change is hard, but we want to assure you that itshouldn’t represent an unbearable burden. Delivery teamswith both SAP components and non-SAP components whoadopt common processes and tools for quality and changemanagement reap the benefits of



24/76


✓ Shared requirements definition

✓ Familiar workflows

✓ Consistent defect management

✓ One source of the truth reporting via real-timedashboards

The next few sections spell out in a bit more detail how such acollaborative process would work.

Collaborating with test partners and teams

It is common on SAP projects to work with multiple teamsand partners. Teams from different organizations need to beable to work together effectively, following standardized testprocesses and leveraging a common technology platform.Geographically dispersed test resources need to be able to

communicate and collaborate effectively, wherever they are inthe world and whatever their working hours. Additionally, thequality/test manager needs to be able to monitor the progressof any outsourced test work in real time.

For complex SAP testing projects to work successfully, the keyis collaboration. Testing teams must be able to communicateand share test data, tools, scripts, and even skills both inter-nally and with each other — in an appropriate and governable

manner. To achieve this, you must have a quality managementplatform that is open (allows other tools to plug into the plat-form) and can ensure consistency and traceability. This willenable all parties to have confidence in how data will be sharedand metric reporting can be assured.

A secure, strong, reliable, and open technology backbone isusually the answer. An open quality management platformmakes it simple for each member of the team to find the

information that is most relevant to them — for example, thebusiness owner might want instant access to overall projectstatus reports, while a tester would prefer specific informa-tion on the tasks assigned to that individual and their immedi-ate team.

Collaborations can be difficult to manage. To keep your head-aches at a minimum, keep the following in mind:



25/76


✓ Make onboarding easy: Bringing new team membersonboard when new or enhanced skills are required needs

to be fast and easy. This will give organizations the flex-ibility they need to leverage business and technical talentwherever it is located.

✓ Ensure governance: Focus on delivering real-time insightinto testing on SAP solution programs, projects, andresource utilization to help teams make better informeddecisions and continuously measure progress againstdesired business outcomes.

✓ Be realistic about standardization: When working withoutsourced partners, it can sometimes be challengingor expensive to insist they use a particular tool. Theanswer is to provide a quality platform that is open forthem to plug in or connect to. IBM’s solution is built onopen Web and OSGi standards and provides an open andextensible architecture designed to provide the flexibilityto assemble a quality platform that can be tailored to theunique needs of any test process leveraged by required

outsourced providers, vendors, and partners.

Testing all business scenarios across both SAP systems andnon-SAP systems As non-SAP products, applications, and vendors are added to theIT landscape, companies must undertake extensive integrationwork and testing to ensure that all components function cor-rectly. As the IT landscape grows in complexity, managingsuch cross-system integrations can be a difficult task.

IBM fosters collaborationA successful collaborative qualitymanagement solution needs to bebuilt with open, flexible services andon an Internet-based architecture.IBM achieves this by delivering acollaborative life cycle management

solution, including quality manage-ment, that’s built on an open plat-form. This enables diverse tools tobe used together, providing userswith an integrated experience.



26/76


Where dependencies exist between SAP projects and non-SAPprojects, SAP delivery teams must keep in sync with other

teams’ release cycles, enabling them to manage projects holis-tically. Without close collaboration and a shared toolset forchange control, ad hoc IT changes or application releases cancause business disruption.

The answer, of course, is to manage and test SAP projects andnon-SAP projects in a unified way. This means you need a com-prehensive and automated approach for requirements, quality,and change management. IBM Rational Quality Manager inte-

grates seamlessly with SAP Solution Manager, allowing testersand developers to exchange SAP Business Blueprints, testresults, and incident information quickly and easily. The busi-ness scenarios from the Business Blueprint may depend onSAP technology and non-SAP technology, but the implementa-tion of the test plan, test case, and test scripts — as well asthe reporting and analysis of the test progress and results — isnot constrained by any particular technology or vendor.

There’s just one hitch: In order to achieve this seamless test-ing of business process across SAP systems and non-SAPsystems, you need a functioning system to test — not alwaysa given. We suggest you consider the option of deploying vir-tualized services, software, and applications to fill any gapsin the end-to-end scenario during testing. The solution fromIBM means you can accelerate the delivery of any complextest environments and enable integration testing to beginearlier in the development cycle, to be performed more fre-

quently, and to be completed faster.

Making Sure You Can Test Across All Environments

The test environment is the environment used for testing — no

surprise there, but keep in mind that it should be set up toresemble the production environment as closely as possible.In addition to the system under test, it also contains the soft-ware tools for test execution (manual and automated), simula-tors, and other supporting software and hardware to conducta test. The test environment may also include additional items,such as the facilities, processes, and personnel required tocarry out the test.



27/76


The setup of a test environment is always an area of potentialhigh risk, particularly from a scheduling point of view. If the

plan for setup does not cover all aspects of selection, acquisi-tion, and training for new components, or is missing compo-nents, there may be significant delays in beginning the actualtests. If the setup is not comprehensive and misses key con-figuration items, there may be significant schedule setbacks.If risks in the setup are not identified and managed, they mayderail the entire project.

Documenting the setup in the form of a test environment

specification is crucial. Such a specification helps ensure that

✓ The results of testing activities are accurate and valid.

✓ Testing activities have a higher likelihood of being sys-tematically reproduced.

A test environment specification documents the processesand tools required to set up, operate, and maintain theinfrastructure, particularly any processes and tools that areunique to the test environment. It often includes documen-tation of any differences from the production environment,likely impacts on testing, and the risk management plan forsuch impacts.

Items covered in the test environment specification include

✓ Project/Application identification

✓ Contacts ✓ Schedule

✓ Availability of the test environment

✓ Test platform requirements

For each node (mainframe, server workstation, and so on),you’ll want to specify test platform requirements for thefollowing:

✓ Hardware

✓ Software

✓ Voice/data network

✓ Security



28/76


✓ Tools and Utilities

✓ Applications

✓ Internal/external application interfaces

✓ Data

✓ Facilities

✓ Personnel staffing to establish and maintain the testenvironment

Important considerations for any test environment are

✓ How many environments will be required, and how willthe code, test data, and infrastructure components bemanaged, transferred between them, and maintained?

✓ Will there be separate environments for each level oftesting? If not, how will the testing be isolated to keepone level from interfering with another?

✓ Configuration management is an essential process. Thetest environment’s documentation requires automatedassistance.

✓ Test environment and configuration automation will alsohelp in managing backup and restore functions for testing.A key element often forgotten is the capture of the environ-ment at pre- and post-test states. This capture is essentialfor efficient test results re-creation, variance investigation,and auditable testing.

✓ At the conclusion of testing, enough environment datashould be captured to enable reconstruction of the envi-ronment to repeat any tests. This requirement may varydirectly in proportion to the risk involved and any legalor statutory considerations.

✓ After the test environment is established, the environmentbuild team should conduct a set of verification tests toensure that the environment is operational and ready for

the test team to conduct their tests. The use of a setupchecklist is helpful.

✓ At the conclusion of testing, once again the use of achecklist is helpful to ensure that all test environmentteardown activities have been completed.



29/76

Chapter 4

Getting StartedIn This Chapter

▶ Setting up the proper environment

▶Working out an effective test management strategy

▶ Ensuring your test execution is flawless

▶ Aiming for excellence

S

AP solution implementations are very seldom done in iso-

lation from other systems the organization may alreadyhave in place to drive their business. When you consider allof the technologies involved as well as the diversity of theprotocols and data shared across such a heterogeneous envi-ronment, it’s clear that a truly effective testing effort will befar from trivial. And with such a large effort comes confu-sion on where to start. We’re here to tell you where your teamshould focus their attention early on so they can be sure toreap the benefits of effective testing in the later stages.

Knowing Where to Start First things first: The trick to ensuring success in any SAPproject is to make doubly sure that your testing techniquesare able to assess the risk of implementing change.

In the context of an SAP project, this means being able to testall possible business scenarios, as spelled out in Chapter 3of this book. We know a common response to such a pieceof advice is to exclaim “Easier said than done.” An equallycommon response is to ask “Where should I start?”



30/76


Here’s the beginning of an answer: To work out a successfulend-to-end testing solution for SAP projects, you need to focus

on three parts of the implementation process — your testenvironment, your test management, and your test execution.

Providing a test environment The test environment needs to be realistic and as com-plete — production like — as possible. In order to test all busi-ness scenarios, the complete test environment must include

all required SAP technology and non-SAP technology. Makingall the required hetergenous services and/or applicationsavailable for testing requires the virtualization of any missingparts of the test environment. By ensuring the team can vir-tualize all or any of the test environment, you ensure that thetesting team(s) can have access to the exact test environmentthey need when they need it. This service virtualization ofyour SAP landscape is your necessary first step in implement-ing a successful end-to-end testing solution.

Preparing your test management Quality management is much more than simply executingtests. The requirements, planning, execution, reporting, andanalysis needs to be supported by a collaborative hub forall appropriate stakeholders, whether they be the businessowners, technical teams, outsourced partners, and/or vendor

support. The idea to is have a central space that can be usedby all to plan, to share information, and to review the latestquality status of a project.

Test management includes the identification and support fora testing process. Whenever possible, it is usually most effec-tive for test teams to follow an iterative testing process, wheretesting early and often is the key. Doing so integrates qualitymanagement and continuous testing across all stages of the

project work flow, as opposed to relegating test activity to theend of the project.

Changes to an SAP landscape may come from implement-ing a new SAP application, an upgrade, an enhancementpack, a new or revised business process, or something elseentirely. Part of preparing test management is defining how to



31/76

Chapter 4: Getting Started 25

determine when the level of risk associated with that changeis sufficiently reduced (and maybe not completely eliminated)

to align with the time-to-market objectives of the business.Sometimes, changes to business processes require moreretesting than other change factors, so the test managementhub also needs to be able to help the test team minimize thecost of rerunning tests. It should do this by identifying theminimum amount of re-testing that needs to be done followinga change to one or more business processes. This is known asbusiness process change analysis.

On an SAP project, an important consideration for test manage-ment involves, first, whether or not the team will utilize SAPSolution Manager, and second, if such a utilization is desired,how precisely that will take place. The test management hubneeds to be able to connect and link to requirements anddependencies from the entire business, technical, and enter-prise landscapes. From a business process perspective, thismeans understanding how SAP Solution Manager is being usedto manage business processes and how the team intends to

interact with vendor support.

Finally, an important activity in preparing for test managementis thinking through how to utilize any existing investment intesting skills, tools, and data that may already be in use. Someorganizations opt for a “Big Bang” approach to implementingcontinuous testing, meaning they introduce a set of standard-ized test processes and tools to as many people and projectsas possible at the same time. This is often to try and reduce the

training costs and to keep disruptions confined. However, orga-nizations are typically dealing with projects that are already inthe process of testing and a Big Bang approach would disruptand add risk to those projects. To address this, the organiza-tion must select a quality management tool that is built on anopen platform that can allow different test execution tools toplug in for common and standardized test planning, manage-ment, reporting, and — most importantly — collaboration.

Executing your tests Having established a test management protocol, the test teamneeds to prepare to start test execution. As part of that prepa-ration, your test team must decide what type of test executionis required. Your choices are as follows:



32/76


✓ Manual testing as the lowest level approach to testingwhere each step of the test script is run by a human

instead of a test automation tool. ✓ Integration testing to ensure all required dependencies

work as expected.

✓ Performance testing to validate the performance andscalability of the business process and/or businessscenario.

✓ Business process validation to ensure the business willrun as expected even after complex changes.

The testing capabilities from IBM support testing across abroad set of middleware, messaging technologies, applica-tions, and industry-specific requirements. This is the onlyway business objectives and test execution can be alignedand an assessment of risk made visible to all necessarystakeholders.

So, that’s the 1-2-3 punch to prepping your successful end-to-end testing solution for SAP projects. In the next section,we want to offer some practical advice about the productsand capabilities that can help you turn your plans for anend-to-end testing solution for SAP projects into a successfulreality.

Creating Your Own SAP TestingCenter of Excellence (CoE)

Having all things test-related under one roof (whether thatroof is literal or virtual) makes sense because it is easierto ensure quality when activities are concentrated in oneplace. SAP, for example, has its own Testing Center ofExcellence (CoE) in Walldorf, Germany, where it uses testingtools from IBM Rational as part of its testing strategy. Many

organizations want to set up and run their own Testing CoEfor their own SAP projects, so here are the key things toconsider to ensure sustainability of your own Testing CoEfrom day one.



33/76

Chapter 4: Getting Started 27

Preparing a flexible test

environment with flexible access to testing tools

One of the most frequent comments we hear back from orga-nizations that set up and run their own CoE for their SAPprojects is that the demand for their services can suddenlyincrease and they then struggle to respond. Not only can itprove difficult just to provide proper access to testing environ-

ments, but it can also be a hassle to provide enough licensesfor test tools. Given these facts, it is important to prepare foraccess and licensing issues from day one.

From an infrastructure point of view, the key to sustainability isto ensure that your teams can virtualize any missing part of thetesting environment at any time. Every team must be able toget access the test environment they need when they need it.

From a test tool licensing perspective, the key to sustainabilityis enabling practitioners to use the right tool at the right time, aswell as factoring the need for additional tools that are potentiallyrequired later in a project life cycle. IBM addresses this chal-lenge by providing license tokens — a flexible method of licens-ing the end-to-end test tools needed by a testing CoE. Tokenshave a predefined value that replaces the traditional view of astatic license quantity. The CoE then maintains a pool of tokensavailable to users so they can deploy the right IBM product at

the right time throughout the life cycle of the project. Often test-ing CoEs charge other departments for use of their services orofferings and so manage the token pool by role or department orproject.

Preparing for different approaches to testing

If there is not a corporate approach for testing, then teamsmay request support for different testing processes, such asagile or waterfall. If there is a corporate standard, then thetesting environment must support governance and templates,



34/76


but if it turns out that project and test managers have thepower to choose their own approach, then the testing envi-

ronment must be able to adapt for that particular project.

A testing CoE supporting SAP projects, needs to be ready tosupport SAP Solution Manager. The specifics about how aproject SAP Solution Manager can vary significantly. Indeed,the way a project wants to utilize the information stored inSAP Solution Manager for the express purposes of testingmay vary. IBM and SAP have worked together to addressesthis challenge and have come up with a simple, flexible

configuration interface that allows teams to choose the rela-tionship (and automation) between information capturedin SAP Solution Manager and how it is used in the rest ofthe testing process. This flexibility allows your testing CoEto support SAP projects that want to utilize SAP SolutionManager.

Playing well with others It is common for SAP projects to outsource part of a project toone or more partners. This could mean that different testingactivities are parceled out to various partners, using differenttools.

In such a world, flexibility is key. To be successful, you mustensure that the testing CoE is open enough to allow otherorganizational (or individual) teams to plug in their own tools

or testing environments as appropriate for their work. Thisway, the testing CoE maintains governance and control whilemaximizing flexibility.



35/76

Chapter 5

Getting to Know theComponents of Continuous

Testing for SAP ProjectsIn This Chapter

▶ Going for an “all testing, all the time” model

▶ Checking under the UI hood

▶ Getting your priorities straight▶ Testing your various components

E very test you execute probably focuses on shifting therisk of software delivery further to the left on your project

timeline — that “shifting left” we talk about in Chapter 1 ofthis book. The aim for all testing is to test as thoroughly asneeded and as early as possible — well before you deploy intoproduction. In this sense, it pays to think of software testingas the headlights of a car — it allows you to see what’s comingon the road ahead in your development life cycle. With thiscapability, you can make better-informed decisions aroundyour system as you’re building it. The better the headlights,the more confidence you have. So where to start?

Testing on SAP Projects Earlier and Continuously

Business demands mean SAP implementations operate ina distributed and highly integrated environment wherestakeholders of a business process can be both internal and



36/76


external. This forces you to think of applications as the sum oftheir parts, and testing is no exception to this rule.

When building a test plan for a new application or a plan totest a change to an existing application, it is important notonly to think about logic, data, and exit criteria but also tounderstand the mechanics of how the application works froman integrated perspective — more specifically, how the systemis architected and how that affects your ability to execute achosen set of tests.

With complex, distributed systems, the primary risk is inte-gration. It is true that SAP systems are designed to talk toother systems — other SAP systems, systems incorporatingcomponents delivered by other vendors, or even systemsthat don’t reside within your enterprise. But even the bestdesigned systems can run into problems, so you need to testthese integrations as early as possible and as continuously aspossible. The problem is, you’re going to encounter a tapestryof differing standards, overlapping testing schedules, conflict-

ing methodologies, and contention around test environmentswhen trying to move forward with your testing strategy.

Testing below the PresentationLayer

The days of solely relying on testing via the user interface (UI)are long gone. The model you are presented tends to haveseveral layers, with the UI often being the outermost.Oftentimes, the majority of the risk lies at the integrationlayer. The integration layer may contain the business logic,data dependencies, and various technologies.

Automated integration testing below the UI provides greatbenefits to projects, uncovering integration defects early in

the development life cycle. In fact, testing below the UI is afundamental tenet of any “shift left” strategy.

The principles for automated integration testing of SAP sce-narios are the same as those for UI testing; you execute teststeps and compare what happens to an expected result. Thereare however, some differences to bear in mind:



37/76

Chapter 5: Getting to Know the Components 31

✓ You’ll have multiple message types and permutations ofdata to validate. See Figure 5-1. It’s not feasible to do all

this manually. To get satisfactory coverage, you need toautomate.

✓ The test steps are contained within the payload of themessage. Payload can be data driven in much the sameway as automated UI tests.

✓ Sending a message via the integration and validatingthe response do not always provide full confidencein a particular scenario. Integration testing introduces

the concept of testing what can be referred to as sideeffects. In other words, what else happens when I sendthis message? Is a database written to as expected? Is thelog file updated as defined? Test tools, like IBM RationalTest Workbench, allow you to validate at this level.

Figure 5-1: Seeing what lies below the user interface.

Service virtualizationIn addition to testing below the UI, the other key developmentin the testing of distributed SAP systems is service virtualization. In a typical end-to-end process, you may encounter anumber of distinct architectural components, which may bedelivered on different timelines and be owned by differentteams. Virtual services can stand in for these components inyour test environment when they are unavailable, as shown inFigure 5-2. This introduces greater flexibility into the testing



38/76


approach. With service virtualization, you can plan to testaround availability bottlenecks and also plan to test much ear-

lier than would have traditionally been the case.

Figure 5-2: Virtualizing components.

Testers in an SAP solution environment should consider twobasic scenarios where they can use service virtualization:

✓ The virtualization of SAP components

✓ The virtualization of non-SAP components

Working beyond the firewall Service virtualization is clearly a great tool to have whenworking with a complex, heterogeneous internal system, butit also can be extremely helpful when you have to deal with asystem that sits outside of the corporate firewall, say a third-party system or a business-to-business (B2B) interaction. Inthe bad old days, when you wanted to test, especially whenyou wanted to test early and end-to-end, you were at themercy of the priorities and availability of another company.These rarely aligned with your own goals and timelines and

caused delays — often weeks of delays — while you waitedfor a suitable slot around availability of those environments(systems) and the resources to provision and manage them.With the coming of service virtualization, you can sidestep atleast some of these barriers and thus mitigate the risk to theproject.

Shifting Left and PrioritizationThe ability to test below the UI and create virtualized serviceshas changed the acceptable risk profile for the delivery ofSAP systems. These two new approaches mean that you canapproach the prioritization of test cases much more aggres-sively. As tempting as it is to execute the easy tests first, tryto avoid the low hanging fruit until later on. Prioritize on the



39/76


more technically challenging and commercially beneficialfunctionality first and expose the bulk of the risk early on in

the software development cycle — shifting the timeline left, asit were.

It used to be the case that we’d put off testing of a particularfunctionality until “later on” because a component in theend-to-end test scenario was not available — the SAP ProcessIntegration (PI) server or SAP ERP Central Component (ECC)server, for example, or a non-SAP component such as a webservice. With service virtualization on the scene, being able to

virtualize an unavailable component means bottlenecks canbe avoided and test managers can plan to execute their testsas early as possible.

Integration testing andvirtualizing the key

SAP components When tackling automated integration testing, you inevitablyhave to deal with two major components in a typical SAP land-scape: the PI server and the ECC server. In addition, you havevarious adapters that are used to integrate the message pro-tocols and middleware. These are important not only becausethey offer test and virtualization points but also because thetester must think about whether testing the functionality ofthe adapter is an important part of the test case.

In the next few sections, you take a look at a typical automatedintegration testing process, where you see the recommendedapproach for the high-level aspects of the landscape — PI andECC — and then move onto more granular considerationsaround the messaging patterns. You look at the specific typesof testing SAP integration testers might do, why they mightwant to test in this way, and possible approaches they couldtake. In each case, you’ll be looking at the specific tools tes-ters would have at their disposal, including discovery andrecording. These concepts apply just as equally to SAP tech-nologies as well as a range of non-SAP technologies.



40/76


Testing and Virtualizing PISAP Process Integration (PI) acts first and foremost as anorchestration engine — a tool for modeling and executing busi-ness processes — but it also provides a bridge between theproprietary technologies used by the ECC and the open stan-dards used in a typical service-oriented architecture (SOA)enterprise. SAP PI accomplishes all this using the standardpattern of adapters, as shown in Figure 5-3. On the way intoSAP PI, messages are first adapted from incoming formats andtechnologies to an internal XML representation known as XI tobe processed and then adapted back into outgoing transports.In this way, an EDIFACT message — an electronic data inter-change (EDI) conforming to a UN-sanctioned standard — canbe received over the message queue (MQ), processed, and thensent as an intermediate document (IDOC) into ECC.

Figure 5-3: The pieces of the PI puzzle.

A PI system can contain two distinct technology stacks: SAP’sown Advanced Business Application Programming (ABAP)language and Java. The ABAP stack has similar functionalityto the remote function call (RFC) and IDOC features of an SAP

ECC system, so we’ll cover that in the ECC section later.

From a tester’s point of view, testing PI means testing thatthe business processes (rules) captured within it are operat-ing correctly, and any side effects are as expected. This isexecuted by sending messages via the adapters and verify-ing the results. As PI will be both retrieving information from



41/76


and sending information to other systems, it is common fora tester to virtualize those downstream systems to provide a

controlled, deterministic set of test conditions. (This type ofvirtualization is called service virtualization; we address thatparticular topic in greater detail in a bit.)

Discovery and SAP PIWithin a typical large organization, an SAP PI system will beorchestrating between large numbers of systems over a widerange of technologies. The ability to discover and record isvery important in an environment such as this in order to testquickly and efficiently without environment dependencies.

Discovery is the process by which you bring in informationabout the SAP environment into the test tool. That informa-tion could be environment variables, business flows, or iden-tification of downstream dependencies. With this information,you’re able to bring recording business processes as they flowacross the SAP environment.

Recording allows the tester to non-destructively pull messagesfrom the system as they flow through. These messages arestored and manipulated within the tool. From these recordings,testers can create automated integration tests and virtualizedservices.

Traditionally, testing tools in an SAP PI context were taskedwith discovering RFCs and IDOCs in the ABAP stack, but testtools today can now discover a whole host of other informa-tion. Generally speaking, this discovered information can bedivided into two categories:

✓ Services that the PI system is hosting: A tester may wantto test hosted services (to check that PI is orchestratingcorrectly) or virtualize them in order to make testing of adependent system easier.

✓ Services that the PI system is dependent on. These are

services that PI uses to fulfill its orchestrations or thatit passes information to. A tester is interested in suchdependent services because they represent validation points (you’ll want to test that PI has sent the correctmessage to a downstream system) and virtualization points (you’ll want to virtualize systems that PI is com-municating with to gather information in order to makethe environment simpler and test cases deterministic).



42/76


An SAP PI system contains a number of interfaces that allowtest tools to discover the landscape that the PI knows it is

operating within. If testers are using a tool that can properlyinterrogate these interfaces, they can quickly discover thelandscape and start recording real interactions that occurbetween the SAP PI and connected components. For example,this capability should allow them to discover web servicesand import Web Services Description Language (WSDL) files,identify queue managers for MQ/JMS communication, andidentify which databases PI is interacting with.

Recording PI interactions The fastest way to create tests, test data, and virtualizedservices is to record real interactions with real systemsbeing tested. With an SAP PI system, you have two recordingoptions, depending on which adapter architecture is present.

For all versions of SAP PI, your first choice is to record themessage stream coming into or out of the PI adapters. Thiscould be an MQ or JMS queue, SOAP, REST, or XML messageover HTTP. Most automated integration testing tools have away to record these messages and to use those recordings tobuild tests. The recordings can provide the template for mes-sages to be injected into the adapters to test PI as if they hadcome from an external system. In addition, such recordingsprovide a template for the validation of message structure andcontent that the PI sends out to other systems.

Virtualization with SAP PITesters should start by thinking about what they are trying toachieve with virtualization. Do they want to get on with testingeven if part of the environment isn’t ready yet? Or do they wantto get a service to respond in a particular way to satisfy testconditions? Or is the idea to introduce errors or delays for nega-tive testing? After figuring out precisely what it is you want toachieve, you’ll be in a better position to identify the most appro-priate virtualization point.

Deciding where to virtualize calls for a book in itself — in fact,there’s already one out there, appropriately entitled ServiceVirtualization For Dummies — so we can’t devote as muchspace to the topic as we’d like, but we’d at least like to look attwo scenarios:



43/76


✓ Virtualizing the entire SAP PI system. In this scenario,the tester is testing a system that depends on the SAP PI

system. It may be using SAP PI to pull information from avariety of sources, or it may be pushing data to SAP PI toupdate other systems. Testers want to simplify their envi-ronment and create deterministic test cases. Virtualizingaway the entire PI system lets them do that.

✓ Virtualizing other systems that SAP PI is communicat-ing with. Generally, you’d take this approach becausethe tester wants to test the orchestrations within SAP PIitself. These orchestrations can be complicated, as theytend to communicate with other downstream systems.Setting up all of these systems to provide just the rightresults can be a time-consuming and complex process.A tester can create virtual services to represent thosesystems and easily create test data to force the orches-tration into various test cases.

Testing SAP ECC The ECC system will be at the heart of many organizations.Although an SAP PI server is set up to provide the SOA-basedcommunication and orchestration capabilities you need, inmany cases it’s also acting as a bridge to allow non-SAP sys-tems to work with ECC. See Figure 5-4.

Figure 5-4: SAP ECC and PI in a sample architecture.



44/76


There are times when a tester needs to test the ECC systemdirectly to ensure that the core is working before stepping

back and testing the outer layers of the integration-onion.Testing the outer layers is easier if you can virtualize awaycertain aspects of the ECC system too, so you’ll want to lookat that option as well.

Given that you’re not working at the UI level, you have threeways to interact with an ECC system:

✓ Call RFC in order to cause some behavior or check some

data. ✓ Publish an IDOC into the ECC system in order to cause

some behavior.

“Publishing” here can include writing an IDOC file to diskto be imported into ECC.

✓ Watch the side effects of the above two actions, whichinclude IDOCs being sent from the ECC system to othersystems.

Let’s start with the simplest use case, testing an RFC.

Testing and virtualizing with RFCs An RFC call can be viewed as a simple operation that exposessome ECC functionality to the outside world. It’s similar to aremote procedure call or stored procedure call in that youpass in some data as structured arguments and get some

data back. There are typically two reasons for testing an RFC:a new one has been written (or an existing one changed) orfor regression

Documents

Testing SAP Solutions for Dummies