Start Oracle Health Insurance Back Office · PDF file · 2017-05-26Start . Oracle Health Insurance Back Office . ... Import Data Masking Definitions into OEM 45 ... Through data subsetting

Start

Oracle Health Insurance Back Office

O H I D a t a M a n a g e m e n t T e c h n i c a l R e f e r e n c e G u i d e

version 1.9

Part number: E87610-01

May 24, 2017

Data Management Technical Reference Guide

2

Copyright © 2013-2017, Oracle and/or its affiliates. All rights reserved.

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

If this software or related documentation is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable:

U.S. GOVERNMENT RIGHTS

Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are “commercial computer software” or “commercial technical data” pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR 52.227-19, Commercial Computer Software License (December 2007). Oracle USA, Inc., 500 Oracle Parkway, Redwood City, CA 94065.

This software is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications which may create a risk of personal injury. If you use this software in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure the safe use of this software. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software in dangerous applications.

Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.

This software and documentation may provide access to or information on content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services.

Where an Oracle offering includes third party content or software, we may be required to include related notices. For information on third party notices and the software and related documentation in connection with which they need to be included, please contact the attorney from the Development and Strategic Initiatives Legal Group that supports the development team for the Oracle offering. Contact information can be found on the Attorney Contact Chart.

The information contained in this document is for informational sharing purposes only and should be considered in your capacity as a customer advisory board member or pursuant to your beta trial agreement only. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described in this document remains at the sole discretion of Oracle.

This document in any form, software or printed matter, contains proprietary information that is the exclusive property of Oracle. Your access to and use of this confidential material is subject to the terms and conditions of your Oracle Software License and Service Agreement, which has been executed and with which you agree to comply. This document and information contained herein may not be disclosed, copied, reproduced, or distributed to anyone outside Oracle without prior written consent of Oracle. This document is not part of your license agreement nor can it be incorporated into any contractual agreement with Oracle or its subsidiaries or affiliates.


3

CHANGE HISTORY

Release Version Changes

10.14.2.0.0 1.3 • Added change history paragraph • Changed product name • Small textual changes

10.15.1.0.0 1.4 • Changes for database 12C • Small textual changes

10.15.3.0.0 1.5 • Added remark for VPD and the created database user for SS/DM

• Added advise for parameter parallel_degree_level for subset export as well as disabling archivelog mode

10.16.1.0.0 1.6 • Added remark about deferred segment creation • Added OHI Data Marts support

10.16.2.0.0 1.7 • Added changes for OEM 13C • Added work-around for subset definition import bug

19828552 10.16.2.2.0 1.8 • Pre-processing masking script SDM0001S.pl made

compulsory (bug 25252355) 10.17.1.0.0 1.9 • Added paragraph 4.2 about masking flex fields


4

Contents

1 Introduction 6

1.1 Subsetting and Masking 6

1.2 Prerequisites 9

1.3 Oracle Database Plug-in 9

2 High Level Design 11

2.1 Subsetting Process 11

2.2 Data Masking Process 13

3 Detailed Process - Subsetting 15

3.1 Provision Source Database to be Subset 15

3.2 Define Policy Selection in the Source Database 16

3.3 Generate and import Subsetting Application Data Model into OEM 19

3.4 Import Subsetting Definitions into OEM 23

3.5 Generate Subset Export File 30

3.6 Provision Target Database 35

3.7 Import Export File 36

3.8 Run the Post Import Script File 37

3.9 Install and Configure Other Custom Localizations 38

4 Detailed Process - Data Masking 39

4.1 Prepare to-be-masked Database (Target Database) 39

4.2 Masking flex fields 39

4.3 Generate and import Data Masking Application Data Model into OEM 40

4.4 Import Data Masking Definitions into OEM 45

4.5 Generate Masking Script 46

4.6 Move Large Tables with Sensitive Columns (Optional) 48

4.7 Run Masking Script on Target Database 49

5 OHI Data Marts 52

5.1 Prepare OHI Data Marts data-store 52

5.2 Loading OHI Data Marts 52


5


6

1 Introduction

Welcome to the technical reference guide for the Oracle Health Insurance Back Office (OHI-BO) Data Management application. This reference guide is intended to help you in using the data subsetting and data masking functionality as provided by the OHI Data Management product for OHI Back Office.

1.1 Subsetting and Masking

This guide describes how you can create a subset of a given OHI-BO data store. The subset contains less data than the given data store but at the same time is still fully functional for the OHI-BO application (that is all data integrity and consistency rules are upheld). Using data subsetting you create smaller versions of, for instance, production-size OHI-BO databases. These smaller databases can then be provisioned to development teams, test teams, or deployed in user-acceptance environments. Through data subsetting you can significantly reduce the total amount of disk storage required to support the various, and often many, non-production OHI-BO environments in your organization.

Next to data subsetting, this guide describes how OHI-BO data stores can be masked. Due to privacy regulations, organizations are obliged to deal with privacy sensitive data in a secure manner. Production environments usually have stringent data access control and auditing mechanisms in place to ensure that only those who need to access privacy sensitive data can do so. Typically those accessing the various non-production environments are not authorized to see privacy sensitive data or the data access control and auditing mechanisms are less stringent, or even absent, in these environments. With data masking you can mask (scramble, anonymize, pseudonymize) the privacy sensitive data elements in non-production OHI-BO environments. Development and test teams that use these masked environments are therefore not able to see privacy sensitive data. The masked data store is still fully functional for the OHI-BO application (that is all data integrity and consistency rules are upheld).

The intended audience for this technical reference guide is the DBA-group that administers the various OHI-BO environments inside an organization. This technical reference guide contains four chapters.

1. Introduction The chapter you are currently reading. This chapter introduces you to the data subsetting and data masking packages of the OHI-BO application.


7

2. High Level Design In chapter 2 you find a high level design of the data subsetting and data masking processes as they have been designed to operate on a data store of the OHI-BO application.

3. Detailed Design In chapters 3 and 4 you find a step-by-step description of the data subsetting and data masking processes. By following these steps you can create a subset of the OHI-BO application data store and mask this data store.

Data subsetting and data masking is implemented through functionality as provided through additional packs in Oracle Enterprise Manager. Both functionalities are accessed through the Quality Management submenu under the Enterprise menu of the Oracle Enterprise Manager. In this submenu you will find three items to access the subsetting and masking packs:

1. Application Data Modeling 2. Data Subsetting Definitions 3. Data Masking Definitions

See Figure 1.1 for a screenshot of this submenu.

Both the data subsetting and data masking packs depend upon an Application Data Model (ADM). ADMs are managed in the Application Data Modeling menu. An ADM captures all tables of the OHI-BO data store involved in the subsetting and data masking processes. For these tables the ADM defines:

• The referential relationships between these tables Subsetting and masking processes use these to "walk through" the data model.

• The type of table Tables can either be of type transactional or config (configuration). The transactional are usually subset and the config are usually not subset.

• Sensitive columns Columns marked as sensitive in the ADM have a masking format defined for them in the masking pack.


8

Figure 1.1: The Quality Management submenu in OEM13c.

Note: The OHI-BO masking process does not make use of the Data Masking Formats (the last entry in the submenu).


9

1.2 Prerequisites

The OHI-BO Data Management data subsetting and masking processes require the following components:

• OHI-BO environment with the appropriate release This environment is usually a dedicated copy of your production environment.

• OEM Cloud Control 13c release 1, version 13.1.0.0.0 • Oracle Database Plug-in release 13.1.0.0.0

See section 1.3 for instructions on how to check and upgrade this component.

Note: In order to use the subsetting and masking packs (which are part of the Oracle Database Plug-in), you must have a license for these two packs.

It is recommended to use a dedicated OEM Cloud Control instance for use with subsetting and masking. A production monitoring instance of OEM Cloud Control might require a different release of the Database Plug-in which is not certified for use in combination with subsetting and masking.

1.3 Oracle Database Plug-in

You ensure that the data subsetting and data masking packs are at the required release level for OHI-BO subsetting and masking by installing the Oracle Database Plug-in release 13.1.0.0.0.

Select Setup Extensibility Plug-ins and verify the version that is installed on the Management Server. See Figure 1.2.


10

Figure 1.2: Verifying current version of database plug-in.

If the required version is not yet installed and not visible in the Latest Available column, you can use the Self Update functionality of OEM13c to download the new version. See Figure 1.3.


11

Figure 1.3: OEM13c Self Update.

Please note that during upgrade of the Oracle Database Plug-in on the Management Server you are required to schedule downtime of the OEM13c instance.

2 High Level Design

This section contains a high level overview of the data subsetting and data masking processes as they apply to OHI-BO application.

2.1 Subsetting Process

Subsetting in general can be performed in two distinct modes:

1. All data that is not part of the subset is deleted from an existing data store. In this case database reorganization is required after running the subset process to reclaim the empty free space in the data files.

2. All data that is part of the subset is exported from an existing data store. The export file is imported into an empty (smaller) database after running the subset process.

The OHI-BO subsetting process as supported and implemented through the OHI Data Management product runs in the second mode.

Before starting with the subset process, you first have to import two XML-files into OEM:

• The ADM xml file holding a description of the OHI-BO tables to be subset. This file is named SDM_OHI_[release]_SUBSET_ADM.xml. This file should be generated through a utility that is part of the OHI Data Management product and delivered in an OHI Back Office release.


12

• The subsetting xml file holding the specification of the subset process. This file is named SDM_OHI_[release]_SUBSET_DEF.xml. This file is another main component of the OHI Data Management product and delivered in an OHI Back Office release.

Through the subsetting pages in OEM you can start the export job. This export job creates a dump file that contains the relevant subset of the source OHI-BO data store. You then have to prepare an empty target OHI-BO data store in a smaller database. Finally run an import job to load the subset into this smaller database. Figure 2.1 shows a high level overview of this process.

Figure 2.1: High level overview of subsetting process.

Next to the dump file, the subsetting export job also generates a SQL script file for import and a post import SQL script file (as shown in Figure 4). This import SQL script file contains the commands that import the dump file into the empty subset database. This script file can be run using SQLPlus. The post import SQL script file needs to be run after the data pump import has completed.

Note: The source database needs to be a quiescent database, as the export job runs for some time and the dump file needs to be a read-consistent snapshot of the source database. The source database is usually a dedicated RMAN copy (or Dataguard copy) from your production environment.


13

The target database needs to be correctly prepared before starting the import job. This is described in Chapter 3.

2.2 Data Masking Process

Data masking is performed in-place. The masking process is run on a source database, which is then masked. Before starting the masking process, you first have to import two XML-files into OEM:

• The ADM xml file holding the tables, their relationships and sensitive columns to be masked (named SDM_OHI_[release]_MASK_ADM.xml). Note that this is a dedicated ADM xml file for Masking; it is not the same ADM xml file as mentioned in Section 2.1 which is used for subsetting. This ADM xml file should be generated.

• The masking xml file holding a masking definition for each sensitive column (named SDM_OHI_[release]_MASK_DEF.xml). The definition xml is part of the OHI Data Management product and delivered in a Back Office release.

Through the masking pages in OEM you can start a job that generates a masking script. This masking script can be run from SQL*Plus and performs the actual masking of the sensitive columns in the target database. Figure 2.2 shows a high level overview of this process.

Figure 2.2: High level overview of masking process.


14

Chapter 4 contains a detailed description of the data masking process.


15

3 Detailed Process - Subsetting

As mentioned in Chapter 2.1 you need to acquire the two XML files that describe how the OHI-BO data store is to be subset, before starting the subsetting process.

• Subsetting application data model (SDM_OHI_[release]_SUBSET_ADM.xml) • Subsetting definitions (SDM_OHI_[release]_SUBSET_DEF.xml)

As part of the subsetting process you need to generate the ADM xml, and import both this generated ADM and the subsetting definition into OEM.

3.1 Provision Source Database to be Subset

The subsetting process starts by designating a source database as the database from which the subset is to be created. Oracle strongly advises that you do not use a live production database for the following reasons:

• The subsetting process places considerable load on the source database. • The data pump export sub process that performs the subset extraction, needs a single read

consistent snapshot of the source database. Therefore, if you use a non-quiescent database as the source database, the data pump export produces more load as rollback segments are heavily used to recreate a read-consistent snapshot.

• Subsetting can run in a delete mode as mentioned in Section 2.1.You run the risk of accidentally choosing the delete mode instead of the data pump export mode as described in step 6 of the subsetting process. In which case you would submit a job that starts deleting data from your live production database.

You would usually use a dedicated RMAN backup copy of the production database as your subsetting source database. Another alternative could be to use a Data Guard environment of your production database.

Note: As part of the subsetting process, you have to load a few test data management packages into the source database. This implies that the Data Guard environment has to be converted into a Snapshot Standby database first. When the subsetting process has completed, you can convert the Data Guard environment back into a Physical Standby database. See the "Data Guard Concepts and Administration" guide for more information.


16

Separate database user and ‘temporary’ tablespaces for subsetting

We recommend to create a separate database user and separate tablespace(s) (which can be deleted afterwards and is ‘temporary’ in that sense) for running the subset export process with. The subset export creates temporary working tables during the subsetting job. These tables are created in the default tablespace of the user credentials you supply when submitting the subset export job.

The default tablespace might need up to tens of Gb’s for an OHI Back Office database of a few terabytes. The temporary tablespace may grow much more, sizing advise is given later on in this document.

The user can be created as follows:

create user <USER> identified by <PASSWORD> default tablespace <SCRATCH TABLESPACE>

temporary tablespace <TEMP TABLESPACE THAT MAY GROW; SEE LATER>;

grant dba to <USER>;

grant execute on DBMS_AQADM to <user>;

By employing a separate database user and separate tablespace for its objects and potentially also a separate TEMP tablespace, you can easily reclaim the disk space occupied by the working tables and temporary segments by dropping these tablespaces afterwards.

For performance reasons it is best to de-activate all maintenance background jobs to prevent for example a parallel statistics gathering job uses unnecessary resources that delay the subset process.

Note: When Virtual Private Database (VPD) is activated in the OHI Back Office scheme this database user needs to be exempt from the VPD policies to be able to collect all required data. This can be done as follows:

grant exempt access policy to <user>

3.2 Define Policy Selection in the Source Database

OHI-BO subsetting currently uses the policy selection concept, as present within OHI Back Office, to determine which rows should be included in the subset.

A policy selection is a set of policy numbers. Typically you should determine a representative set of policies to be included in your target subset environment as this forms the base of the data that will be transferred to the subset database. All policy related data, including relevant claims, will be incorporated in the the subset. So determing a well thought over policy selection is an important aspect in setting up your subset environment.


17

So you have to create a policy selection that includes all relevant policy numbers that need to be in the subset. There are two ways you can do this:

1. Use the OHI Back Office Policy Selection window to setup a policy selection. 2. Use SQL*Plus to directly populate the underlying two tables of a policy selection.

Figure 3.1 shows the Policy Selection window. You have to enter a name for the policy selection (this name will be input for one of the steps later on). The description is optional. All other items can be left empty or at their default value. In the second block you enter all required policy numbers.

Figure 3.1: The Policy Selection window.

The two underlying tables for the Policy Selection screen are depicted in Figure 3.2. They are:

• VER#POLICY_SELECTIONS_ (Dutch name VER_POLIS_SELECTIES), and • VER#POL_PER_POS_ (Dutch name VER_POL_PER_POS).


18

It might be more convenient to use a SQL tool, such as SQL*Plus, to create the necessary rows manually in these tables.

Figure 3.2: Policy selection tables.

For example, the following two insert statements set up a policy selection named SAMPLE_OF_5_PERCENT that holds a random sample of 5% of the total number of policies available in the source database.

insert into ver#policy_selections_ (name) values('SAMPLE_OF_5_PERCENT');

insert into ver#pol_per_pos_ ( pos_id, pol_num )

select ( select id from ver#policy_selections_ where name = 'SAMPLE_OF_5_PERCENT')

, num

from ver#policies_ SAMPLE(5);

commit;

Note: The subsetting process uses a policy selection named OHI_SDM_POS_SUBSET internally. This name is reserved by the OHI-BO subsetting development team. You cannot use this name as it would interfere with the subsetting process.


19

3.3 Generate and import Subsetting Application Data Model into OEM

The ADM contains the data model for the OHI-BO application. This model is used by the subsetting process to calculate the subset. The ADM is specific to an OHI-BO release and must be generated for each new release. Prior to importing you should make sure that the ADM to be imported corresponds with the OHI-BO release installed for the source database.

To generate the ADM for subsetting, connect with SQL*Plus to the source database. Invoke the following command (as OHI Back Office application table owner):

exec sdm_adm_drv_pck.write_adm_files('DB_DIR');

Replace DB_DIR with the database directory of your choice.

Two files will be written to this directory:

• SDM_OHI_[release]_MASK_ADM.xml • SDM_OHI_[release]_SUBSET_ADM.xml

Once these files have been generated, you may want to transfer these to a file system that is accessible from your Desktop, so you can import them using Enterprise Manager. Open the Application Data Models page. See Figure 3.5.


20

Figure 3.5: Open Application Data Models

Make sure the ADM XML file is on your local desktop. Then select Actions Import File from Desktop. See Figure 3.6.


21

Figure 3.6: Starting the File from Desktop import.

Next enter a name for this ADM. Specifying the release number as part of the name of the ADM will help identify the correct ADM later on. Enter a description and select the subsetting source database. Then press Choose File and navigate to the ADM XML file on your local desktop. Finally press the OK button to start the import. See Figure 3.7.

Figure 3.7: Specifying Subsetting ADM XML file to be imported.

The ADM import process may take a while.


22

After the import process has completed, the ADM must be verified against the source database. Select the imported ADM in the list of available ADMs. Then select Actions Verify, and click Create Verification Job. See Figure 3.8.

Figure 3.8: Create Verification Job.

Make sure to uncheck Synchronize Application Data Model before submitting, see Figure 3.9.

Figure 3.9: Uncheck Synchronize Application Data Model.

When the verification job has completed, you need to check the verification results. For this select the ADM from the list of available ADMs, and select Action Verify. Now select the row with the database that you associated with the verification job. Look at the Verification Results. It should say: No verification results. See Figure 3.10.


23

Figure 3.10: Checking verification results.

3.4 Import Subsetting Definitions into OEM

The subsetting definitions XML file is imported from the Data Subset Definitions page. Navigate to this page by selecting Quality Management->Database Subset Definitions as shown in Figure 3.11. Make sure you have the XML file loaded on your desktop.


24

Figure 3.11: Open Data Subset Definitions

Select Actions Import File from Desktop in the Data Subset Definitions page. See Figure 3.12.


25


In the Import Data Subset Definition page enter a name (preferably include release name here too) and optionally a description. Select the corresponding ADM and select the source database. Finally select the subsetting XML file from your local desktop and press the Continue button. See Figure 3.13.


26

Figure 3.13: Import Data Subset Definition.

Specify the subset user credentials of the source database in the next page and press the Submit button See Figure 3.14.


27

Figure 3.14: Specifying credentials for import job.

A job is created to import the subsetting definitions from the XML file. Verify the job succeeded. This job should only run for a couple of minutes.

Note: Due to BUG 23249155 - Skipping addition of rule with Rule Type: TABLE_TYPE_RULE Schema, the result of the import definitions job may be Failed, and the following error may be reported in the output log:

Invalid schema/object - Skipping addition of rule with Rule Type: TABLE_TYPE_RULE Schema: OZG_OWNER Object: Configuration Data Tables Subset Type: ALL_ROWS Where Clause: All Rows Scope: ancestors

The subset table rule to import all tables containing configuration data failed to be imported. Other rules should have however be successfully imported. This bug has been fixed and will be patched in OEM 13c version 13.2. We advise customers to request a backport for OEM 13c version 13.1. However, the following work around may be applied to create the table rule for configuration data after the import has been completed:


28

Navigate to the import subset definition by selecting Quality Management Database Subset Definitions, and then open the edit window by selecting Actions Edit from the menu, as shown in Figure 3.a.

Figure 3.a: Edit the imported subset definition.

In the Edit winow, select the Object Rules tab, and click the Create button. In the creation window, select Objects of type Configuration Data, as shown in Figure 3.b, and click OK.


29

Figure 3.c: Create Object rule for configuration data.

Click Return to return to the Data Subsetting Definitions.

Note: When there is a need to import the subset definition for a second time after a subset export has been run, purging the internal SDM selection tables will speed up the import job. During the import, an estimation on the size of the subset is performed. When these internal tables are filled, this estimation will take considerable amount of time.

The internal SDM tables can be purged with the following command (as OHI Back Office application table owner):

exec sdm_util_pck.purge_sdm_data;


30

3.5 Generate Subset Export File

Recommended database configuration during export

Before starting the actual export job, the source database should be configured to maximize performance. The following parameters are recommended (and deviate from parameters required for OHI Back Office runtime):

• parallel_degree_limit = IO To limit the number of parallel query workers to the optimum found during IO Calibration. This calibration must be performed before setting this parameter to ‘IO’ (see below).

• parallel_max_servers - default value (>1), to enable parallel query during subset export

• parallel_degree_level - default value (100), as reducing the Degree Of Parallellism as wished for OHI Back Office during regular is not wanted for this export job

• optimizer_mode – default value (ALL_ROWS) • optimizer_index_cost_adjust – default value (100)

With respect to memory allocation, make sure the SGA (in particular the BUFFER_CACHE value) and PGA (for in-memory sorting and hash-area) are generously sized. As an indication: for a high volume environment (5+ TB database size) a sga_target of 16GB and a pga_aggregate_target of 12GB are found to be suitable values. These values of course depend on several system properties and the optimal values should be determined when running the subset export. The Memory Advisor included in Oracle Enterprise Manager can be a guide to find the optimum.

It is also advisable to disable the archivelog mode on the source database prior to starting the export job.

IO Calibration

To effectuate the ‘parallel_degree_limit=IO’ setting, IO Calibration needs to be performed. There are a number of ways to perform this calibration. One way is to start the IO Calibration from OEM: navigate to the ‘Performance Home’ of the database, click on the “I/O” tab and click the “I/O Calibration button” (see Figure 3.15). After specifying the number of disks and the maximum tolerable latency (10ms minimum), the calibration is started. The calibration process will take about 15 minutes.


31

Figure 3.15: Start IO Calibration from OEM

Another possibility is to start calibration from SQL*Plus. See http://docs.oracle.com/cd/E11882_01/server.112/e16638/iodesign.htm#PFGRF94384 for more information.

In MOS note “Automatic Degree of Parallelism in 11.2.0.2 (Doc ID 1269321.1)” a method is described to set the IO calibration data manually, without the need to run the IO Calibration. This could be useful when the used production-copy runs on hardware with different characteristics.

Temporary tablespace requirements

Make sure the temporary tablespace of the subset user (so the tablespace that is specified as temporary tablespace during the create user command and not its ‘scratch’ tablespace) is set to auto extend, and enough disk space is available for it to grow. During the export, tables can be written to temp space first before being written to the export file. This can consume large amount of temp space. As a guideline: make sure the temporary tablespace can hold at least the “subset %” of the size of the largest application table. For example, 0.05 times the largest application table when a 5% subset is created. When using multiple worker threads during export, temporary table space can be claimed by each worker. Multiply the subsetted size of the largest table by the number of workers.

𝑇𝐸𝑀𝑃 𝑠𝑖𝑧𝑒 ≈ (#𝑤𝑜𝑟𝑘𝑒𝑟𝑠) ∗ (𝑠𝑖𝑧𝑒 𝑜𝑓 𝑙𝑎𝑟𝑔𝑒𝑠𝑡 𝑎𝑝𝑝𝑙𝑖𝑐𝑎𝑡𝑖𝑜𝑛 𝑡𝑎𝑏𝑙𝑒)∗ (𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑠𝑢𝑏𝑠𝑒𝑡 𝑓𝑟𝑎𝑐𝑡𝑖𝑜𝑛)

For example, when using 2 worker threads for a 5% subset and the largest table being 1000GB: expect 2 * 1000 * 0.05 = 100GB of required temporary tablespace.

http://docs.oracle.com/cd/E11882_01/server.112/e16638/iodesign.htm#PFGRF94384�


32

Submitting the export job

The generation of the subset export file can now be started. From the Data Subset Definitions page select the subset definition to use for generating the subset and then select Actions Generate Subset. See Figure 3.17.

Figure 3.17: Generate Subset.

Specify the database from which the subset is to be exported (this is called target database here). Keep the ‘Create Subset By’ radio button default (Writing Subset Data to Export Files). Select the database and host credentials and specify the name of the policy selection to drive the subset generation. You created this policy selection in step 2 (Section 3.1.2). Then press the Continue button. See Figure 3.18.


33

Figure 3.18: Specify general parameters for subset process.

You enter the data pump parameters in the next page and specify the directory object that data pump must use to create the dump file in. This must be an existing directory object (one that the DBA must have created on beforehand, using the CREATE DIRECTORY command) inside the source database and the application owner schema, usually OZG_OWNER, must have read/write access on this directory object.

Provide the export file name, maximum file size and the number of worker threads that data pump must use whilst querying the source database and generating the export files.


34

The maximum file size should not be set too low, because the total export size is limited by the nr of files, which is 100 files maximum to be used and those files should together be large enough to contain the subset export dump. This means the exported database dump should fit into 100 * {maximum file size}. Hence the default file size of 100M accommodates for a 10G maximum export size.

Consider setting the number of worker threads to 2. Depending on the number of available CPU’s and IO capacity it will decrease the time needed for the export. When using two workers the table data is exported in parallel.

Please note that the Advanced Compression and Advanced Security licenses are required in order to use the compress and encrypt options. Specify the log file name and press the Continue button. See Figure 3.19.

Figure 3.19: Specify data pump parameters for the subset process.

Finally schedule the job and submit it. See Figure 3.20.


35

Figure 3.20: Schedule and submit the subset process job.

This job generates the following:

• One or more data pump export files and a corresponding log file These contain the subset of the data in the source database.

• An import sql script (tdm_import.sql) This script contains the commands that import the data pump export files

• A post import sql script (tdm_post_script.sql) This script contains commands that must be run on the target database after the data pump import has completed.

Once the subset generation job has completed, check the data pump log file for any errors.

Note: don’t forget to revert the changes made to the database parameters when using the source in an OHI Back Office runtime environment.

3.6 Provision Target Database


36

The data pump export that was created in the previous step, will have to be imported into a smaller target database. The export contains a user-mode export of the owner-schema of the OHI-BO application. This is usually the OZG_OWNER schema. You have to prepare an empty target database so that this user-mode export file can be successfully imported.

• Make sure the database is loaded with the required options. Those are currently the XDB and JVM options.

• Make sure the instance is running with all the required (s)pfile settings. Refer to the Oracle Health Insurance : Installation, Configuration and DBA Manual to verify these settings.

• Make sure the database has all the necessary OHI-BO tablespaces. In addition to the SYSTEM, TEMP, UNDO, USER and SYSAUX tablespaces, there are currently eighteen required application tablespaces (ozg_fact_rel_tab, ozg_fact_rel_ind, etc.). Create these as small tablespaces and ensure that they can grow (autoextend on).

• Make sure the database has the required OHI-BO schemas and roles. Running script $OZG_BASE/sql/OZGI001S.sql will create these. The import job ensures that these schemas and roles receive all the object privileges again.

• Optionally create your own custom schemas and roles. When your OHI-BO environments have additional (custom) schemas, roles or both that have object grants from the OHI-BO application owner schema, the import job successfully restores the object grants to these schemas also when they are created in advance.

It is also advisable to disable the archivelog mode on the target database prior to starting the import job.

Note: The above describes only the database-side of the target environment. Of course you also need a client environment (contents of $OZG_BASE) with the forms, reports, batch scheduler and so forth. This client environment can simply be a one-on-one copy of the client environment for the source.

3.7 Import Export File

The subset can now be loaded into the target database that was prepared in the previous section. You have to transfer the export dump files from the source database host to the target database host and create a directory object in the target database that points to the directory on the host holding


37

the export files. Also transfer the tdm_import.sql and tdm_post_script.sql scripts that were generated in step 6.

Next start up SQL*Plus on the target host, connect as SYS and start the tdm_import.sql script. This script creates a few objects and then requests two inputs: the state of the schemas and the directory object name. Select option 2 for the first input and enter the name of the directory object that holds the export files. See Figure 3.21.

Figure 3.21: Enter data pump import parameters.

Errors are logged during the import which causes objects in the OHI-BO application schema to remain invalid due to a known issue in the way the subsetting pack interacts with data pump. These errors are resolved by the post import script.

You supplied a password for the application owner schema (typically OZG_OWNER) during preparation of the target database (as described in the previous Section). The tdm_import.sql script drops this schema and has it recreated by the data pump import process. This means that you lose the password that you have supplied: after the import, it is again set to the password that was in place in the source environment.

3.8 Run the Post Import Script File

Go into SQL*Plus on the target database, connect as SYS and run the post import script: tdm_post_script.sql. To run the post import script on the target, first set the environment correctly, i.e.:

. ozg_init.env <env>

. ozg_init.env $OZG_ORATAB_DB12102


38

Make sure the [SID]_install wallet entry exists and connects to the application owner schema of the target database (you may want to reset the application owner password in the target database at this stage).

After running this script all stored PL/SQL objects should be valid. You now have a subset of the OHI-BO data store. Note however that some tasks still remain. For instance, you still need to set up the application users. This target database will have the ALG#USERS_ table contents (Dutch name: ALG_FUNCTIONARISSEN) of the source database. You may want to update this table and create the corresponding Oracle application user schemas for it.

As a final verification you can run the object check script for the OHI-BO application (script SYS9006S). Note: This script assumes that the batch scheduler has been started. Whether the client and database are in a correct state is verified by Script SYS9006S.

3.9 Install and Configure Other Custom Localizations

Any adjacent schemas containing for instance custom code and data should be installed and configured as the last step for local customizations. If you need to subset the custom data also you need to develop your own custom subset implementation for this.


39

4 Detailed Process - Data Masking

Contrary to subsetting, where an empty target database is filled with a subset of data, the data masking process executes "in-place" within an existing OHI-BO data store. As mentioned in Section 2.2 you need to acquire the two XML files that describe how the OHI-BO data store is to be masked before starting the masking process.

• Masking application data model (SDM_OHI_[release]_MASK_ADM.xml) • Masking definitions (SDM_OHI_[release]_MASK_DEF.xml)

4.1 Prepare to-be-masked Database (Target Database)

Data masking can be performed on a full-size OHI-BO data store, or on a subset OHI-BO data store. Obviously the masking process takes more time on full-size data stores. During masking, tables with sensitive columns are temporarily duplicated (this is further explained in Section 4.4). For this reason it is necessary to check that tablespaces holding the table segments either have enough free space available, or are able to grow (autoextend) during the masking process. Upon completion of data masking a database (or tablespace) reorganization of some kind would have to be performed to reclaim the then remaining free space. Oracle describes an approach in Section 4.5 to prevent having to execute such reorganization.

Before masking the target database, it is advisable to create a backup of the database. Also depending upon the size of the archivelog destination file system, it may be advisable to disable archive logging during the masking process, and re-enable it afterwards.

4.2 Masking flex fields

In the OHI BO application it is possible to indicate if the value for a specific flex field should be masked. In window ZRG7019F (Flex field) the indication “Mask?” can be set to “Yes” in order to mask the value for this flex field. If set to “No” (default value) masking will not take place.


40

Note: The value of a flex field can be steering for processes like the claims processing. Masking of these flex fields can undo this and will lead to unpredictable and/or undesirable results. Therefore, the advice is to mask only those flex fields that are actual privacy-sensitive.

4.3 Generate and import Data Masking Application Data Model into OEM

The ADM contains the data model for the OHI-BO application. This model is used by the masking process to identify the sensitive columns and relationships between the tables. The ADM is specific to an OHI-BO release and must be generated using the source database. Prior to importing you should make sure that the ADM to be imported corresponds with the OHI-BO release installed at the source database.

To generate the ADM for masking, connect with SQL*Plus to the source database. Invoke the following command (as OHI Back Office application owner):

exec sdm_adm_drv_pck.write_adm_files('DB_DIR');

Replace DB_DIR with the database directory of your choice.


41

Two files will be written to this directory:

• SDM_OHI_[release]_MASK_ADM.xml • SDM_OHI_[release]_SUBSET_ADM.xml

Once these files have been generated, you may want to transfer these to a file system that is accessible from your Desktop, so you can import them using Enterprise Manager.

Open the Data Discovery and Modeling page. See Figure 4.1.

Figure 4.1: Open Data Discovery and Modeling


42

Make sure the ADM XML file is on your local desktop. Then select Actions Import File from Desktop. See Figure 4.2.


Next enter a name for this ADM. Specifying the release number as part of the name of the ADM will help identify the correct ADM later on. Enter a description and select the masking source database. Then press Choose File and navigate to the ADM XML file on your local desktop. Finally press the OK button to start the import. See Figure 4.3.


43

Figure 4.3: Specifying masking ADM XML file to be imported.

The ADM import process may take a while.

After the import process has completed, the ADM must be verified against the source database. Select the imported ADM in the list of available ADMs. Then select Actions Verify, and click Create Verification Job. See Figure 4.6.

Figure 4.6: Create Verification Job.

Make sure to uncheck Synchronize Application Data Model before submitting. See Figure 4.7.


44

Figure 4.7: Uncheck Synchronize Application Data Model.

When the verification job has completed, you need to check the verification results. For this select the ADM from the list of available ADMs, and select Action Verify. Now select the row with the database that you associated with the verification job. Look at the Verification Results. It should say: No verification results. See Figure 4.8.


45

Figure 4.8: Checking verification results.

4.4 Import Data Masking Definitions into OEM

The masking definitions XML file is imported from the Data Masking Definitions page. Press the Import button at the top right (Figure 4.9). Make sure you have the XML file loaded on your desktop.

Figure 4.9: Starting masking definitions import.

Adapting the definition for a specific application schema name

When the name of the application schema is other than ‘OZG_OWNER’, the subset definition needs to be modified before being imported. All references to OZG_OWNER in the SDM_.._MASK_DEF.xml must be replaced with the actual name of the application schema.

This renaming can be performed using the Linux command ‘sed’, for example for alternative schema ‘OZADMIN’: $> sed -i.bak -e 's/OZG_OWNER/OZADMIN/g' <masking_definition.xml>

It will create a backup of the original.

Import the (modified) definition

In the Import Masking Definitions page enter a name (preferably include release name here too) and optionally a description. Select the corresponding masking ADM and select the source database (which is also the target database). Finally select the masking XML file from your local desktop and press the Continue button. See Figure 4.10.


46

Figure 4.10: Import Masking Definition.

The masking definitions xml file will now be imported and should be available on the Data Masking Definitions page.

4.5 Generate Masking Script

Once the masking definitions have been imported, the next step is to generate a masking script from these definitions. On the Data Masking Definitions page, select the masking definition and press the Generate Script button. See Figure 4.11. Important: it is necessary to generate this script against the to-be-masked database. Running the script against a different target might result in errors.


47

Figure 4.11: Generate Script (Masking).

The Schedule Script Generation Job page displays. Specify the SYS credentials and submit the job. This job runs for a considerable amount of time during which all tables with sensitive columns are inspected. The end result of this job is a SQL script file which holds the actual commands to perform the data masking later in Section 4.6.

As mentioned earlier in Section 4.1, tables with sensitive columns are temporarily duplicated during the actual masking. The way the masking script performs is as follows:

• Tables with sensitive columns are renamed. Indexes and triggers are removed from the renamed tables.

• For each renamed table a "create table … as select …" (CTAS) is performed to recreate the table under the original name. During this CTAS the actual data masking is performed.

• Indexes and triggers are restored on the newly created tables. Also object-privileges for these tables are restored.

• The renamed original tables are dropped. • Invalid objects are recompiled.

The CTAS statements are performed with the space definition clauses that were in place on the original table during the generation of the masking script. This means that there are two segments for these tables in the tablespaces allocated to these tables during masking. When data masking is performed on a full-size OHI-BO data store this could substantially increase the tablespaces holding tables with sensitive columns. This is the reason that why in Section 4.1 it was mentioned that during masking these tablespaces should either have enough free space available or be able to extend as required.

Take a close look at the output of the ‘Generate Masking Script’ Job. The Job will check whether enough free space is available in all tablespaces, it will report issues in the output log.

For example, it may report:

RESOURCE WARNING TABLESPACE OZG_FACT_FIN_TAB Insufficient free space in Tablespace OZG_FACT_FIN_TAB even with automatic extension.

Some operations involving Tablespace OZG_FACT_FIN_TAB may fail.

Starting Freespace: 25610MB.

Ending Freespace (assuming increase in size): 9536MB.

Lowest Freespace: -18960MB.

Increase size of Tablespace OZG_FACT_FIN_TAB to at least 163418MB.


48

Make sure to add additional data files or extend options when reported, before running the masking script.

When the job reports errors like the following example:

ERROR TABLE OZADMIN.COM_COMMISSIEFACTUREN Unable to reorganize Table OZADMIN.COM_COMMISSIEFACTUREN. Either the object does not exist in the target database, or its reorganization is not supported. Remove the object from the reorganization and regenerate the script.

Run the following statement to materialize tables without data

BEGIN DBMS_SPACE_ADMIN.MATERIALIZE_DEFERRED_SEGMENTS(schema_name => 'OZG_OWNER'); END; /

When OZG_OWNER is not the owner of the OHI Back Office scheme, please replace OZG_OWNER with the correct owner.

Als the following warning could be reported:

WARNING null The columns [OZADMIN.RBH_EIGEN_REKENINGEN.FILIAALCODE] are masked using Shuffle format. Data Masking cannot maintain data distribution due to one of the following reasons: There are dependent columns or the columns are conditionally masked.

This specific warning for this table can be ignored for Dutch customers as this field is not used for bank account information in the Netherlands. The reason for this warning is the fact that this column is part of a unique key and in theory could give issues when masked.

4.6 Move Large Tables with Sensitive Columns (Optional)

A way to prevent the necessary tablespace reorganizations to reclaim the free space after running data masking is as follows:

• After the masking script has been generated, inspect this script and locate all tables with sensitive columns. A way to do this is to use the grep utility:

grep "CREATE TABLE \"OZG_OWNER\"" [generated-masking-sql-script]


49

This assumes that OZG_OWNER is the application owner schema. • Temporarily create a tablespace into which these tables can be moved. Also assign a

tablespace quota to the application owner on this new tablespace. • Using the output of the grep command generate "alter table … move … tablespace"

commands. • Run these commands prior to starting the masking script.

The CTAS statements creates new table segments in the corresponding table tablespaces which should now have enough free space, due to the fact that the original tables have been moved out of them during data masking. After completion of the masking script the temporary created tablespace should be empty again and can be dropped thus reclaiming the extra space that was required during data masking.

4.7 Run Masking Script on Target Database

Recommended database configuration during masking

Before running the masking script on the target, it should be configured to maximize performance. The following parameters are recommended (and deviate from parameters required for Back Office runtime):

• parallel_degree_limit = IO (See chapter 3.5 for more details about this parameter and the importance of IO calibration)

• parallel_max_servers - default value (>1), to enable parallel query during subset export

• optimizer_mode – default value (ALL_ROWS) • optimizer_index_cost_adjust – default value (100)

As noted in the subsetting chapter, the sizing of both PGA and SGA is important to achieve a good performance.


50

The data masking script that was generated in Section 4.4, can be downloaded from OEM by selecting the masking definition in the Data Masking Definitions page, and by selecting Save Script. See Figure 4.12.

Figure 4.12: Save masking script.

Next, press the Go button. The masking script downloads to your desktop. Transfer the masking script to the target database server.

Prior to running the masking script, it is necessary to run the perl pre-processing script SDM0001S.pl. This script has 2 functions:

1. If you are using a application schema name other than ‘OZG_OWNER’, you may need to replace this in the masking script. The perl script SDM0001S.pl has been provided to alter the script and replace the application schema name.

2. As a solution to bug 25252355, the script will remove procedures which cause problems during the execution of the script. These procedures re-create instead-of-triggers, but fail because of invalid public synonyms. The procedures may be safely disabled because they are not necessary for masking data.

This script SDM0001S.pl may be found in the $OZG_BASE/sh directory. Run the script with option –h for help on how to use it.

Finally, start the masking script from SQL*Plus as SYS.

After that the masking proceeds. Depending on the size of the data store that is to be masked, this can take a considerable amount of time.


51

The masking script spools a log file in the current directory. This script runs in the "whenever sqlerror exit" mode. Upon completion of the masking script execution, inspect the log file and check for unexpected "ORA-" errors. Depending on the situation and the stage at which the error occurred, you might be able to fix the issue and rerun the script. Alternatively you have to restore the target database first, solve the cause of the issue, and then rerun the masking script.

If you had moved the tables with sensitive columns to a temporarily created tablespace (as described in Section 4.5) then you can drop that tablespace now.

Note: revert the changes made to the database parameters for use in a Back Office runtime environment.


52

5 OHI Data Marts

From release 10.16.1.0.0 onwards OHI Data Marts (OHI-DM) is certified to extract data from sub-setted and/or data-masked OHI Back Office (OHI-BO) data stores.

The following types of OHI-BO data stores are supported:

• sub-setted • sub-setted and data-masked • data-masked

5.1 Prepare OHI Data Marts data-store

An empty OHI-DM data store is required to be able to load data from a sub-setted and/or data-masked OHI-BO data store into OHI-DM. The SQL script OBDRESET.sql (available within OZG_TEMPLATES.zip as of release 10.16.1.0.0) is provided to create an empty data store by truncating all the necessary OHI-DM tables. This script can be run using SQL*Plus using the OBD_OWN account.

It is strongly advised to create a clone from an existing OHI-DM environment which is at the same patch-level as the OHI-BO data store, and use this cloned environment to run the SQL script OBDRESET.sql against. Make sure the database link SRC_OPENZORG is referring to the correct OHI-BO data store.

NOTE: This SQL script should never be used directly within a production environment!

When it is required to be able to install patch-sets and/or interim patches also the steps described in 'APPENDIX D: OWB 11GR2 POST-CLONING PROCESS FOR OHI DATA MARTS' of the Oracle Health Insurance Data Marts Administrator Reference need to be performed as well.

5.2 Loading OHI Data Marts

Performing loads from an sub-setted and/or data-masked OHI-BO environment is identical to loading from a normal OHI-BO environment. See online help for information on loading (topic ‘Loading OHI Data Marts’).

Documents

Start Oracle Health Insurance Back Office · PDF file · 2017-05-26Start . Oracle Health Insurance Back Office . ... Import Data Masking Definitions into OEM 45 ... Through data subsetting