5
1. Process Control Definition Process control or Auditing of a workflow in an Informatica is capturing the job information like start time, end time, read count, insert count, update count and delete count. This information is captured and written into table as the workflow executes 2. Structure of Process Control/Audit table The table structure of process control table is given below, Table 1: Process Control structure PROCESS_RUN_ID Number(p,s) 11 A unique number used to identify a specific process run. PROCESS_NME Varchar2 120 The name of the process (this column will be populated with the names of the informatica mappings.) START_TMST Date 19 The date/time when the process started. END_TMST Date 19 The date/time when the process ended. ROW_READ_CNT Number(p,s) 16 The number of rows read by the process. ROW_INSERT_CNT Number(p,s) 16 The number of rows inserted by the process. ROW_UPDATE_CNT Number(p,s) 16 The number of rows updated by the process. ROW_DELETE_CNT Number(p,s) 16 The number of rows deleted by the process ROW_REJECT_CNT Number(p,s) 16 The number of rows rejected by the process. USER_ID Varchar2 32 The etl user identifier associated with the process.

Audit Table Process Control

Embed Size (px)

Citation preview

Page 1: Audit Table Process Control

1. Process Control – Definition

Process control or Auditing of a workflow in an Informatica is capturing the job information like start time, end time,

read count, insert count, update count and delete count. This information is captured and written into table as the

workflow executes

2. Structure of Process Control/Audit table

The table structure of process control table is given below,

Table 1: Process Control structure

PROCESS_RUN_ID Number(p,s) 11

A unique number

used to identify a

specific process run.

PROCESS_NME Varchar2 120

The name of the

process (this

column will be

populated with the

names of the

informatica

mappings.)

START_TMST Date 19

The date/time when

the process started.

END_TMST Date 19

The date/time when

the process ended.

ROW_READ_CNT Number(p,s) 16

The number of rows

read by the process.

ROW_INSERT_CNT Number(p,s) 16

The number of rows

inserted by the

process.

ROW_UPDATE_CNT Number(p,s) 16

The number of rows

updated by the

process.

ROW_DELETE_CNT Number(p,s) 16

The number of rows

deleted by the

process

ROW_REJECT_CNT Number(p,s) 16

The number of rows

rejected by the

process.

USER_ID Varchar2 32

The etl user

identifier associated

with the process.

Page 2: Audit Table Process Control

3. Mapping Logic and Build Steps

The process control flow has two data flows, one is an insert flow and the other is an update flow. The insert flow runs

before the main mapping and update flows runs after the main mapping, this option is chosen in “Target Load Plan”.

The source for both the flows could be a dummy source which will return one record as output, for exampleselect

‘process’ from dual or select count(1) from Table_A. The following list of mapping variable is to be created,

Table 2: Mapping Parameter and variables

$$PROCESS_ID

$$PROCESS_NAME

$$INSERT_COUNT

$$UPDATE_COUNT

$$DELETE_COUNT

$$REJECT_COUNT

Page 3: Audit Table Process Control

Steps to create Insert flow:

1. Have “select „process‟ from dual” as Sequel in source qualifier

2. Have a sequence generator to create running process_run_Id ‟s

3. In an expression SetVariable ($$PROCESS_RUN_ID,NEXTVAL), $$PROCESS_NAME to o_process_name,

a output only field

4. In an expression assign $$SessionStarttime to o_Starttime, an output only field

5. In an expression accept the sequence id from sequence generator

6. Insert into target‟ process control table‟ with all the above three values

Table 3:

Process Control Image after Insert flow

Page 4: Audit Table Process Control

PROCESS_RUN_ID 1

PROCESS_NME VENDOR_DIM_LOAD

START_TMST 8/23/2009 12:23

END_TMST

ROW_READ_CNT

ROW_INSERT_CNT

ROW_UPDATE_CNT

ROW_DELETE_CNT

ROW_REJECT_CNT

USER_ID INFA8USER

Steps in main mapping,

1. After the source qualifier, increment the read count in a variable (v_read_count) for each record been read in

an expression and SetMaxVariable ($$READ_COUNT,v_read_count)

2. Before the update strategy of target instances, do the same for Insert/Update/Delete counts; all the variables

are now set with all their respective counts

Steps to create Update flow:

1. Have “select „process‟ from dual” as Sequel in source qualifier

2. Use SetMaxvariable to get the process_run_id created in insert flow

3. In an expression assign $$INSERT_COUNT to an o_insert_count, a output only field, assign all the counts in

the same way

4. In an expression assign $$SessionEndtime to o_Endtime, an output only field

Page 5: Audit Table Process Control

5. Update the target „Process Control Table‟ with all the above three values where process_run_id equals the

process_run_id generated in Insert flow

Table 4: Process Control Image after Update flow

PROCESS_RUN_ID 1

PROCESS_NME VENDOR_DIM_LOAD

START_TMST 8/23/2009 12:23

END_TMST 8/23/2009 12:30

ROW_READ_CNT 1000

ROW_INSERT_CNT 900

ROW_UPDATE_CNT 60

ROW_DELETE_CNT 40

ROW_REJECT_CNT 0

USER_ID INFA8USER

4. Merits over Informatica Metadata

This information is also available in Informatica metadata, however maintaining this within our system has following

benefits,