Db2 For I Parallel Data Load

DB2 for i External Stored Procedure Example

Page 1 of 10 1/3/2011 4:49:44 PM

Rational for i and Optim Studio Development Environment ............................................. 1

Load Process Flowchart (ACS_i5Load and ACSGenesis)................................................ 2

Run Stored Procedure from Optim and Rational for i packages (client) ............................ 3

CPYFRMIMPF commands are submitted for batch processing on i ................................. 3

Module syb_i5_LoadFromDelimitedFile in service program ACSGENESIS ................... 4

DB2 External Stored Procedure Interface........................................................................... 6

DB2 External Stored Procedure Program........................................................................... 7

Benefits of Stored Procedure Processing versus ODBC/JDBC statements........................ 8

Two Types of Stored Procedures .................................................................................... 8

Maintenance performed in one place.............................................................................. 8

Get Job Completion Status of Stored Procedure with a Select Statement.......................... 9

References......................................................................................................................... 10

Rational for i and Optim Studio Development Environment

• We would like to load many tables as fast as possible without locking our GUI for

an extended timeframe.


Page 2 of 10 1/3/2011 4:49:44 PM

Load Process Flowchart (ACS_i5Load and ACSGenesis)


Page 3 of 10 1/3/2011 4:49:44 PM

Run Stored Procedure from Optim and Rational for i packages (client)

CPYFRMIMPF commands are submitted for batch processing on i


Page 4 of 10 1/3/2011 4:49:44 PM

Module syb_i5_LoadFromDelimitedFile in service program ACSGENESIS

// * ============================================================== *

// * DB2 for i: Load from delimited files (syB) to ACS tables (db2) *

// * ============================================================== *

p syb_i5_LoadFromDelimitedFile...

p b export

d syb_i5_LoadFromDelimitedFile...

d pi n

d i5_Context 32 const

d i5_Working 128 const

d i5_Schema 128 const

d i5_BegPos s 5u 0 inz(0)

d i5_EndPos s 5u 0 inz(0)

d i5_LibFil s 20 inz(' ')

d i5_CpyFil s 1024 inz(' ')

d exist s 1 inz('0')

d existIndicator s n based(sqlPointer)

d sqlPointer s * inz(%ADDR(exist))

d i5_LibNme s 10 inz(' ')

/Free

Monitor;

// * ============================================================== *

// * Check / Set import application context and working directories *

// * ============================================================== *

If isApplicationWorkingDirectoryValid( %trim(i5_Context):

%trim(i5_Working) );

i5_dirName = %trim(i5_Context) + '/' + %trim(i5_Working);

If i5_Schema = 'QUSER';

i5_LibNme = RtvCurLib( pgmStatus.jobName:

pgmStatus.userId:

pgmStatus.jobNumber );

Else;

i5_LibNme = pgmStatus.userId;

EndIf;

i5_dirPtr = OpenDir(%trim(i5_dirName));

If (i5_dirPtr <>*NULL);

DoU i5_dirEntPtr = *Null;

i5_dirEntPtr = ReadDir( i5_dirPtr );

Monitor;

On-Error *All;

EndMon;

If i5_dirEntPtr = *Null;

Leave;

Else;

i5_entryName = %str(%addr(i5_nameFull));

// * ============================================================== *

// * Filter navigation entries (.) and select .data delimited files *

// * ============================================================== *

If %subst(i5_entryName: 1: 1) <> '.';

i5_EndPos = %scan('.data':

%xlate(upper: lower: %trim(i5_entryName)): 1);

If i5_EndPos <> 0;

i5_BegPos = %scan('.': %xlate(upper: lower:

%subst(%trim(i5_entryName): 1: i5_EndPos - 1)): 1);


Page 5 of 10 1/3/2011 4:49:44 PM

// * ============================================================== *

// * i5 internal system object name from DB2 schema catalog entries *

// * ============================================================== *

i5_LibFil = syb_i5_getTableName(%trim(i5_LibNme):

%subst(%trim(i5_entryName)

:(i5_BegPos + 1)

:((i5_EndPos - i5_BegPos) - 1)));

If i5_LibFil <> ' ';

// * ============================================================== *

// * Construct i5 CpyFrmImpF (Copy From Import File) command string *

// * ============================================================== *

i5_CpyFil = 'CPYFRMIMPF FROMSTMF(' + ticMark +

%trim(i5_Context) + '/' +

%trim(i5_working) + '/' +

%trim(i5_entryName) + ticMark +

') TOFILE(' + %subst(i5_LibFil: 1: 10) +

'/' + %subst(i5_LibFil: 11: 10) +

') MBROPT(*REPLACE) RCDDLM(*CRLF' +

') DTAFMT(*DLM) STRDLM(' + ticMark +

'"' + ticMark + ') RMVBLANK(*TRAILING' +

') FLDDLM(' + ticMark + ',' + ticMark + ')';

// * ============================================================== *

// * Remove joblog messages, prepare for submit job status capture *

// * ============================================================== *

RemoveMessage(rmv_PgmQue:

rmv_StkCnt:

rmv_MsgKey:

rmv_Remove:

error_Code);

// * ============================================================== *

// * Construct i5 SbmJob (Submit Job) command (CpyFrmImpF) queuing *

// * ============================================================== *

i5_CmdStr = 'SBMJOB CMD(' + %trim(i5_CpyFil) +

') INLLIBL(*CURRENT) JOBD(*USRPRF) JOBQ(*JOBD' +

') JOB(' + %trim(%subst(i5_LibFil: 11: 10)) +

') MSGQ(*LIBL/ACS_GEN500) HOLD(*YES) PRTTXT(' +

ticMark + %subst(%trim(i5_entryName)

:(i5_BegPos + 1)

:((i5_EndPos - i5_BegPos) - 1)) +

ticMark + ')';

CallP(e) i5_RunCmd(%trim(i5_CmdStr): %len(%trim(i5_CmdStr)));

If %Error;

snd_Data = 'LOAD FAILURE: ' + %trim(i5_entryName) +

' in ' + %trim(pgmStatus.mainProcName);

Else;

snd_Data = 'LOAD SUCCESS: ' + %trim(i5_entryName) +


// * ============================================================== *

// * Receive joblog messages prepare for submit job status capture *

// * ============================================================== *

ReceiveMessage(MessageInfo :%size(MessageInfo)

:'RCVM0100' :'*' :0 :'*ANY' :MsgKey

:0 :'*OLD' :error_Code);

If i5_ByteProv = 0;

If messageInfo.common.QMHMI03 = 'CPC1221';

i5_JobNme = %subst(messageInfo.errMsgDta: 1: 10);

i5_UsrNme = %subst(messageInfo.errMsgDta: 11: 10);


Page 6 of 10 1/3/2011 4:49:44 PM

i5_JobNbr = %subst(messageInfo.errMsgDta: 21: 6);

i5_UsrDta = %subst(%trim(i5_entryName)

:(i5_BegPos + 1)

:((i5_EndPos - i5_BegPos) - 1));

If Not WrtLogSts('ACSLOAD_01': i5_TskCnt: i5_JobNme:

i5_UsrNme: i5_JobNbr: i5_UsrDta);

// Add additional error handling logic here ======>

EndIf;

EndIf;

Else;

// Add additional error handling logic here ======>

EndIf;

EndIf;

EndIf;

Else;

snd_Data = 'LOAD WARNING: ' + %trim(i5_entryName) +


EndIf;

CallP(e) sndjobmsg(snd_Data); // Job Log Message Entry

EndIf;

EndIf;

EndDo;

existIndicator = *ON; // Return SUCCESS code

Else;

i5_errNoPtr = errno();

snd_Data = 'Open directory failed from ' + %trim(i5_dirName) +

' in program ' + %trim(pgmStatus.mainProcName);


EndIf;

closeDir(i5_dirPtr);

EndIf;

On-Error *All;

snd_Data = 'Unexpected error in syb_i5_loadFromDelimitedFile()' +

' in program ' + %trim(pgmStatus.mainProcName);


EndMon;

Return existIndicator;

/End-Free

p syb_i5_LoadFromDelimitedFile...

p e

DB2 External Stored Procedure Interface

Create Procedure i5ParallelDataLoader (

IN Type char( 10) )

LANGUAGE RPGLE

SPECIFIC ACS_I5LOAD

NOT DETERMINISTIC

External Name ACS_I5LOAD

CALLED ON NULL INPUT

PARAMETER STYLE DB2SQL;


Page 7 of 10 1/3/2011 4:49:44 PM

DB2 External Stored Procedure Program

* ================================================================== *

* Program name...: ACS_i5LOAD *

* Purpose........: Load database tables from delimited files *

* Programmer.....: Thomas and Victoria Wolfe *

* Create Date....: December 7, 2010 *

* *

* Maintenance Log: *

* *

* PGMR SEARCH DATE DESC. OF CHANGE *

* *

* ================================================================== *

/COPY qCPYSRC,ACS_H_PGM

/COPY QPRTSRC,ACSGENESIS

d i5_LoadType s 10

d i5_NullInd1 s 5i 0

d sqlStateCode s 5

d functionName s 517

d specificName s 128

d errorMessage s 70

d i5_ProdLoad s 10 inz(' ')

d message s 80 inz(*Blank)

d command s 2048 inz(' ')

* ================================================================== *

* Common program status information data structure (SDS) *

* ================================================================== *

/COPY QCPYSRC,ACS_SQLOPT

* ================================================================== *

* i5 DB2 Parallel Data Loader: Submit CpyFrmImpF command to batch *

* ================================================================== *

c *entry plist

c parm i5_LoadType

c parm i5_NullInd1

c parm sqlStateCode

c parm functionName

c parm specificName

c parm errorMessage

/Free

Monitor; // Global error handler

sqlStateCode = '00000';

If i5_NullInd1 = 0;

i5_ProdLoad = i5_LoadType;

Select;

When %xlate(upper: lower: %trim(i5_ProdLoad)) = '*sybase';

If syb_i5_LoadFromDelimitedFile(%trim(syB_Context):


Page 8 of 10 1/3/2011 4:49:44 PM

%trim(syB_dataDump):

%trim(pgmStatus.UserId) );

Else;


errorMessage = 'Error occured in external stored ' +

'procedure. Contact support and report ' +

'this error message.';

EndIf;

Other;


errorMessage = 'Invalid input load parameter ' +

%trim(i5_ProdLoad) + ' not processed.';

EndSL;

Else;


errorMessage = 'Null input load parameter not' +

' processed.';

EndIf;

On-Error *All; // Unmonitored Errors?

Dump(a);

EndMon;

*inlr = *On;

Return;

/End-Free

Benefits of Stored Procedure Processing versus ODBC/JDBC statements

• Secure access to DB2 data.

• Access non-DB2 data if necessary –VSAM, IMS, etc.

• Code business logic once

o Maintenance is simpler

• Performance improvement?

o Instead of multiple SQL statements being sent

• Lots of languages to choose from

Two Types of Stored Procedures

• SQL Stored Procedures

• External Procedures

Maintenance performed in one place


Page 9 of 10 1/3/2011 4:49:44 PM

Get Job Completion Status of Stored Procedure with a Select Statement

Select PROCESS,

PRCSEQ,

Cast(Case

When STATUS = 'N'

Then 'Success'

When STATUS = ‘A’

Then ‘Failure’

Else 'Running’

End As Character(7)) As Status,

STARTSTMP,

ENDSTMP,

RUNNAME,

RUNUSER,

RUNNUMBER,

ERRORID,

USRDATA

From ACSJOBLOG

Order by 4,1,2


Page 10 of 10 1/3/2011 4:49:44 PM

References

Writing Stored Procedures in RPG MC Press Online 15 January 2008, Susan Gantner

http://www.mcpressonline.com/programming/rpg/writing-stored-procedures-in-rpg.html

Stored Procedures, Triggers, and User-Defined Functions on DB2 UDB for iSeries

http://www.redbooks.ibm.com/abstracts/sg246503.html

DB2 UDB for AS/400 Object Relational Support


Cross-Platform DB2 Stored Procedures: Building and Debugging

http://www.redbooks.ibm.com/redbooks/pdfs/sg245485.pdf

DB2 9 for z/OS Stored Procedures: Through the Call and Beyond


DB2 Java Stored Procedures: Learning by Example

http://www.redbooks.ibm.com/redbooks/pdfs/sg245945.pdf

Modernizing IBM eServer iSeries Application Data Access – A Roadmap Cornerstone


Improving Stored Procedure Performance 18 December 2009, Kent Milligan, Jarek

Miszczyk, and Gene Cobb http://tinyurl.com/2ayskqo

The API Corner MC Press online: Automating Recovery (or Keeping the Help Desk Out

of the Loop) 21 July 2010 – October 2010, Bruce Vining http://tinyurl.com/32gkc9h

Documents

Db2 For I Parallel Data Load