Medical Research Database Tuning

Marina Balina

N16636038

Advanced Database Systems 2007

Medical Research Database Tuning

TABLE OF CONTENTS

INTRODUCTION ..............................................................................................3

DATABASE SYSTEM ........................................................................................3

USING COVERING INDEX................................................................................4

DIRECT PATH LOAD.....................................................................................10

USING EXISTS VS. IN .................................................................................14

ANALYZING LARGE QUERIES.......................................................................18

USING MATERIALIZED VIEWS AND OPTIMIZER HINTS ................................26

CONCLUSION................................................................................................33

APPENDIX A .............................................................................................34 GENE DATABASE DATA DICTIONARY ....................................................................................................... 34

APPENDIX B .............................................................................................36 TXP DATABASE DATA DICTIONARY........................................................................................................... 36 ViewTXPDonorReport ...................................................................................................................................... 41 ViewLab ............................................................................................................................................................. 48 FINAL VERSION OF TXP LAB TEST QUERY .............................................................................................. 49

APPENDIX C .............................................................................................54 PKD DATABASE DATA DICTIONARY .......................................................................................................... 54 PKD Entity Relationship Diagram................................................................................................................... 61 Materialized View SQL ..................................................................................................................................... 62 ViewEventQuestion ........................................................................................................................................... 69 ViewPatientInfo ................................................................................................................................................. 70 Original PKD Query Execution Plan .............................................................................................................. 71

Introduction

Our Rogosin Institute database server houses four database instances. Three of them

are clinical and basic research data repositories: • Polycystic Kidney Disease Patient Registry (PKD) • A repository for Affymetrix gene expression chip data for our cancer

research program (Gene DB) • A kidney transplant database (TXP)

The databases serve as a backend for our web-based clinical research system, which uses Macromedia ColdFusion Server, based on JRun4 as a middle tier. All the regular users, except for the application developers, interact with the database exclusively through the web interface.

The specific of our clinical research application is that most of its users are

researchers, who are impatient to see the results of their analysis. Furthermore, they like to rerun their data requests multiple times in a row with slightly changed parameters, in order to look at the data from one angle and then from another angle and so on. Needless to say, the query response time is only one aspect of the total time required before users can see the output on their screens. Thus every second is important. Therefore, the goal of my tuning project is to minimize the query response time as much as possible.

I will be analyzing five different tuning cases, three from the Gene database and one

from each, Transplant and PKD databases.

Database System

DBMS: Oracle Release 9.2.0.1.0 Manufacturer: IBM Model: x345 Operating System Red Hat Enterprise Linux ES release 3 (Taroon Update 9) RAM 4G Number of CPUs 4 Hard Drive Raid 1+0 Number of Database instances: 4 db_block_size 8192 (each db) db_buffer_size 32 (each db) select name,VALUE from v$sysstat where lower(name) like '%sort%'; NAME VALUE ---------------------------------------------------------------- ---------- sorts (memory) 335267 sorts (disk) 19 sorts (rows) 173338019

Using Covering Index

Sometimes it is as easy as adding a couple of covering indices. One of the queries in our Gene Database displays the results of gene chip data analysis. The query is not particularly fast, it runs more than a minute, followed by some time required to generate a nice graphical display. So I decided to see if there is anything I could do about the query.

The query looks as follows (see appendix A for the data dictionary).

SELECT a.probesetID, a.probeset, DECODE(InStr(a.genedescription,'/'), 0, a.genedescription, Substr(a.genedescription,1,InStr(a.genedescription,'/')-1)) AS geneTitle, a.geneDescription, a.genbankid, a.unigene ,d.chipTypeName, a.ontologyClasses , TO_CHAR(b503_1.signal) AS signal_1 , TO_CHAR(b504_2.signal) AS signal_2 , TO_CHAR(b505_3.signal) AS signal_3 , TO_CHAR(b500_4.signal) AS signal_4 , TO_CHAR(b501_5.signal) AS signal_5 , TO_CHAR(b502_6.signal) AS signal_6 , TO_CHAR(b497_7.signal) AS signal_7 , TO_CHAR(b498_8.signal) AS signal_8 , TO_CHAR(b499_9.signal) AS signal_9 , TO_CHAR(b494_10.signal) AS signal_10 , TO_CHAR(b495_11.signal) AS signal_11 , TO_CHAR(b496_12.signal) AS signal_12 , TO_CHAR(b509_13.signal) AS signal_13 , TO_CHAR(b510_14.signal) AS signal_14 , TO_CHAR(b511_15.signal) AS signal_15 , TO_CHAR(b506_16.signal) AS signal_16 , TO_CHAR(b507_17.signal) AS signal_17 , TO_CHAR(b508_18.signal) AS signal_18, TO_CHAR(c_0.pValue, '9.99EEEE') as pValue_0 FROM TBLPROBESET a, tblAnalysisResult c_0 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=819 ) AND groupNormalizationID= 89 ) b503_1 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=820 ) AND groupNormalizationID= 89 ) b504_2 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=821 ) AND groupNormalizationID= 89 ) b505_3 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=816 ) AND groupNormalizationID= 89 ) b500_4 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=817 ) AND groupNormalizationID= 89 ) b501_5 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=818 ) AND groupNormalizationID= 89 ) b502_6 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=813 ) AND groupNormalizationID= 89 ) b497_7 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=814 ) AND groupNormalizationID= 89 ) b498_8 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=815 ) AND groupNormalizationID= 89 ) b499_9 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=810 ) AND groupNormalizationID= 89 ) b494_10 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=811 ) AND groupNormalizationID= 89 ) b495_11 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=812 ) AND groupNormalizationID= 89 ) b496_12 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=825 ) AND groupNormalizationID= 89 ) b509_13 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=826 ) AND groupNormalizationID= 89 ) b510_14 ,

(SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=827 ) AND groupNormalizationID= 89 ) b511_15 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=822 ) AND groupNormalizationID= 89 ) b506_16 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=823 ) AND groupNormalizationID= 89 ) b507_17 , (SELECT probesetID, signal , detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=824 ) AND groupNormalizationID= 89 ) b508_18 ,tblChipType d WHERE a.probesetID = c_0.probesetID AND c_0.normalizationAnalysisID=831 AND a.probeSetID = b503_1.probeSetID AND a.probeSetID = b504_2.probeSetID AND a.probeSetID = b505_3.probeSetID AND a.probeSetID = b500_4.probeSetID AND a.probeSetID = b501_5.probeSetID AND a.probeSetID = b502_6.probeSetID AND a.probeSetID = b497_7.probeSetID AND a.probeSetID = b498_8.probeSetID AND a.probeSetID = b499_9.probeSetID AND a.probeSetID = b494_10.probeSetID AND a.probeSetID = b495_11.probeSetID AND a.probeSetID = b496_12.probeSetID AND a.probeSetID = b509_13.probeSetID AND a.probeSetID = b510_14.probeSetID AND a.probeSetID = b511_15.probeSetID AND a.probeSetID = b506_16.probeSetID AND a.probeSetID = b507_17.probeSetID AND a.probeSetID = b508_18.probeSetID AND d.chipTypeID = a.chipTypeID ORDER BY c_0.pValue

Apparently, the query joins several identical subqueries, varying only in the chipID parameter. (SELECT probesetID, signal, detection FROM TBLRESULTNormal WHERE (1=0 OR chipid=VALUE ) AND groupNormalizationID = 89 )

Here is the execution plan for that subquery. Execution Plan ---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=CHOOSE (Cost=3327 Card=21453 Bytes=407607) 1 0 TABLE ACCESS (FULL) OF 'TBLRESULTNORMAL' (Cost=3327 Card=21453 Bytes=407607)

Statistics ---------------------------------------------------------- 244 recursive calls 0 db block gets 38277 consistent gets 34582 physical reads 0 redo size 1142213 bytes sent via SQL*Net to client 40579 bytes received via SQL*Net from client 3646 SQL*Net roundtrips to/from client 6 sorts (memory) 0 sorts (disk)

54675 rows processed

I decided to see what would happen if we avoid accessing the table TblResultNormal completely by simply creating a covering index on GroupNormalizationID, ChipId, ProbesetID,

signal, detection).

Now the execution plan and the statistics on a single such subquery look much more promising. The number of physical reads decreased extremely. Execution Plan ---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=CHOOSE (Cost=86 Card=21260 Bytes= 403940) 1 0 INDEX (RANGE SCAN) OF 'NDXTBLRESULTN_COV' (NON-UNIQUE) (Cost=86 Card=21260 Bytes=403940) Statistics ---------------------------------------------------------- 696 recursive calls 0 db block gets 4006 consistent gets 233 physical reads 0 redo size 1142336 bytes sent via SQL*Net to client 40579 bytes received via SQL*Net from client 3646 SQL*Net roundtrips to/from client 14 sorts (memory) 0 sorts (disk) 54675 rows processed

Comparing the full execution plans of the original query and the one after adding the index, we can see that completely eliminating the access to the table TblResultNormal, we reduced the cost tremendously. Here is the execution plan for the whole query

ORIGINAL QUERY COST Rows Plan --------- --------------------------------------------------------- 18052 1 SELECT STATEMENT 18052 1 SORT ORDER BY 18050 1 NESTED LOOPS 18050 1 NESTED LOOPS 18050 1 NESTED LOOPS 18050 1 NESTED LOOPS 18050 1 NESTED LOOPS 18050 1 NESTED LOOPS 18050 1 NESTED LOOPS 18050 1 NESTED LOOPS 18050 1 NESTED LOOPS 18050 1 NESTED LOOPS 18050 1 NESTED LOOPS 18050 1 NESTED LOOPS 18050 1 NESTED LOOPS 18050 4 NESTED LOOPS 16207 19 HASH JOIN 12879 111 HASH JOIN 9548 638 HASH JOIN 6203 3837 HASH JOIN 327 35865 TABLE ACCESS BY INDEX ROWID TBLANALYSISRESULT 81 36 INDEX RANGE SCAN NDXTBLANALYSISRESULT_NORMANAL 5808 17008 HASH JOIN 2 7 TABLE ACCESS FULL TBLCHIPTYPE 5805 17008 HASH JOIN 3327 17008 TABLE ACCESS FULL TBLRESULTNORMAL 2199 159948 TABLE ACCESS FULL TBLPROBESET 3327 17395 TABLE ACCESS FULL TBLRESULTNORMAL

3327 18168 TABLE ACCESS FULL TBLRESULTNORMAL 3327 18361 TABLE ACCESS FULL TBLRESULTNORMAL 97 1 TABLE ACCESS BY INDEX ROWID TBLRESULTNORMAL 2 37 INDEX RANGE SCAN NDXTBLRESULTN_PROBESETID 1 TABLE ACCESS BY INDEX ROWID TBLRESULTNORMAL 2 37 INDEX RANGE SCAN NDXTBLRESULTN_PROBESETID 1 TABLE ACCESS BY INDEX ROWID TBLRESULTNORMAL 2 37 INDEX RANGE SCAN NDXTBLRESULTN_PROBESETID 1 TABLE ACCESS BY INDEX ROWID TBLRESULTNORMAL 2 37 INDEX RANGE SCAN NDXTBLRESULTN_PROBESETID 1 TABLE ACCESS BY INDEX ROWID TBLRESULTNORMAL 2 37 INDEX RANGE SCAN NDXTBLRESULTN_PROBESETID 1 TABLE ACCESS BY INDEX ROWID TBLRESULTNORMAL 2 37 INDEX RANGE SCAN NDXTBLRESULTN_PROBESETID 1 TABLE ACCESS BY INDEX ROWID TBLRESULTNORMAL 2 37 INDEX RANGE SCAN NDXTBLRESULTN_PROBESETID 1 TABLE ACCESS BY INDEX ROWID TBLRESULTNORMAL 2 37 INDEX RANGE SCAN NDXTBLRESULTN_PROBESETID 1 TABLE ACCESS BY INDEX ROWID TBLRESULTNORMAL 2 37 INDEX RANGE SCAN NDXTBLRESULTN_PROBESETID 1 TABLE ACCESS BY INDEX ROWID TBLRESULTNORMAL 2 37 INDEX RANGE SCAN NDXTBLRESULTN_PROBESETID 1 TABLE ACCESS BY INDEX ROWID TBLRESULTNORMAL 2 37 INDEX RANGE SCAN NDXTBLRESULTN_PROBESETID 1 TABLE ACCESS BY INDEX ROWID TBLRESULTNORMAL 2 37 INDEX RANGE SCAN NDXTBLRESULTN_PROBESETID 1 TABLE ACCESS BY INDEX ROWID TBLRESULTNORMAL 2 37 INDEX RANGE SCAN NDXTBLRESULTN_PROBESETID 1 TABLE ACCESS BY INDEX ROWID TBLRESULTNORMAL 2 37 INDEX RANGE SCAN NDXTBLRESULTN_PROBESETID

AFTER ADDING A COVERING INDEX ON TBLRESULTNORMAL

COST Rows Plan ---------- ---------- ------------------------------------------------------------- 2637 1 SELECT STATEMENT 2637 1 SORT ORDER BY 2635 1 NESTED LOOPS 2597 1 NESTED LOOPS 2596 1 NESTED LOOPS 2595 1 NESTED LOOPS 2594 1 NESTED LOOPS 2593 1 NESTED LOOPS 2592 1 NESTED LOOPS 2591 1 NESTED LOOPS 2590 1 NESTED LOOPS 2589 1 NESTED LOOPS 2588 1 NESTED LOOPS 2587 1 NESTED LOOPS 2586 1 NESTED LOOPS 2585 1 NESTED LOOPS 2582 3 NESTED LOOPS 2557 25 HASH JOIN 2553 213 HASH JOIN 2541 1861 HASH JOIN 2 17395 INDEX RANGE SCAN NDXTBLRESULTN_COV 2483 17008 HASH JOIN 2 7 TABLE ACCESS FULL TBLCHIPTYPE 2480 17008 HASH JOIN 2 17008 INDEX RANGE SCAN NDXTBLRESULTN_COV 2199 159948 TABLE ACCESS FULL TBLPROBESET 2 18168 INDEX RANGE SCAN NDXTBLRESULTN_COV 2 18361 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV

1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 38 1 TABLE ACCESS BY INDEX ROWID TBLANALYSISRESULT 2 36 INDEX RANGE SCAN NDXTBLANALYSISRESULT_PROBESETID

As the next step I decided to check what would happen if I create a covering index on TblAnalysisResult (NormalizationAnalysisID, probesetID, pvalue) to eliminate the full table access. Here I got some improvement, but not crucial.

AFTER ADDING A COVERING INDEX ON TBLANALYSISRESULT

COST Rows Plan ---------- ---------- ------------------------------------------------------------- 2600 1 SELECT STATEMENT 2600 1 SORT ORDER BY 2598 1 NESTED LOOPS 2597 1 NESTED LOOPS 2596 1 NESTED LOOPS 2595 1 NESTED LOOPS 2594 1 NESTED LOOPS 2593 1 NESTED LOOPS 2592 1 NESTED LOOPS 2591 1 NESTED LOOPS 2590 1 NESTED LOOPS 2589 1 NESTED LOOPS 2588 1 NESTED LOOPS 2587 1 NESTED LOOPS 2586 1 NESTED LOOPS 2585 1 NESTED LOOPS 2582 3 NESTED LOOPS 2557 25 HASH JOIN 2553 213 HASH JOIN 2541 1861 HASH JOIN 2 17395 INDEX RANGE SCAN NDXTBLRESULTN_COV 2483 17008 HASH JOIN 2 7 TABLE ACCESS FULL TBLCHIPTYPE 2480 17008 HASH JOIN 2 17008 INDEX RANGE SCAN NDXTBLRESULTN_COV 2199 159948 TABLE ACCESS FULL TBLPROBESET 2 18168 INDEX RANGE SCAN NDXTBLRESULTN_COV 2 18361 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV

1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLRESULTN_COV 1 1 INDEX RANGE SCAN NDXTBLANALYSISRESULT_COV

The results of adding each of the covering indices are below. Adding an index on the

table TblResultNormal was more impressive (82% response time decrease). Nevertheless, adding a covering index on the table TblAnalysisResult was still significant (22% decrease in response time).

Affect of Covering Index on Query

Performance

96.59

17.8713.98

0

20

40

60

80

100

120

Original Covering Index

on TblResultNormal

Covering Index

on TblAnalysisResult

Resp

on

se T

ime (

seco

nd

s)

Original Covering Index

on TblResultNormal

Covering Index

on TblAnalysisResult

Cost 16,425 1,030 992

Disk Reads Per Execution 382,064 319,406 161,510

CPU Time 95,910,000 17,620,000 10,880,000

Elapsed Time 96,587,503 17,865,467 13,975,581

Speaking about the trade-off between the index benefit and the maintenance cost, I

chose the benefit over the cost, since both tables update is done overnight, making it less painful. However, the second index might not be worth it.

Conclusion: use covering index to avoid accessing the underlying table, especially if the

index maintenance cost is low or could be taken care of during off hours. A covering

index gives performance equal to or even better than a clustering index as long as the

keys used in the where clause make up a leading portion. However, make sure the

performance improvement caused by a covering index is worth its maintenance cost.

Direct Path Load

Not all of the large update in the Gene database, however, are done off hours. The

gene chip data, representing gene expressions, is normally uploaded from a file during normal work hours. The procedure involves the following steps and is quite time consuming:

• Uploading the data from the chip file as is to an intermediate table, called TblResultIntermediate

• Deleting all the existing records pertaining to that particular chip from the permanents table, called TblResult

• Transferring the data from TblResultIntermediate to TblResult making necessary validations.

I will analyze the first two steps in this and the following cases (the last step has been

already tuned). The first step is a simple data load. The data is read from a file by a Java program and

inserted as a batch into the table using the Insert command into the table

TblResultIntermediate: Probeset VARCHAR2 (50) B-TREE INDEX Signal NUMBER (15,5) Detection VARCHAR2 (1) Here is the output of that first step of the entire procedure, which inserts 499080 records.

Started reading file: 2007-11-27 11:16:59.419 Started inserting data: 2007-11-27 11:17:13.262

Insert Temp Data query:

INSERT INTO tblResultIntermediate ( probeset, signal, detection ) VALUES ( ?, ?, ? )

Ended inserting data: 2007-11-27 11:21:53.637 Inserted records: 499080

Elapsed time: 00:04:54:218

Apparently the procedure is executed almost 5 minutes, which is not that great, taking into account that it is a simple procedure not requiring any data processing and is followed by two more steps. So why not use a Direct Path Load? All we need for that is a control file as the one below. The task is executed with the command below, C:\> sqlldr userid=riadmin/*******@rigenedb log=c:\chipupload.log

bad=chipupload.bad control=chipload.ctl direct=true

which produces the following log file. The elapsed time was a little over 1 minute.

Chipload.ctl

LOAD DATA INFILE ‘chip7841.txt’

INTO TABLE TBLRESULTINTERMEDIATE FIELDS TERMINATED BY ‘\t’ TRAILING NULLCOLS

( PROBESET CHAR NULLIF (PROBESET=BLANKS) , SIGNAL DECIMAL EXTERNAL NULLIF (SIGNAL=BLANKS)

, DETECTION CHAR NULLIF (DETECTION=BLANKS) )

Chipload.log

SQL*Loader: Release 9.2.0.1.0 - Production on Thu Nov 29 10:10:51 2007

Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.

Control File: chipload.ctl

Data File: chip7841.txt

Bad File: chipupload.bad

Discard File: none specified

(Allow all discards)

Number to load: ALL

Number to skip: 0 Errors allowed: 50

Continuation: none specified

Path used: Direct

Table TBLRESULTINTERMEDIATE, loaded from every logical record.

Insert option in effect for this table: INSERT

TRAILING NULLCOLS option in effect

Column Name Position Len Term Encl Datatype

------------------------------ ---------- ----- ---- ---- ---------------------

PROBESET FIRST * WHT CHARACTER NULL if PROBESET = BLANKS

SIGNAL NEXT * WHT CHARACTER

NULL if SIGNAL = BLANKS

DETECTION NEXT * WHT CHARACTER

NULL if DETECTION = BLANKS

The following index(es) on table TBLRESULTINTERMEDIATE were processed:

index NDXTBLRESINTER_PROBESET loaded successfully with 499080 keys

Table TBLRESULTINTERMEDIATE:

499080 Rows successfully loaded.

0 Rows not loaded due to data errors. 0 Rows not loaded because all WHEN clauses were failed.

0 Rows not loaded because all fields were null.

Bind array size not used in direct path.

Column array rows : 5000

Stream buffer bytes: 256000

Read buffer bytes: 1048576

Total logical records skipped: 0

Total logical records read: 499080

Total logical records rejected: 0 Total logical records discarded: 0

Total stream buffers loaded by SQL*Loader main thread: 111

Total stream buffers loaded by SQL*Loader load thread: 0

Run began on Thu Nov 29 10:10:51 2007

Run ended on Thu Nov 29 10:12:01 2007

Elapsed time was: 00:01:09.31

CPU time was: 00:00:00.91

The direct path load turned out to be 4 times faster than a batch load using INSERT as shown in the chart below. The effect would most likely be even more significant given a larger record size.

Batch load with INSERT vs.

Direct Path Load Performance

272.40

65.40

0

50

100

150

200

250

300

Batch Load Direct Path Load

Resp

on

se T

ime (

seco

nd

s)

Conclusion: use direct path load when possible to load data into not clustered table

because

• It does not execute any SQL INSERT statements; therefore, the processing

load on the Oracle database is reduced.

• It uses multiblock asynchronous I/O for writes to the database files.

• It does not use Oracle’s buffer cache, minimizing contention with other

Oracle users.

Using EXISTS vs. IN

During the second step of the chip data upload all the existing chip records are

deleted from the permanent table, TblResult, given the records correspond to the same chip type. The table TblResult has indexes on chipid and probesetid; the table TblProbeset has indexes on probesetid and chiptypeid (bitmap). See Appendix A for the data dictionary.

The SQL delete statement looks as follow: DELETE FROM tblresult WHERE chipid=779 AND probesetid IN (SELECT probesetid

FROM tblprobeset WHERE chiptypeid=6);

45101 rows deleted. Elapsed: 00:00:23.03 Statistics ---------------------------------------------------------- 104 recursive calls 79096 db block gets 108998 consistent gets 54500 physical reads 20796276 redo size 510 bytes sent via SQL*Net to client 622 bytes received via SQL*Net from client 3 SQL*Net roundtrips to/from client 1 sorts (memory) 7 sorts (disk) 45101 rows processed

Below is the explain plan for the query. Apparently, the subuqery is evaluated before the parent query. Why do a full index scan of the table TblProbeset if only a part of the records are needed? It looks like the index on chiptypeid in the TblProbeset is not used, because a big portion of the total records satisfy the criteria chiptypeid=6, so the index is not particularly selective.

It would be better if the query first evaluated the parent query, which is executed on the TblResult and then only accessed those records in TblProbeset where probesetID matches those in TblResult. In order to force the optimizer to process the parent query first I tried using “EXISTS” instead of “IN” as a subquery operator. The SQL Statement becomes:

DELETE FROM tblresult a WHERE chipid=779 AND EXISTS (SELECT 1

FROM tblprobeset b WHERE a.probesetid = b.probesetid AND b.chiptypeid=6);

45101 rows deleted. Elapsed: 00:00:08.02 Statistics ---------------------------------------------------------- 104 recursive calls 79090 db block gets 21200 consistent gets 29420 physical reads 20320428 redo size 510 bytes sent via SQL*Net to client 644 bytes received via SQL*Net from client 3 SQL*Net roundtrips to/from client 1 sorts (memory) 6 sorts (disk) 45101 rows processed

Graphs below illustrate the execution plans using IN (left) and EXISTS (right)

•

We can see that with EXISTS the table TblResult is evaluated first, resulting in 15 seconds response time reduction. It might not be a striking difference at that point, since the TblProbeset is not that huge. However, as new data comes and TblProbeset grows in size the gain will become much more significant.

IN vs Exists Performance

23.03

8.02

0

5

10

15

20

25

IN EXISTS

Resp

on

se T

ime (

seco

nd

s)

Conclusion:

• Use EXISTS if the most selective filter appears in the parent query. This allows

the selective predicates in the parent query to be applied before filtering the

rows against the EXISTS criteria.

• Conversely, use IN if the most selective filter is in the subquery and there are

indexes on the join columns.

Analyzing Large Queries

There is a query in our Transplant (TXP) database, which selects all patients transplanted within the given period and for each of them retrieves their lab test results for the given labs (here vital signs). The following patient lab results for each lab test are retrieved:

- Those taken at certain time points - The most recent lab test result before the transplantation - The most recent ones after the transplantation.

In addition, for each of the lab test and time point the query calculates the statistics - mean, median, standard deviation, total transplanted, total non-null values - over all the patients.

The query selects the data from 2 subqueries:

• Subquery A retrieves all available patient lab results for the provided time points and the most recent values before and after transplantation using UNION ALL operator

• Subquery B creates a grid with patients, queried lab tests and time points, which every selected patient is supposed to have.

The grid B is then outer joined to the available data A. See appendix B for the TXP data dictionary. The SQL below runs about 30 seconds, not terribly long, but I was sure it could be executed faster. SELECT DECODE(SIGN(nvl(vitalsignvalue,minnormal)-nvl(minnormal,vitalsignvalue)), -1, 'smallhead', DECODE(SIGN(nvl(maxnormal,vitalsignvalue)-nvl(vitalsignvalue,maxnormal)), -1, 'smallhead', '')) as abnormal, round(vitalsignvalue,2) as vitalsignvalue, b.vitalsignid, b.timeline, b.transplantid, b.transplantdate, round(STDDEV_SAMP (vitalsignvalue) over (partition by b.vitalsignid, b.timeline),2) as STANDARDDEVIATION , round(AVG (vitalsignvalue) over (partition by b.vitalsignid, b.timeline),2) as AVERAGE, round(PERCENTILE_DISC (0.5) WITHIN GROUP (ORDER BY vitalsignvalue) over (partition by b.vitalsignid, b.timeline),2) as MEDIAN, round(COUNT(vitalsignvalue) over (partition by b.vitalsignid, b.timeline)) as TOTALOBSERVED, round(COUNT(*) over (partition by b.vitalsignid, b.timeline)) as TOTALTRANSPLANTS, DECODE(b.vitalsignid, 11, 'Blood Pressure', 12, 'Blood Pressure', b.vitalsignname) as labname, b.maxnormal, b.minnormal ,b.fullname ,b.transplantdatedisplay ,b.transplantphysician FROM (SELECT a.vitalsignid,a.vitalsign,a.vitalsignname, TO_CHAR(TIMELINE) AS TIMELINE,c.transplantid, TO_CHAR(c.TRANSPLANTDATE, 'MM/DD/YYYY') as TRANSPLANTDATE, A.VITALSIGNVALUE, timeline as timelinesort FROM viewlab a, tbltransplant c, tbldonor d WHERE (1=0 OR a.vitalsignid=5 OR a.vitalsignid=7

OR a.vitalsignid=9 OR a.vitalsignid=10 OR a.vitalsignid=11 OR a.vitalsignid=12 OR a.vitalsignid=13 OR a.vitalsignid=15 OR a.vitalsignid=16 OR a.vitalsignid=17 OR a.vitalsignid=18 ) AND (1=0 OR a.timeline=0 OR a.timeline=30 OR a.timeline=90 OR a.timeline=180 OR a.timeline=360 OR a.timeline=0 ) AND c.donorid=d.donorid AND c.transplantDate >= TO_DATE('01/01/1995','mm/dd/yyyy') AND c.transplantDate <= TO_DATE('10/31/2007','mm/dd/yyyy') AND c.typeOfOrganID = 1 AND nvl(c.SPK,-9) = -9 AND a.TRANSPLANTID=C.TRANSPLANTID(+) AND d.donortypeid in (3,4,5) GROUP BY a.vitalsignid,a.vitalsign,a.vitalsignname,TO_CHAR(TIMELINE) ,a.timeline, c.transplantid, TO_CHAR(c.TRANSPLANTDATE, 'MM/DD/YYYY') , A.VITALSIGNVALUE , timeline UNION ALL select a.vitalsignid,a.vitalsign,a.vitalsignname,'LAST' AS TIMELINE,c.transplantid, TO_CHAR(c.TRANSPLANTDATE, 'MM/DD/YYYY') as TRANSPLANTDATE, A.VITALSIGNVALUE, -1000 as timelinesort from viewlab a, viewlab b, tbltransplant c, tbldonor d where (1=0 OR a.vitalsignid=5 OR a.vitalsignid=7 OR a.vitalsignid=9 OR a.vitalsignid=10 OR a.vitalsignid=11 OR a.vitalsignid=12 OR a.vitalsignid=13 OR a.vitalsignid=15 OR a.vitalsignid=16 OR a.vitalsignid=17 OR a.vitalsignid=18 ) and a.transplantid=b.transplantid AND a.vitalsignid=b.vitalsignid AND c.donorid=d.donorid AND c.transplantDate >= TO_DATE('01/01/1995','mm/dd/yyyy') AND c.transplantDate <= TO_DATE('10/31/2007','mm/dd/yyyy') AND c.typeOfOrganID = 1 AND nvl(c.SPK,-9) = -9 AND a.TRANSPLANTID=C.TRANSPLANTID(+) AND d.donortypeid in (3,4,5) group by a.vitalsignid,a.vitalsign,a.vitalsignname,'LAST' ,a.timeline, c.transplantid, TO_CHAR(c.TRANSPLANTDATE, 'MM/DD/YYYY') , A.VITALSIGNVALUE, -1000 HAVING A.TIMELINE=MAX(b.TIMELINE) UNION ALL

SELECT a.vitalsignid,a.vitalsign,a.vitalsignname,'BEFORE' AS TIMELINE,c.transplantid, TO_CHAR(c.TRANSPLANTDATE, 'MM/DD/YYYY') as TRANSPLANTDATE, ROUND(a.vitalsignvalue,2), -999 as timelinesort FROM viewlab a, viewlab b, tbltransplant c, tbldonor d WHERE (1=0 OR a.vitalsignid=5 OR a.vitalsignid=7 OR a.vitalsignid=9 OR a.vitalsignid=10 OR a.vitalsignid=11 OR a.vitalsignid=12 OR a.vitalsignid=13 OR a.vitalsignid=15 OR a.vitalsignid=16 OR a.vitalsignid=17 OR a.vitalsignid=18 ) and a.transplantid=b.transplantid AND a.vitalsignid=b.vitalsignid AND c.donorid=d.donorid AND c.transplantDate >= TO_DATE('01/01/1995','mm/dd/yyyy') AND c.transplantDate <= TO_DATE('10/31/2007','mm/dd/yyyy') AND c.typeOfOrganID = 1 AND nvl(c.SPK,-9) = -9 AND a.TRANSPLANTID=C.TRANSPLANTID(+) AND B.TIMELINE < 0 AND d.donortypeid in (3,4,5) GROUP BY a.vitalsignid,a.vitalsign,a.vitalsignname,'BEFORE' ,a.timeline, c.transplantid, TO_CHAR(c.TRANSPLANTDATE, 'MM/DD/YYYY'), A.VITALSIGNVALUE, -999

HAVING A.TIMELINE=MAX(b.TIMELINE)) A, (select a.vitalsignid,a.vitalsign,a.displayname as vitalsignname,C.timeline, b.transplantid, TO_CHAR(B.TRANSPLANTDATE, 'MM/DD/YYYY') as TRANSPLANTDATE,a.maxnormal, a.minnormal ,fullname,transplantdatedisplay,transplantphysician FROM TBLVITALSIGN a, ViewTXPDonorReport b , ( SELECT 'BEFORE' AS TIMELINE FROM DUAL UNION ALL SELECT '30' AS TIMELINE FROM DUAL UNION ALL SELECT '90' AS TIMELINE FROM DUAL UNION ALL SELECT '180' AS TIMELINE FROM DUAL UNION ALL SELECT '360' AS TIMELINE FROM DUAL UNION ALL SELECT 'LAST' AS TIMELINE FROM DUAL ) C WHERE (1=0 OR a.vitalsignid=5 OR a.vitalsignid=7 OR a.vitalsignid=9 OR a.vitalsignid=10

OR a.vitalsignid=11 OR a.vitalsignid=12 OR a.vitalsignid=13 OR a.vitalsignid=15 OR a.vitalsignid=16 OR a.vitalsignid=17 OR a.vitalsignid=18 ) AND transplantDate >= TO_DATE('01/01/1995','mm/dd/yyyy') AND transplantDate <= TO_DATE('10/31/2007','mm/dd/yyyy') AND typeOfOrganID = 1 AND upper(SPK) = DECODE(-9, 1, 'YES', 'NO') AND donortypeid in (3,4,5) GROUP BY a.vitalsignid,a.vitalsign,a.displayname, C.TIMELINE, b.transplantid, TO_CHAR(B.TRANSPLANTDATE, 'MM/DD/YYYY'),a.maxnormal, a.minnormal ,fullname,transplantdatedisplay,transplantphysician

) B WHERE B.TRANSPLANTID=A.TRANSPLANTID(+) AND B.VITALSIGNID=A.VITALSIGNID(+) AND B.TIMELINE=A.TIMELINE(+) ORDER BY DECODE(b.vitalsignid, 11, 'Blood Pressure', 12, 'Blood Pressure', b.vitalsignname), b.transplantDate, b.transplantid, fullname, b.timeline, b.vitalsignid, a.timelinesort;

So I started experimenting with it.

* Note, I used count(*) from the query while doing the experiments. Here is the statistics for count(*) from the original query COUNT(*) ---------- 48180

Statistics - ORIGINAL ---------------------------------------------------------- 5195 recursive calls 3110 db block gets 923101 consistent gets 15316 physical reads 448956 redo size 332 bytes sent via SQL*Net to client 4771 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 12 sorts (memory) 6 sorts (disk) 1 rows processed Elapsed: 00:00:19.05

1) First, I noticed a GROUP BY clause in the B subquery (strikethrough in the SQL text), which does not do anything and deleted it. This resulted in fewer physical reads and memory sorts. Statistics - DELETED GROUP BY ---------------------------------------------------------- 42 recursive calls 41 db block gets 922178 consistent gets 13923 physical reads 0 redo size 332 bytes sent via SQL*Net to client

4571 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 3 sorts (memory) 5 sorts (disk) 1 rows processed Elapsed: 00:00:18.09

2) The view VIEWTXPDONORREPORT joins 24 tables (appendix B), while all we need for this query is 4 tables: tbltransplant, tblpatient, tblphysician, tbldonor. Therefore, in order to remove the view I replaced the subquery B with the following (the changes are in green): (SELECT a.vitalsignid,a.vitalsign,a.displayname as vitalsignname,C.timeline, b.transplantid, TO_CHAR(B.TRANSPLANTDATE, 'MM/DD/YYYY') as transplantdate ,a.maxnormal, a.minnormal ,INITCAP(LOWER(e.lastname || decode(e.firstname, null, '', ', ' || e.firstname))) as fullname, TO_CHAR(b.transplantdate, 'mm/dd/yyyy') as transplantdatedisplay,d.transplantphysician FROM TBLVITALSIGN a, tbltransplant b, tblpatient e, tblphysician d, tbldonor f , ( SELECT 'BEFORE' AS TIMELINE FROM DUAL UNION ALL SELECT '30' AS TIMELINE FROM DUAL UNION ALL SELECT '90' AS TIMELINE FROM DUAL UNION ALL SELECT '180' AS TIMELINE FROM DUAL UNION ALL SELECT '360' AS TIMELINE FROM DUAL UNION ALL SELECT 'LAST' AS TIMELINE FROM DUAL ) C WHERE (1=0 OR a.vitalsignid=5 OR a.vitalsignid=7 OR a.vitalsignid=9 OR a.vitalsignid=10 OR a.vitalsignid=11 OR a.vitalsignid=12 OR a.vitalsignid=13 OR a.vitalsignid=15 OR a.vitalsignid=16 OR a.vitalsignid=17 OR a.vitalsignid=18 ) AND b.patientid=e.patientid AND b.donorid=f.donorid AND b.transplantphysicianid=d.transplantphysicianid (+) AND transplantDate >= TO_DATE('01/01/1995','mm/dd/yyyy') AND transplantDate <= TO_DATE('10/31/2007','mm/dd/yyyy') AND typeOfOrganID = 1 AND NVL(SPK,-9) = -9 AND NVL(e.testpatient, 0)=0 AND f.donortypeid in (3,4,5) ) B

Statistics - REPLACED VIEWTXPDONORREPORT

35 recursive calls 41 db block gets 831473 consistent gets 13927 physical reads 0 redo size 332 bytes sent via SQL*Net to client 4871 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 2 sorts (memory) 5 sorts (disk) 1 rows processed Elapsed: 00:00:18.01

This appeared to be not extremely helpful in terms of the response time, however resulted in less consistent gets. 3) Then I noticed that there is another query viewlab b (appendix B), which is used in the subquery A twice. This view joins 6 tables, while only 2 are needed, TblLinkTransplantVitalSign and TblTimeLine.

Therefore, I replaced viewlab b with TblLinkTransplantVitalSign b, TBLTIMELINE BB WHERE B.TIMELINEID=BB.TIMELINEID Statistics - REPLACED VIEWLAB B BY TBLLINKTRANSPLANTVITALSIGN AND TBLTIMELINE

105 recursive calls 41 db block gets 306193 consistent gets 9744 physical reads 0 redo size 332 bytes sent via SQL*Net to client 5041 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 2 sorts (memory) 5 sorts (disk) 1 rows processed Elapsed: 00:00:18.07

This resulted in consistent gets reduction by the factor of 2 and less physical reads. 4) Next, I noticed that there is a unique index on table TblLinkTransplantVitalSign (vitalSignID, transplantID, timelineID), therefore, I decided to replace it with a covering unique index on (TransplantID, VitalSignID, TimeLineID, vitalSignValue), which would eliminate the need to access the tblLinkTransplantVitalSign overall by adding one field to the existing index and rearranging the field order.

Statistics - REPLACED UNIQUE INDEX ON (VITALSIGNID, TRANSPLANTID, TIMELINEID) BY UNIQUE COVERING INDEX (TRANSPLANTID, VITALSIGNID, TIMELINEID, VALUE) ---------------------------------------------------------- 1055 recursive calls 397 db block gets 137842 consistent gets 10026 physical reads 53080 redo size 332 bytes sent via SQL*Net to client 5041 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 11 sorts (memory) 5 sorts (disk) 1 rows processed Elapsed: 00:00:08.00

Finally, we got a better response time. 5) Deleting the existing indices on TblLinkTransplantVitalSign (vitalsignid) and (timeLineID) got us even less physical reads and consistent gets, plus a slightly better response time. Statistics - REMOVED INDEXES ON TBLLINKTRANSPLANTVITALSIGN (VITALSIGNID) AND (TIMELINEID) ---------------------------------------------------------- 296 recursive calls 41 db block gets 77935 consistent gets 7039 physical reads 0 redo size 332 bytes sent via SQL*Net to client 5041 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 8 sorts (memory) 5 sorts (disk) 1 rows processed Elapsed: 00:00:06.09

As the final test with the query that selects all the necessary fields instead of simply counting them, I ran experiments in different order, first replacing the unique index with the covering one and then removing the group by clause and replacing the views with the base tables.

As a result of all these modifications, here is what we got.

See appendix B for the final version of the query.

Query and Index Tuning Affect on

Performance

29

23

10

0

5

10

15

20

25

30

35

Original Replaced Unique

Index with Covering

Removed Group By

and Views

Resp

on

se T

ime (

seco

nd

s)

Conclusions:

• Querying from a view requires all tables from the view to be accessed for the

data to be returned. Therefore, before using a view, determine whether all tables

in the view need to be accessed to return the data. If not, replace the view with

the necessary base tables only.

• Delete unnecessary group by clauses

• Remove non-selective indexes.

• Add columns to an index to improve selectivity or to cover the table in order to

eliminate the base table access.

• Consider reordering columns in existing concatenated indexes.

Using Materialized Views and Optimizer Hints

There are cases when nothing seems to help. An example is a query in our PKD database, which produces a report with metadata

(clinical questions asked at each time point) as the headers and patient data (answers to the clinical questions at each time point) as the rows (1 row per patient) given the protocol, list of events (event IDs) and list of time lines.

The query below, which is impossible to understand, selects patient data for 4

timelines runs for more than 5 minutes and produces 1,122,517 rows, joining 22 tables. It is important to mention that this query involves only 132 patients. The record set grows enormously with each additional patient and timeline.

The Unions in the query are needed in order to display different types of clinical

events in different ways. The view ViewEventQuestion is a metadata view, joining tables Project, Area, Event and Question. The view ViewPatientInfo is based on the table TblPatientInfo and TblRace joined by raceid. See appendix C for the PKD datadictionary and complete query execution plan.

SELECT a.*, g.schedulename, c.patientID, c.anonymousID, c.gender, To_char(c.birthDate, 'mm/dd/yyyy') as birthdate, DECODE(c.hispanicLatino,'H','Hispanic or Latino','N','Not Hispanic or Latino','Unknown') AS hispanicLatino, NVL(c.race, 'Unknown') AS race, To_char(e.visitDate, 'mm/dd/yyyy hh:mi:ss') AS firstTimeLineDate FROM ( SELECT a.areaName, a.eventName,a.questionText, a.dateDriven, a.categorySOrt, a.areaSort, a.eventSort, a.questionSort, a.reportTotalOccur, a.reportFirstOccur, a.reportLastOccur, a.reportCompilation, a.patientID, a.timeLineSort, a.optionSort, nvl(a.timeLineName,'') as timelinename, a.eventID, a.questionID, a.questionTypeID, a.questionType, a.optionID,a.optionText, a.isOptionOther, b.patientEventID, TO_CHAR(b.eventDate,'mm/dd/yyyy hh:mi:ss') AS eventDate, b.answerText, b.isOtherOption,b.yesno FROM (SELECT e.areaName, e.eventName,e.questionText, e.dateDriven, e.categorySOrt, e.areaSort, e.eventSort, e.questionSort, e.reportTotalOccur, e.reportFirstOccur, e.reportLastOccur, e.reportCompilation, b.patientID, d.timeLineID, d.timeLineName, d.sortBy as timeLineSort, e.eventID, e.questionID, e.questionTypeID, e.questionType, f.optionID, f.optionText, DECODE(e.questionTypeID,3,1, f.sortBy) AS optionSort, 0 AS isOptionOther FROM tblPatientInfo b, tblTimeLine d, VIEWEVENTQUESTION e, tblOption f WHERE b.projectID = 1 AND d.projectID = 1 AND d.sortBy >= 21 AND d.sortBy <= 67 AND (1= 0 OR e.eventID = 66) AND DECODE(e.questionTypeID,3,0,e.questionID) = f.questionID (+) UNION ALL SELECT e.areaName, e.eventName,e.questionText, e.dateDriven, e.categorySOrt, e.areaSort, e.eventSort, e.questionSort, e.reportTotalOccur, e.reportFirstOccur, e.reportLastOccur,

e.reportCompilation, b.patientID, d.timeLineID, d.timeLineName, d.sortBy as timeLineSort, e.eventID, e.questionID, 0 AS questionTypeID, 'Text' AS questionType, 0 AS optionID, 'Other' AS optionText, 1000 AS optionSort, 1 AS isOptionOther FROM tblPatientInfo b, tblTimeLine d, VIEWEVENTQUESTION e WHERE b.projectid = 1 AND d.projectid = 1 AND d.sortBy >= 21 AND d.sortBy <= 67 AND (1= 0 OR e.eventID = 66) AND NVL(e.isOptionOther,0) = 1 AND (e.questionTypeID = 3 OR e.questionTypeID = 4) ) a , (SELECT e.patientID, e.patientEventID, e.eventDate, e.timeLineID, q.answerText, q.questionID, q.answerText AS optionID, q.isOtherOption, e.yesno FROM tblLinkPatientEvent e, tblLinkPatientQuestion q WHERE e.patientEventID = q.patientEventID AND (1= 0 OR e.eventID = 66) ) b WHERE a.patientID = b.patientID (+) AND a.timeLineID = b.timeLineID(+) AND a.questionID = b.questionID (+) AND TRIM(TO_CHAR(DECODE(a.isOptionOther,1,1, DECODE(a.questionTypeID,4,a.optionID,0)))) = DECODE(a.isOptionOther,1,TRIM(TO_CHAR(b.isOtherOption(+))), DECODE(a.questionTypeID,4,b.optionID(+),'0')) UNION ALL SELECT a.areaName, a.eventName, a.questionText, a.dateDriven, a.categorySOrt, a.areaSort, a.eventSort, a.questionSort, a.reportTotalOccur, a.reportFirstOccur, a.reportLastOccur, a.reportCompilation, a.patientID, a.timeLineSort, a.optionSort, a.timeLineName, a.eventID, a.questionID, a.questionTypeID, a.questionType, a.optionID, a.optionText, a.isOptionOther, a.patientEventID, TO_CHAR(a.eventDate,'mm/ dd/yyyy hh:mi:ss') AS eventDate, a.answerText, a.isOtherOption, a.yesNo FROM ( SELECT a.areaName, a.eventName, a.questionText, a.dateDriven, a.categorySOrt, a.areaSort, a.eventSort, a.questionSort, a.reportTotalOccur, a.reportFirstOccur, a.reportLastOccur, a.reportCompilation, a.patientID, a.eventID, a.questionID, a.questionTypeID, a.questionType, a.optionID, a.optionText, a.optionSort, a.isOptionOther, b.patientEventID, b.eventDate, b.answerText, b.isOtherOption, b.yesNo, c.sortBy AS timeLineSort, c.timeLineName FROM ( SELECT e.areaName, e.eventName, e.questionText, e.dateDriven, e.categorySOrt, e.areaSort, e.eventSort, e.questionSort, e.reportTotalOccur, e.reportFirstOccur, e.reportLastOccur, e.reportCompilation, b.patientID, e.eventID, e.questionID, e.questionTypeID, e.questionType, 0 AS optionID, '' AS optionText, 0 AS optionSort, 0 AS isOptionOther FROM tblPatientinfo b, VIEWEVENTQUESTION e WHERE b.projectid = 1 AND (1= 0 OR e.eventID = 230) AND (e.questionTypeID = 1 OR e.questionTypeID = 2 OR e.questionTypeID = 5) UNION ALL SELECT e.areaName, e.eventName,e.questionText, e.dateDriven, e.categorySOrt, e.areaSort, e.eventSort, e.questionSort, e.reportTotalOccur, e.reportFirstOccur, e.reportLastOccur, e.reportCompilation, b.patientID, e.eventID, e.questionID, e.questionTypeID, e.questionType, f.optionID, f.optionText, f.sortBy AS optionSort, 0 AS isOptionOther FROM tblPatientinfo b, VIEWEVENTQUESTION e, tblYesNo f WHERE b.projectid = 1

AND (1= 0 OR e.eventID = 230) AND (e.questionTypeID = 6) UNION ALL SELECT e.areaName, e.eventName,e.questionText, e.dateDriven, e.categorySOrt, e.areaSort, e.eventSort, e.questionSort, e.reportTotalOccur, e.reportFirstOccur, e.reportLastOccur, e.reportCompilation, b.patientID, e.eventID, e.questionID, e.questionTypeID, e.questionType, f.optionID, f.optionText, f.sortBy AS optionSort, 0 AS isOptionOther FROM tblPatientinfo b, VIEWEVENTQUESTION e, tblNormalAbnormal f WHERE b.projectid = 1 AND (1= 0 OR e.eventID = 230) AND (e.questionTypeID = 8) UNION ALL SELECT e.areaName, e.eventName,e.questionText, e.dateDriven, e.categorySOrt, e.areaSort, e.eventSort, e.questionSort, e.reportTotalOccur, e.reportFirstOccur, e.reportLastOccur, e.reportCompilation, b.patientID, e.eventID, e.questionID, e.questionTypeID, e.questionType, f.statusID AS optionID, f.status AS optionText, f.sortBy AS optionSort, 0 AS isOptionOther FROM tblPatientinfo b, VIEWEVENTQUESTION e, tblStatus f WHERE b.projectid = 1 AND (1= 0 OR e.eventID = 230) AND (e.questionTypeID = 10) UNION ALL SELECT e.areaName, e.eventName,e.questionText, e.dateDriven, e.categorySOrt, e.areaSort, e.eventSort, e.questionSort, e.reportTotalOccur, e.reportFirstOccur, e.reportLastOccur, e.reportCompilation, b.patientID, e.eventID, e.questionID, e.questionTypeID, e.questionType, f.frequencyTypeID AS optionID, f.frequencyType AS optionText, f.sortBy AS optionSort, 0 AS isOptionOther FROM tblPatientinfo b, VIEWEVENTQUESTION e, viewFrequencyType f WHERE b.projectid = 1 AND (1= 0 OR e.eventID = 230) AND (e.questionTypeID = 7) AND e.frequencyGroupID = f.frequencyGroupID UNION ALL SELECT e.areaName, e.eventName,e.questionText, e.dateDriven, e.categorySOrt, e.areaSort, e.eventSort, e.questionSort, e.reportTotalOccur, e.reportFirstOccur, e.reportLastOccur, e.reportCompilation, b.patientID, e.eventID, e.questionID, e.questionTypeID, e.questionType, f.optionID, f.optionText, f.sortBy AS optionSort, 0 AS isOptionOther FROM tblPatientinfo b, VIEWEVENTQUESTION e, tblOption f WHERE b.projectid = 1 AND (1= 0 OR e.eventID = 230) AND (e.questionTypeID = 3 or e.questionTypeID = 4) AND e.questionID = f.questionID UNION ALL SELECT e.areaName, e.eventName,e.questionText, e.dateDriven, e.categorySOrt, e.areaSort, e.eventSort, e.questionSort, e.reportTotalOccur, e.reportFirstOccur, e.reportLastOccur, e.reportCompilation, b.patientID, e.eventID, e.questionID, 0 AS questionTypeID, 'Text' AS questionType, 0 AS optionID, 'Other' AS optionText, 1000 AS optionSort, 1 AS isOptionOther FROM tblPatientinfo b, VIEWEVENTQUESTION e WHERE b.projectid = 1 AND (1= 0 OR e.eventID = 230) AND NVL(e.isOptionOther,0) = 1 AND (e.questionTypeID = 3 OR e.questionTypeID = 4) UNION ALL SELECT e.areaName, e.eventName,'' AS questionText, e.dateDriven, e.categorySOrt, e.areaSort, e.eventSort, 0 AS questionSort, e.reportTotalOccur, e.reportFirstOccur, e.reportLastOccur, e.reportCompilation, b.patientID,

e.eventID, 0 AS questionID, 0 AS questionTypeID, '0' AS questionType, 0 AS optionID, '0' AS optionText, 0 AS optionSort, 0 AS isOptionOther FROM tblPatientinfo b, viewCategoryEvent e WHERE b.projectid = 1 AND (1= 0 OR e.eventID = 230) ) a ,(SELECT e.patientEventID, e.patientID, e.eventID, e.eventDate, q.answerText, q.questionID, q.answerText AS optionID, q.isOtherOption, e.yesNo FROM tblLinkPatientEvent e, tblLinkPatientQuestion q WHERE e.patientEventID = q.patientEventID AND NVL(e.yesNo,0) = 1 AND (1= 0 OR e.eventID = 230) UNION ALL SELECT e.patientEventID, e.patientID, e.eventID, e.eventDate, '0' AS answerText, 0 AS questionID, '0' AS optionID, '0' AS isOtherOption, e.yesNo FROM tblLinkPatientEvent e WHERE (1= 0 OR e.eventID = 230) ) b, tblTimeLine c WHERE a.patientID = b.patientID (+) AND DECODE(a.questionID,0,a.eventID,a.questionID) = DECODE(a.questionID, 0, b.eventID (+), b.questionID (+)) AND DECODE(questionTypeID,1,'1',2,'1',5,'1', TRIM(TO_CHAR(DECODE(a.isOptionOther,1,1,a.optionID)))) = DECODE(questionTypeID,1,'1',2,'1',5,'1', DECODE(a.isOptionOther,1,TRIM(TO_CHAR(b.isOtherOption(+))), b.optionID(+))) AND c.projectid = 1 AND c.sortBy >= 21 AND c.sortBy <= 67 ) a ) a, viewPatientInfo c, tblSchedule g, tblLinkPatientTimeLine e WHERE a.patientID = c.patientID AND (1 = 0 OR c.institutionID = 1) AND NVL(c.testPatient,0) = 0 AND NVL(c.ExcludeData,0) = 0 AND a.patientID = e.patientID (+) AND 129 = e.timeLineID (+) AND C.STARTDATE>=TO_DATE('01/01/2002', 'mm/dd/yyyy') AND C.STARTDATE<=TO_DATE('12/01/2007', 'mm/dd/yyyy') AND c.scheduleid = g.scheduleid ORDER BY c.anonymousID , a.timeLineSort, a.categorySOrt, a.areaSort, a.eventSort, TO_DATE(DECODE(a.dateDriven, 1, NULL, a.eventDate),'mm/dd/ yyyy hh:mi:ss'), DECODE(a.dateDriven, 1, NULL, a.patientEventID), a.questionSort, a.optionSort

It looks like a materialized view, which would be refreshed overnight, is the answer.

Besides, since the query is always run on each of the projects separately, there would be a separate materialized view for each of them.

After I created the materialized view (see the appendix C for the complete SQL

statement) and the bitmap indices on (TimeLineSort, timeLineID and eventID) I decided to experiment with selecting count(*) from the materialized view. SELECT count(*) FROM MVDATA WHERE (1=0 OR eventid = 336 … -- here goes the complete list of desired event IDs (usually most of the eventIDs are selected) ) AND (1 = 0

OR institutionid = 1) AND 129 = timelineid (+) AND startdate >= to_date('01/01/2002', 'mm/dd/yyyy') AND startdate <= to_date('12/01/2007', 'mm/dd/yyyy') AND TIMELINESORT>=21 AND TIMELINESORT<=67 ORDER BY <SAME THING>; COUNT(*) ---------- 1122517 1 row selected. Elapsed: 00:00:28.04 Execution Plan ---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=CHOOSE (Cost=13636 Card=1 Bytes=2 3) 1 0 SORT (AGGREGATE) 2 1 TABLE ACCESS (FULL) OF 'MVDATA' (Cost=13636 Card=249915 Bytes=5748045) Statistics ---------------------------------------------------------- 0 recursive calls 0 db block gets 141160 consistent gets 138686 physical reads 0 redo size 333 bytes sent via SQL*Net to client 8362 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 1 rows processed

The optimizer did the full table scan. I decided to see what happens if it uses the bitmap index on the timelinesort. So I ran the same query with the following hint and got a much better response time. /*+ INDEX(MVDATA,NDXMVDATA_TIMELINESORT_B) */ COUNT(*) ---------- 1122517 1 row selected. Elapsed: 00:00:03.03 Execution Plan ---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=CHOOSE (Cost=46834 Card=1 Bytes=23) 1 0 SORT (AGGREGATE) 2 1 TABLE ACCESS (BY INDEX ROWID) OF 'MVDATA' (Cost=46834 Card=249915 Bytes=5748045) 3 2 BITMAP CONVERSION (TO ROWIDS)

4 3 BITMAP AND 5 4 BITMAP INDEX (SINGLE VALUE) OF 'NDXMVDATA_TIMELINEID_B' 6 4 BITMAP MERGE 7 6 BITMAP INDEX (RANGE SCAN) OF 'NDXMVDATA_TIMELINESORT_B' 8 4 BITMAP OR 9 8 BITMAP INDEX (SINGLE VALUE) OF 'NDXMVDATA_EVENTID' …-- here comes the whole list of BITMAP INDEX (SINGLE VALUE) OF ‘NDXMVDATA_EVENTID' Statistics ---------------------------------------------------------- 0 recursive calls 0 db block gets 70498 consistent gets 70139 physical reads 0 redo size 333 bytes sent via SQL*Net to client 8409 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 1 rows processed

Apparently, if almost all of the events are selected, going through that index might not make sense. After using the hint /*+ INDEX(MVDATA,NDXMVDATA_TIMELINESORT_B) NO_INDEX(MVDATA,NDXMVDATA_EVENTID) */

I got the following COUNT(*) ---------- 1122517 1 row selected. Elapsed: 00:00:08.07 Execution Plan ---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=CHOOSE (Cost=47676 Card=1 Bytes=23) 1 0 SORT (AGGREGATE) 2 1 TABLE ACCESS (BY INDEX ROWID) OF 'MVDATA' (Cost=47676 Card=249915 Bytes=5748045) 3 2 BITMAP CONVERSION (TO ROWIDS) 4 3 BITMAP AND 5 4 BITMAP INDEX (SINGLE VALUE) OF 'NDXMVDATA_TIMELINEID_B' 6 4 BITMAP MERGE 7 6 BITMAP INDEX (RANGE SCAN) OF 'NDXMVDATA_TIMELINESORT_B' Statistics ---------------------------------------------------------- 0 recursive calls 0 db block gets 69167 consistent gets 69166 physical reads 0 redo size 333 bytes sent via SQL*Net to client 8444 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 1 rows processed

The response time increased by 5 seconds. Here is the comparison of regular query versus the materialized view performance that uses the bitmap indices on timelinesort, timelineid and eventid with 4 timelines selected.

Original Query Materialized View

Cost 4,014,701 136,349

Disk Reads 118,012 176,400

Buffer Gets 11,478,036 69,552

CPU Time 331,240,000 31,550,000

Elapsed Time 344,654,446 56,526,193

Query vs. Materialized View Performance

344.65

56.53

0

50

100

150

200

250

300

350

400

Original Query Materialized View

Resp

on

se T

ime (

seco

nd

s)

Conclusions:

• Consider using materialized views for complex queries with many joins

and large number of records.

• As an application designer, who has more information about a particular

applications’ data than is available to the optimizer, you can choose a

more effective way to execute a statement, using hints in SQL statements.

Conclusion

To summarize my tuning project experience I would like to point out several observations:

• First, I was very happy to find out that our database is pretty well tuned. There was nothing completely outrageous that would create a major disaster.

• Second, it is essential to know what kind of data you are dealing with. It is extremely

helpful to get some statistics on the data before doing any tuning. Even though the database optimizer is smart, the developer might possess some valuable information, not available to the optimizer and use it for tuning.

• I think it is also important to take care of the future. Even though some queries might

work well now, they might become a disaster later on when the data set grows. Smart design from the beginning can save a lot of time and effort in the future.

• Overall, it has been a rewarding experience to see how one little twick can change the

performance significantly in one or the other direction.

APPENDIX A

GENE DATABASE DATA DICTIONARY

(includes tables used in the tuning project)

TBLPROBESET

Number of rows 159948

Last Analyzed 3/10/2007

COLUMN DATA TYPE NULL? IND.TYPE CONSTRAINTS

PROBESETID NUMBER(10,0) N UNIQUE P

CHIPTYPEID NUMBER(10,0) Y BITMAP F_TBLCHIPTYPE

AFFYDESCRIPTION CLOB(4000) Y

CHROMOSOMALLOCATION CLOB(4000) Y

DATECREATED DATE(7) Y

DATEMODIFIED DATE(7) Y

EC CLOB(4000) Y

ENSEMBL CLOB(4000) Y

ENTREZGENE CLOB(4000) Y

GENBANKID VARCHAR2(26) Y

GENEDESCRIPTION CLOB(4000) Y

ISAFFYHK NUMBER(1,0) Y

MODIFIEDBYUSER VARCHAR2(10) Y

MODIFIEDBYUSERID NUMBER(10,0) Y

ONTOLOGYCLASSES CLOB(4000) Y

PATHWAY CLOB(4000) Y

PROBESET VARCHAR2(50) Y NORMAL

SORTBY NUMBER(5,0) Y

SWISSPROTID CLOB(4000) Y

UNIGENE VARCHAR2(100) Y

UNIGENEDESCRIPTION CLOB(4000) Y

UPPERDESCRIPTION CLOB(4000) Y

TBLRESULT




RESULTID NUMBER(15,0) N UNIQUE P

CHIPID NUMBER(10,0) N NORMAL F_TBLCHIP

PROBESETID NUMBER(10,0) N NORMAL

DATECREATED DATE(7) Y NORMAL


DETECTION VARCHAR2(1) Y

EXPERIMENTID NUMBER(10,0) N NORMAL F_TBLEXPERIMENT



PVALUE NUMBER(15,7) Y

SIGNAL NUMBER(15,5) Y NORMAL

TBLANALYSISRESULT






F NUMBER(30,20) Y

GROUPANALYSISID NUMBER(10,0) Y NORMAL F_TBLGROUPANALYSIS



NORMALIZATIONANALYSISID NUMBER(10,0) N NORMAL

PROBESETID NUMBER(10,0) N NORMAL F_TBLPROBESET

PVALUE NUMBER(25,20) Y NORMAL

TBLRESULTNORMAL




CHIPID NUMBER(10,0) N NORMAL F_TBLCHIP



DETECTION VARCHAR2(1) Y

GROUPID NUMBER(10,0) Y NORMAL F_TBLGROUP

GROUPNORMALIZATIONID NUMBER(10,0) N NORMAL F_TBLLINKGROUPNORMALIZATION



PROBESETID NUMBER(10,0) N NORMAL F_TBLPROBESET

PVALUE NUMBER(15,7) Y

SIGNAL NUMBER(15,5) Y NORMAL

TBLCHIPTYPE

Number of rows 7



CHIPTYPEDESC VARCHAR2(100) Y

CHIPTYPEID NUMBER(10,0) N UNIQUE TBLCHIPTYPE_P

CHIPTYPENAME VARCHAR2(50) N





VENDORID NUMBER(22) N BITMAP TBLCHIPTYPE_F_TBLCHIPVENDOR

APPENDIX B

TXP DATABASE DATA DICTIONARY


TBLDONOR

Number of rows 3072


COLUMN DATA TYPE NULL

?

IND.TYP

E CONSTRAINTS



DONORABOTYPE VARCHAR2(3) Y

DONORAGE NUMBER(3,0) Y

DONORCMV NUMBER(3,0) Y

DONORCREATININE NUMBER(10,3) Y

DONORCVA VARCHAR2(3) Y BITMAP

DONORDIASTOLIC NUMBER(10,0) Y

DONORFIRSTNAME VARCHAR2(50) Y

DONORHISPANICLATINO VARCHAR2(3) Y

DONORHOSPITAL VARCHAR2(30) Y

DONORHOSPITALID VARCHAR2(7) Y

DONORHYPERTENSION VARCHAR2(3) Y

DONORID NUMBER(7,0) N UNIQUE TBLDONOR_P_DONORID

DONORLASTNAME VARCHAR2(50) Y

DONORMATCHID VARCHAR2(10) Y

DONORRACEID NUMBER(2,0) Y BITMAP

DONORRECOVERYSITE NUMBER(1,0) Y

DONORSEX VARCHAR2(1) Y

DONORSSN VARCHAR2(9) Y

DONORSYSTOLIC NUMBER(10,0) Y

DONORTYPEID NUMBER(2,0) N BITMAP

DONORUNOSID VARCHAR2(10) Y



RELATIONSHIPID NUMBER(2,0) Y

TIEDIID VARCHAR2(7) Y

TBLLINKTRANSPLANTVITALSIGN



COLUMN DATA TYPE NULL?

IND.TYPE CONSTRAINTS





TRANSPLANTVITALSIGNID NUMBER(7,0) N UNIQUE TBLLINKTXPVITALTXPVITALID_P

TIMELINEID NUMBER(3,0) N

TBLLINKTXPVITALTXPVITALTIME_U, TBLLINKTXPVITAL_U

TRANSPLANTID NUMBER(7,0) N

TBLLINKTXPVITALTXPID_F_TBLTXP TBLLINKTXPVITALTXPVITALTIME_U, TBLLINKTXPVITAL_U

VITALSIGNID NUMBER(3,0) N

UNIQUE

TBLLINKTXPVITALSIID_F_TBLVITAL

TBLLINKTXPVITALTXPVITALTIME_U ,TBLLINKTXPVITAL_U

VITALSIGNVALUE NUMBER(15,5) Y TBLLINKTXPVITALTXPVITALTIME_U

TBLPATIENT Contains information on kidney and/or pancreas transplant recipients

Number of rows 2751




ABOTYPE VARCHAR2(3) Y

BIRTHDATE DATE(7) Y

CITY VARCHAR2(30) Y

COUNTY VARCHAR2(30) Y



DEATHCAUSESUBTYPEID NUMBER(3,0) Y NORMAL TBLPATIENT_F_TBLDEATHCAUSESUB

DEATHCAUSETYPEID NUMBER(3,0) Y TBLPATIENT_F_TBLDEATHCAUSETYP

E

DEATHDATE DATE(7) Y

FIRSTNAME VARCHAR2(50) Y NORMAL

HISPANICLATINO VARCHAR2(3) Y BITMAP

HOSPITALID VARCHAR2(7) Y

KIDSNUMBER NUMBER(2,0) Y

LASTFOLLOWUPDATE DATE(7) Y

LASTNAME VARCHAR2(50) Y NORMAL

MARITALSTATUS VARCHAR2(1) Y

MIQSID VARCHAR2(7) Y



OCCUPATION VARCHAR2(30) Y

PATIENTID NUMBER(10,0) N UNIQUE TBLPATIENT_P

PHYSICIANLASTFOLLOWUPDATE DATE(7) Y

RACEID NUMBER(2,0) Y NORMAL TBLPATIENT_F_TBLRACE

RECIPIENTSTATUSID NUMBER(10,0) Y BITMAP TBLPATIENT_F_TBLRECIPSTATUS

SEX VARCHAR2(1) Y

SSN VARCHAR2(9) Y UNIQUE TBLSSN_U

STATE VARCHAR2(2) Y

TESTPATIENT VARCHAR2(1) Y

ZIPCODE VARCHAR2(9) Y

TBLPHYSICIAN

Number of rows 22






ISSURGEON NUMBER(1,0) Y BITMAP



TRANSPLANTPHYSICIAN VARCHAR2(50) Y UNIQUE

TRANSPLANTPHYSICIANID NUMBER(10,0) N UNIQUE TBLPHYSICIAN_P_PHYSICIANID

UPIN VARCHAR2(6) Y UNIQUE

USERID NUMBER(5,0) Y

TBLRACE

Number of rows 10






LONGRACE VARCHAR2(50) Y



RACE VARCHAR2(50) N UNIQUE

RACEID NUMBER(2,0) N UNIQUE TBLRACE_P

UNOSRACEID NUMBER(20,0) Y

TBLTIMELINE

Number of rows 65



?

IND.TYP

E CONSTRAINTS






TIMELINE NUMBER(5,0) Y UNIQUE TBLTIMELINETIMELINE_U

TIMELINEID NUMBER(10,0) N UNIQUE TBLTIMELINE_P_TIMELINEID

TIMELINENAME VARCHAR2(20) N

TBLTRANSPLANT

Number of rows 3172



?

IND.TYP

E CONSTRAINTS

ADMISSIONDATE DATE(7) Y

ANGIOPLASTY VARCHAR2(2) Y

ANGIOPLASTYDATE DATE(7) Y

APLINDEX NUMBER(10,0) Y

BIOPSYPERFORMED VARCHAR2(3) Y

BIOPSYPROTOCOL VARCHAR2(2) Y

COMMENTS VARCHAR2(500) Y

CREATEDBYUSERID NUMBER(10,0) Y

CREATININEATDISCHARGE NUMBER(10,3) Y



DATEONLIST DATE(7) Y

DGF24 VARCHAR2(2) Y

DGF24REASON VARCHAR2(200) Y

DIALYSISUSED NUMBER(2,0) Y

DISCHARGEDATE DATE(7) Y

DONORARTERIES NUMBER(1,0) Y

DONORARTERIES2 NUMBER(1,0) Y

DONORID NUMBER(7,0) Y NORMAL TBLTRANSDONORID_F_TBLDONOR

DONORTYPEID NUMBER(2,0) Y BITMAP TBLTRANSPLANT_F_TBLDONORTYPE

DRAGMISMATCHED NUMBER(1,0) Y

ENBLOCK VARCHAR2(2) Y

ESRDDISEASEID NUMBER(3,0) Y NORMAL TBLTRANSPLANT_F_TBLESRDDISEAS

E

FAILUREDATE DATE(7) Y

FAILUREREASONID NUMBER(2,0) Y NORMAL TBLTRANSPL_F_TBLFAILUREREASON

GRAFTSTATUSID NUMBER(2,0) N BITMAP TBLTRANS_F_TBLGRAFT

IRPROCEDUREDATE DATE(7) Y

ISCHEMICTIME NUMBER(5,0) Y

ISCHEMICTIME2 NUMBER(5,0) Y

ISPEDIATRIC VARCHAR2(2) Y BITMAP

MEDICATIONPROTOCOLID NUMBER(5,0) Y BITMAP TBLTRANSMEDPROT_F_TBLMEDPROT



NONMEDICARE VARCHAR2(1) Y

OPPORTUNISTICINFECTIONFOUND VARCHAR2(3) Y BITMAP

ORGANSIDE VARCHAR2(1) Y

ORGANTYPECOUNT NUMBER(7,0) Y

OTHERTRANSPLANT NUMBER(1,0) Y

PANCREASCOUNT NUMBER(7,0) Y UNIQUE TBLTRANSPLANT_PANCREASCOUNT_

U

PATIENTID NUMBER(10,0) Y NORMAL TBLTRANSPL_F_TBLPATIENT

PAYBACK VARCHAR2(2) Y

POINTS NUMBER(6,4) Y

POSTDIABETES VARCHAR2(2) Y

POSTDIABETESDATE DATE(7) Y

POSTOPREJECTION VARCHAR2(2) Y

PREDIABETES VARCHAR2(3) Y

PREDIABETESTYPE VARCHAR2(3) Y

PROTOCOLID NUMBER(10,0) Y

RECIPIENTAGE NUMBER(2,0) Y BITMAP

RECIPIENTCMV NUMBER(3,0) Y

RECIPIENTUNOSID VARCHAR2(10) Y

REFERRINGCONTACT VARCHAR2(200) Y

REFERRINGDIALYSISUNIT NUMBER(4,0) Y

REFERRINGPHYSICIANID NUMBER(5,0) Y TBLTRANS_F_TBLREFERRPHYS

RELATIONSHIPID NUMBER(2,0) Y TBLTRANSPLAN_F_TBLRELATIONSHIP

SPK VARCHAR2(2) Y BITMAP

SPKCOUNT NUMBER(7,0) Y NORMAL

SPKTRANSPLANTID NUMBER(10,0) Y UNIQUE TBLTRANS_SPKTRANSID_U

STEROIDFREE VARCHAR2(1) Y

STEROIDFREEEXCLUSIONID NUMBER(5,0) Y

SURGERYIRDAYS NUMBER(4,0) Y

SURGERYIRREQUIRED NUMBER(2,0) Y

TIMEONDIALYSIS NUMBER(3,0) Y

TRANSPLANTCOUNT NUMBER(7,0) Y NORMAL

TRANSPLANTDATE DATE(7) Y NORMAL

TRANSPLANTID NUMBER(10,0) N UNIQUE TBLTRANSPLANT_P_TRNSPLANTID

TRANSPLANTNUMBER NUMBER(2,0) N

TRANSPLANTNUMBERNYH NUMBER(4,0) Y BITMAP

TRANSPLANTOPO VARCHAR2(30) Y

TRANSPLANTPHYSICIANID NUMBER(5,0) Y NORMAL TBLTRANSPLANT_F_TBLPHYSICIAN

TRANSPLANTSIDE VARCHAR2(1) Y

TRANSPLANTSURGEONID NUMBER(5,0) Y NORMAL

TYPEOFORGANID NUMBER(3,0) Y BITMAP

WARMISCHEMICTIME NUMBER(5,0) Y

WARMISCHEMICTIME2 NUMBER(5,0) Y

WEIGHT NUMBER(3,0) Y

TBLVITALSIGN

Number of rows 18






DISPLAYNAME VARCHAR2(50) Y

HIDE VARCHAR2(1) Y

MAXNORMAL NUMBER(10,2) Y

MAXVALUE NUMBER(10,5) Y

MINNORMAL NUMBER(10,2) Y

MINVALUE NUMBER(10,5) Y




UNITS VARCHAR2(10) Y

VITALSIGN VARCHAR2(20) N

VITALSIGNID NUMBER(10,0) N UNIQUE TBLVITALSIGN_P_TBLVITALSIGNID

ViewTXPDonorReport

SELECT TO_CHAR(a.admissiondate, 'mm/dd/yyyy') as admissiondate, DECODE(a.angioplasty, 1, 'YES', -9, 'NO', '') as angioplasty, DECODE (To_Char(a.angioplastydate, 'mm/dd/yyyy'), '01/01/1901', 'N/A', To_Char(a.angioplastydate, 'mm/dd/yyyy')) as angioplastydate, DECODE(a.biopsyperformed, 1, 'YES', -9, 'NO', '') as biopsyperformed, DECODE(a.biopsyprotocol, 1, 'YES', 'NO') AS biopsyprotocol, a.comments, a.creatinineatdischarge, DECODE(a.dialysisused, 1, 'YES', -9, 'NO', '') as dialysisused, DECODE(a.dialysisused, 1, 'YES', -9, 'NO', '') as dgf, to_char(a.dischargedate, 'mm/dd/yyyy') as dischargedate, TO_CHAR(a.dateonlist, 'mm/dd/yyyy') as dateonlist, DECODE(a.dgf24, 1, 'YES', -9, 'NO', '') as dgf24, a.dgf24reason, a.donorarteries, DECODE(NVL(ENBLOCK, -9), 1, TO_CHAR(a.donorarteries2), 'N/A') AS donorarteries2, TO_Number(a.dragmismatched) as dragmismatched, DECODE(NVL(a.enblock, -9), 1, 'YES', -9, 'NO') as enblock, a.esrddiseaseid, a.failurereasonid, DECODE(b.donortypeid, 1, 'N/A', 2, 'N/A', DECODE(a.ischemictime, -9, '0', a.ischemictime)) as ischemictime, DECODE(b.donortypeid, 1, 'N/A', 2, 'N/A', DECODE(NVL(ENBLOCK, -9), -9, 'N/A' , DECODE(a.ischemictime2, -9, '0', a.ischemictime2))) as ischemictime2, aa.lastfollowupdate, TO_CHAR(aa.lastfollowupdate, 'mm/dd/yyyy') as lastfollowupdatedisplay, (dischargedate-admissiondate) as lengthofstay, a.medicationProtocolID, DECODE (a.opportunisticinfectionfound, 1, 'YES', -9, 'NO', -99, 'Unknown', '') as opportunisticinfectionfound, DECODE(a.typeoforganid, 2, 'N/A', DECODE(a.organside, 'L', 'LEFT', 'R', 'RIGHT', 'B', 'BOTH', '')) AS organside, DECODE (a.payback, 1, 'YES', 'NO') as payback, a.points, DECODE (a.postoprejection, 1, 'YES', -9, 'NO', '') as postoprejection, DECODE (a.postoprejection, 1, DECODE(SIGN(182-(rej.rejectiondate-a.transplantdate)),-1, 'NO', 'YES'), -9, 'NO', '') as postoprejection6, DECODE(a.medicationprotocolid,1,to_char(rej.rejectiondate-a.transplantdate),2,to_char(rej.rejectiondate-a.transplantdate), 'N/A') as timetorejection, DECODE(a.prediabetes, 1, 'YES', -9, 'NO', '') as prediabetes, DECODE(a.postdiabetes, 1, 'YES', -9, 'NO', '') as postdiabetes, DECODE(a.prediabetes, -9, 'N/A', DECODE(a.prediabetestype, 1, 'I', 2, 'II', -99, 'Other/Unknown', -9, 'N/A', '')) as prediabetestype,

DECODE (To_Char(a.postdiabetesdate, 'mm/dd/yyyy'), '01/01/1901', 'N/A', To_Char(a.postdiabetesdate, 'mm/dd/yyyy')) as postdiabetesdate, a.protocolid, DECODE(a.recipientage, NULL, round(months_between(transplantdate, birthdate)/12), a.recipientage) as recipientage, DECODE(a.recipientcmv, 1, 'Pos', -1, 'Neg', -99, 'Unknown', '') as recipientcmv, UPPER(a.recipientunosid) AS recipientunosid, a.referringdialysisunit as dialysisunitid, a.referringphysicianid, DECODE(a.surgeryirrequired, 1, 'YES', -9, 'NO', '') as surgeryirrequired, DECODE (To_Char(a.irproceduredate, 'mm/dd/yyyy'), '01/01/1901', 'N/A', To_Char(a.irproceduredate, 'mm/dd/yyyy')) as irproceduredate, DECODE(a.timeondialysis, -9, 'NO', NULL, '', 'YES') as priordialysis, DECODE(a.timeondialysis, -9, '0', a.timeondialysis) as timeondialysis, a.transplantcount, a.pancreascount, a.spkcount, a.transplantdate, TO_CHAR(a.transplantdate, 'mm/dd/yyyy') as transplantdatedisplay, TO_CHAR(A.transplantDate, 'mm') AS TransplantMonthSTRING, TO_CHAR(A.transplantDate, 'yyyy') AS TransplantYearSTRING, a.transplantid, a.transplantnumber, a.transplantopo, a.transplantphysicianid, DECODE(a.typeoforganid, 2, 'N/A', DECODE(a.transplantside, 'L', 'LEFT', 'R', 'RIGHT', 'B', 'BILATERAL', '')) AS transplantside, a.transplantsurgeonid, a.graftstatusid, a.failuredate, DECODE(a.graftstatusid, 2, TO_CHAR(a.failuredate, 'mm/dd/yyyy'), 'N/A') as failuredatedisplay, DECODE(NVL(a.ispediatric, -9), 1, 'YES', -9, 'NO') as ispediatric, a.typeoforganid, DECODE(NVL(a.spk, -9), 1, 'YES', -9, 'NO') as spk, a.spktransplantid, DECODE(NVL(a.steroidFree, 0) , 1, 'YES', 0, 'NO') as steroidFree, a.steroidFreeExclusionID, round(MONTHS_BETWEEN(a.transplantdate, a.dateonlist), 2) as waitingtime, DECODE(b.donortypeid, 1, 'N/A', 2, 'N/A', DECODE(a.warmischemictime, -9, '0', a.warmischemictime)) as warmischemictime, DECODE(b.donortypeid, 1, 'N/A', 2, 'N/A', DECODE(NVL(ENBLOCK, -9), -9, 'N/A' , DECODE(a.warmischemictime2, -9, '0', a.warmischemictime2))) as warmischemictime2, a.weight, a.modifiedbyuserID, b.donorid, b.donorfirstname,

INITCAP(LOWER(b.donorlastname || decode(b.donorfirstname, null, '', ', ' || b.donorfirstname))) as donorfullname, b.donorlastname, b.donorssn, DECODE(b.donorcreatinine, -99, 'Unknown', b.donorcreatinine) as donorcreatinine, DECODE(b.donorhypertension, 1, 'YES', -9, 'NO', -99, 'Unknown', b.donorhypertension) as donorhypertension, DECODE(b.donortypeid, 1, 'N/A', 2, 'N/A', DECODE(b.donorcva, 1, 'YES', -9, 'NO', -99, 'Unknown', b.donorcva)) as donorcva, DECODE(b.donorsystolic, -99, 'Unknown', b.donorsystolic) as donorsystolic, DECODE(b.donordiastolic, -99, 'Unknown', b.donordiastolic) as donordiastolic, b.donorhospitalid, b.donorhospital, UPPER(b.donormatchid) AS donormatchid, UPPER(b.donorunosid) as donorunosid, DECODE(b.donorage, -99, '', b.donorage) as donorage, b.donorabotype, DECODE(b.donorcmv, 1, 'Pos', -1, 'Neg', -99, 'Unknown', '') as donorcmv, b.donorraceid, DECODE(b.donorhispaniclatino, 1, 'Hispanic/Latino', -9, 'Non Hispanic/Latino', -99, 'Unknown', '') as donorhispaniclatino, DECODE(b.donorrecoverysite,1, 'Local', 2, 'Imported') as donorrecoverysite, b.donorsex, b.donortypeid, b.relationshipid, UPPER(b.tiediid) as tiediid, aa.abotype as recipientabotype, aa.city as recipientcity, aa.county as recipientcounty, aa.state as recipientstate, aa.zipcode as recipientzipcode, aa.deathcausetypeid, aa.deathcausesubtypeid, aa.deathdate, DECODE(aa.recipientstatusid, 2, TO_CHAR(aa.deathdate, 'mm/dd/yyyy'), 'N/A') as deathdatedisplay, aa.firstname, DECODE(aa.hispaniclatino, 1, 'Hispanic/Latino', -9, 'Non Hispanic/Latino', -99, 'Unknown', '') as hispaniclatino, INITCAP(LOWER(aa.lastname || decode(aa.firstname, null, '', ', ' || aa.firstname))) as fullname, aa.lastname, aa.kidsnumber, DECODE(UPPER(aa.maritalstatus), 'M', 'Married', 'D', 'Divorced', 'S', 'Single', 'Unknown') as maritalstatus, aa.miqsID, aa.occupation,

aa.patientid, aa.physicianlastfollowupdate, INITCAP(LOWER(uu.race)) as recipientrace, aa.raceid, aa.recipientstatusid, aa.sex as recipientsex, aa.ssn, aa.testpatient, aa.hospitalid, aa.birthdate, TO_CHAR(aa.birthdate, 'mm/dd/yyyy') as birthdatedisplay, DECODE(ct.recipientB, -1, 'Neg', 1, 'Pos') as recipientB, DECODE(ct.recipientT, -1, 'Neg', 1, 'Pos') as recipientT, DECODE(ct.recipientPeakB, -1, 'Neg', 1, 'Pos') as recipientPeakB, DECODE(ct.recipientPeakT, -1, 'Neg', 1, 'Pos') as recipientPeakT, DECODE(ct.donorB, -1, 'Neg', 1, 'Pos') as donorB, DECODE(ct.donorT, -1, 'Neg', 1, 'Pos') as donorT, DECODE(ct.donorPeakB, -1, 'Neg', 1, 'Pos') as donorPeakB, DECODE(ct.donorPeakT, -1, 'Neg', 1, 'Pos') as donorPeakT, hla.txppra, hla.peakpra, hla.historicalpra, hla.recipienta1, hla.recipienta2, hla.recipientb1, hla.recipientb2, hla.recipientdr1, hla.recipientdr2, hla.donora1, hla.donora2, hla.donorb1, hla.donorb2, hla.donordr1, hla.donordr2, hla.collectiondate, INITCAP(LOWER(d.unitname)) as dialysisunit, DECODE(a.graftstatusid, 2, INITCAP(LOWER(f.failurereason)), 'N/A') as failurereason, h.typeoforgan, j.transplantphysician as transplantsurgeon, k.transplantphysician as transplantphysician, decode(a.graftstatusid, 1, round((months_between(sysdate,nvl(a.transplantdate,sysdate))),2), 2, round((months_between(a.failureDate, a.transplantDate)),2), 3, round((months_between(aa.lastfollowupdate, a.transplantDate)),2)) as graftsurvivaltime, decode(a.graftstatusid, 1, round(sysdate-nvl(a.transplantdate,sysdate)), 2, round(a.failureDate-a.transplantDate), 3, round(aa.lastfollowupdate-a.transplantDate)) as graftsurvivaltimedays, decode(aa.sex, 'F', 'Female', 'M', 'Male') as fullsex,

decode(b.donorsex, 'F', 'Female', 'M', 'Male') as fullDonorsex, decode(aa.recipientstatusid, 1, round(months_between(sysdate,nvl(a.transplantdate,sysdate)),2), 2, decode(trunc(months_between(sysdate, a.failuredate)/6), 0, round(months_between(sysdate, a.transplantDate),2), round(months_between(a.failuredate, a.transplantDate),2)+6), 3, round(months_between(aa.deathdate, a.transplantdate),2), 4, null) AS SurvivalTime, -------------------------------------------------------------------------------------- dragmismatch, abmismatch, drmismatch, ------------------------------------------------------------------------------------- decode(a.graftstatusid, 1, trunc((months_between(aa.lastfollowupdate, a.transplantDate)),0), 2, trunc((months_between(a.failureDate, a.transplantDate)),0), 3, trunc((months_between(a.failureDate, a.transplantDate)),0), 4, trunc((months_between(a.failureDate, a.transplantDate)),0), 5, trunc((months_between(a.failureDate, a.transplantDate)),0), 6, trunc((months_between(aa.lastfollowupdate, a.transplantDate)),0)) as graftstatustime, decode(aa.recipientstatusid, 1, trunc((months_between(sysdate, a.transplantDate)),0), 2, trunc((months_between(aa.DeathDate, a.transplantDate)),0), 3, trunc((months_between(sysdate, a.transplantDate)),0)) as recipientstatustime, --------------------------------------------------------------------------- DECODE(SIGN(TO_NUMBER(TO_CHAR(a.transplantdate, 'yyyy')) - 1974), -1, '1963-1973', DECODE(SIGN(TO_NUMBER(TO_CHAR(a.transplantdate, 'yyyy')) - 1984), -1, '1973-1983', DECODE(SIGN(TO_NUMBER(TO_CHAR(a.transplantdate, 'yyyy')) - 1994), -1, '1983-1993', DECODE(SIGN(TO_NUMBER(TO_CHAR(a.transplantdate, 'yyyy')) - 2004), -1, '1993-2003', '2004-' || to_char(sysdate, 'yyyy'))))) as transplantdecade, --------------------------------------------------------------------------- DECODE (DONORTYPE, 'NDD', DECODE(SIGN(DONORAGE-50), -1, 'NDD', DECODE(SIGN(59-DONORAGE), -1, 'EDD', DECODE(SIGN(DECODE(NVL(DONORCVA,0), 1, DONORCVA, 0) + DECODE(NVL(DONORHYPERTENSION,0), 1, DONORHYPERTENSION, 0)+ DECODE(SIGN(DECODE(DONORCREATININE, -99, NULL, DONORCREATININE)-1.5), 1, 1, 0)-1), 1, 'EDD', 'NDD'))), DONORTYPE) AS ASSIGNEDDONORTYPE, ------------------------------------------------------------------- INITCAP(LOWER(n.graftstatus)) as graftstatus, INITCAP(LOWER(m.recipientstatus)) as recipientstatus, o.protocolName AS medicationProtocolName, rej.rejectiondate as firstrejectiondate, DECODE (a.postoprejection, -9, 'N/A', 1, to_char(rej.rejectiondate, 'mm/dd/yyyy')) as firstrejectiondatedisplay, rej.totalrejections, rp.referringphysician,

INITCAP(LOWER(r.esrddisease)) as esrddisease, t.esrdtypeid, INITCAP(LOWER(t.esrdtype)) as esrdtype, s.relationship, INITCAP(LOWER(u.race)) as donorrace, INITCAP(LOWER(sf.exclusion)) as steroidfreeexclusion, v.donortype, INITCAP(LOWER(v.donortypedescription)) as donortypedescription, v.isliving, DECODE(v.isliving, 1, 'Living', 'Deceased') as LivingDeceased, DECODE(aa.recipientstatusid, 2, INITCAP(LOWER(dt.deathcausetype)), 'N/A') as deathcausetype, DECODE(aa.recipientstatusid, 2, INITCAP(LOWER(dc.deathcausesubtype)), 'N/A') as deathcausesubtype from tbltransplant a, tbldonor b, tbldialysisunit d, tblpatient aa, tblfailurereason f, tblorgantype h, tbltransplanthla hla, tblphysician j, tblphysician k, tblgraftstatus n, tblrecipientstatus m, tblMedicationprotocol o, tblesrddisease r, tblreferringphysician rp, tblesrdtype t, tblrelationship s, tblrace u, tblrace uu, tblsteroidfreeexclusion sf, tbldonortype v, tbldeathcausetype dt, tbldeathcausesubtype dc, tblcytometry ct, (SELECT transplantid, min(rejectiondate) as rejectiondate, count(*) as totalrejections from tbllinktransplantrejection group by transplantid) rej where aa.patientid = a.patientid (+) and a.donorid=b.donorid(+) and a.referringdialysisunit=d.unitid(+) and a.failurereasonid=f.failurereasonid(+) and a.transplantsurgeonid = j.transplantphysicianid(+) and a.transplantphysicianid = k.transplantphysicianid(+)

and a.graftstatusid = n.graftstatusid (+) and aa.recipientstatusid = m.recipientstatusid (+) and a.esrddiseaseid = r.esrddiseaseid (+) and r.esrdtypeid = t.esrdtypeid (+) and a.typeoforganid = h.typeoforganid (+) and b.donorraceid=u.raceid(+) and a.medicationProtocolID = o.ProtocolID (+) and aa.raceid=uu.raceid(+) and b.relationshipid = s.relationshipid (+) and b.donortypeid = v.donortypeid (+) and dc.deathcausetypeid=dt.deathcausetypeid(+) and aa.deathcausesubtypeid=dc.deathcausesubtypeid(+) and a.referringphysicianid=rp.referringphysicianid(+) and a.steroidfreeexclusionid=sf.exclusionid(+) and a.transplantid=hla.transplantid (+) and a.transplantid=rej.transplantid(+) and a.transplantid=ct.transplantid(+) and nvl(aa.testpatient, 0)=0

ViewLab

SELECT a.TRANSPLANTID, aa.PATIENTID, aa.LastName || ', ' || aa.FirstName as FULLNAME, a.TRANSPLANTDATE, aa.LASTFOLLOWUPDATE, a.TYPEOFORGANID, DECODE(nvl(a.spk, -9), 1, 'YES', 'NO') as spk, e.donortypeid, b.TIMELINEID, b.TIMELINE, b.TIMELINENAME, d.VITALSIGNID, d.DISPLAYNAME AS VITALSIGNNAME, d.VITALSIGN, d.minnormal, d.maxnormal, C.VITALSIGNVALUE, (a.transplantdate+b.timeline) as labdate, TO_CHAR(a.transplantdate+b.timeline, 'mm/dd/yyyy') as labdatedisplay FROM

tblTransplant a, tblPatient aa, tblTimeLine b, tblLinkTransplantVitalSign c, tblVitalSign d, tblDonor e WHERE aa.patientid=a.patientid AND a.TRANSPLANTID=c.TRANSPLANTID AND a.donorid = e.donorid AND b.timelineid=c.timelineid AND d.vitalsignid=c.vitalsignid

FINAL VERSION OF TXP LAB TEST QUERY

SELECT count(*) FROM (SELECT DECODE(SIGN(nvl(vitalsignvalue,minnormal)-nvl(minnormal,vitalsignvalue)), -1, 'smallhead', DECODE(SIGN(nvl(maxnormal,vitalsignvalue)-nvl(vitalsignvalue,maxnormal)), -1, 'smallhead', '')) as abnormal, round(vitalsignvalue,2) as vitalsignvalue, b.vitalsignid, b.timeline, b.transplantid, b.transplantdate, round(STDDEV_SAMP (vitalsignvalue) over (partition by b.vitalsignid, b.timeline),2) as STANDARDDEVIATION , round(AVG (vitalsignvalue) over (partition by b.vitalsignid, b.timeline),2) as AVERAGE, round(PERCENTILE_DISC (0.5) WITHIN GROUP (ORDER BY vitalsignvalue) over (partition by b.vitalsignid, b.timeline),2) as MEDIAN, round(COUNT(vitalsignvalue) over (partition by b.vitalsignid, b.timeline)) as TOTALOBSERVED, round(COUNT(*) over (partition by b.vitalsignid, b.timeline)) as TOTALTRANSPLANTS, DECODE(b.vitalsignid, 11, 'Blood Pressure', 12, 'Blood Pressure', b.vitalsignname) as labname, b.maxnormal, b.minnormal ,b.fullname ,b.transplantdatedisplay ,b.transplantphysician FROM (SELECT a.vitalsignid,a.vitalsign,a.vitalsignname, TO_CHAR(TIMELINE) AS TIMELINE,c.transplantid, TO_CHAR(c.TRANSPLANTDATE, 'MM/DD/YYYY') as TRANSPLANTDATE, A.VITALSIGNVALUE, timeline as timelinesort FROM viewlab a, tbltransplant c, tbldonor d WHERE (1=0 OR a.vitalsignid=5 OR a.vitalsignid=7 OR a.vitalsignid=9 OR a.vitalsignid=10 OR a.vitalsignid=11 OR a.vitalsignid=12 OR a.vitalsignid=13 OR a.vitalsignid=15 OR a.vitalsignid=16 OR a.vitalsignid=17 OR a.vitalsignid=18 ) AND (1=0 OR a.timeline=0 OR a.timeline=30 OR a.timeline=90 OR a.timeline=180 OR a.timeline=360

OR a.timeline=0 ) AND c.donorid=d.donorid AND c.transplantDate >= TO_DATE('01/01/1995','mm/dd/yyyy') AND c.transplantDate <= TO_DATE('10/31/2007','mm/dd/yyyy') AND c.typeOfOrganID = 1 AND nvl(c.SPK,-9) = -9 AND a.TRANSPLANTID=C.TRANSPLANTID(+) AND d.donortypeid in (3,4,5) GROUP BY a.vitalsignid,a.vitalsign,a.vitalsignname,TO_CHAR(TIMELINE) ,a.timeline, c.transplantid, TO_CHAR(c.TRANSPLANTDATE, 'MM/DD/YYYY') , A.VITALSIGNVALUE , timeline UNION ALL SELECT a.vitalsignid,a.vitalsign,a.vitalsignname,'LAST' AS TIMELINE,c.transplantid, TO_CHAR(c.TRANSPLANTDATE, 'MM/DD/YYYY') as TRANSPLANTDATE, A.VITALSIGNVALUE, -1000 as timelinesort FROM viewlab a, TBLLINKTRANSPLANTVITALSIGN b, TBLTIMELINE BB, tbltransplant c, tbldonor d WHERE (1=0 OR a.vitalsignid=5 OR a.vitalsignid=7 OR a.vitalsignid=9 OR a.vitalsignid=10 OR a.vitalsignid=11 OR a.vitalsignid=12 OR a.vitalsignid=13 OR a.vitalsignid=15 OR a.vitalsignid=16 OR a.vitalsignid=17 OR a.vitalsignid=18 ) AND a.transplantid=b.transplantid AND B.TIMELINEID=BB.TIMELINEID AND a.vitalsignid=b.vitalsignid AND c.donorid=d.donorid AND c.transplantDate >= TO_DATE('01/01/1995','mm/dd/yyyy') AND c.transplantDate <= TO_DATE('10/31/2007','mm/dd/yyyy') AND c.typeOfOrganID = 1 AND nvl(c.SPK,-9) = -9 AND a.TRANSPLANTID=C.TRANSPLANTID(+)

AND d.donortypeid in (3,4,5) GROUP BY a.vitalsignid,a.vitalsign,a.vitalsignname,'LAST' ,a.timeline, c.transplantid, TO_CHAR(c.TRANSPLANTDATE, 'MM/DD/YYYY') , A.VITALSIGNVALUE, -1000 HAVING A.TIMELINE=MAX(BB.TIMELINE) UNION ALL SELECT a.vitalsignid,a.vitalsign,a.vitalsignname,'BEFORE' AS TIMELINE,c.transplantid, TO_CHAR(c.TRANSPLANTDATE, 'MM/DD/YYYY') as TRANSPLANTDATE, ROUND(a.vitalsignvalue,2), -999 as timelinesort FROM viewlab a, TBLLINKTRANSPLANTVITALSIGN b, TBLTIMELINE BB, tbltransplant c, tbldonor d WHERE (1=0 OR a.vitalsignid=5 OR a.vitalsignid=7 OR a.vitalsignid=9 OR a.vitalsignid=10 OR a.vitalsignid=11 OR a.vitalsignid=12 OR a.vitalsignid=13 OR a.vitalsignid=15 OR a.vitalsignid=16 OR a.vitalsignid=17 OR a.vitalsignid=18 ) AND a.transplantid=b.transplantid AND B.TIMELINEID=BB.TIMELINEID AND a.vitalsignid=b.vitalsignid AND c.donorid=d.donorid AND c.transplantDate >= TO_DATE('01/01/1995','mm/dd/yyyy') AND c.transplantDate <= TO_DATE('10/31/2007','mm/dd/yyyy') AND c.typeOfOrganID = 1 AND nvl(c.SPK,-9) = -9 AND a.TRANSPLANTID=C.TRANSPLANTID(+) AND BB.TIMELINE < 0 AND d.donortypeid in (3,4,5) GROUP BY a.vitalsignid,a.vitalsign,a.vitalsignname,'BEFORE' ,a.timeline, c.transplantid, TO_CHAR(c.TRANSPLANTDATE, 'MM/DD/YYYY'), A.VITALSIGNVALUE, -999

HAVING A.TIMELINE=MAX(BB.TIMELINE)) A, (SELECT a.vitalsignid,a.vitalsign,a.displayname as vitalsignname,C.timeline, b.transplantid, TO_CHAR(B.TRANSPLANTDATE, 'MM/DD/YYYY') as TRANSPLANTDATE,a.maxnormal, a.minnormal ,INITCAP(LOWER(e.lastname || decode(e.firstname, null, '', ', ' || e.firstname))) as fullname, TO_CHAR(b.transplantdate, 'mm/dd/yyyy') as transplantdatedisplay,d.transplantphysician FROM TBLVITALSIGN a, tbltransplant b, tblpatient e, tblphysician d, tbldonor f , ( SELECT 'BEFORE' AS TIMELINE FROM DUAL UNION ALL SELECT '30' AS TIMELINE FROM DUAL UNION ALL SELECT '90' AS TIMELINE FROM DUAL UNION ALL SELECT '180' AS TIMELINE FROM DUAL UNION ALL SELECT '360' AS TIMELINE FROM DUAL UNION ALL SELECT 'LAST' AS TIMELINE FROM DUAL ) C WHERE (1=0 OR a.vitalsignid=5 OR a.vitalsignid=7 OR a.vitalsignid=9 OR a.vitalsignid=10 OR a.vitalsignid=11 OR a.vitalsignid=12 OR a.vitalsignid=13 OR a.vitalsignid=15 OR a.vitalsignid=16 OR a.vitalsignid=17 OR a.vitalsignid=18 ) AND b.patientid=e.patientid AND b.donorid=f.donorid AND b.transplantphysicianid=d.transplantphysicianid (+) AND transplantDate >= TO_DATE('01/01/1995','mm/dd/yyyy') AND transplantDate <= TO_DATE('10/31/2007','mm/dd/yyyy') AND typeOfOrganID = 1 AND NVL(SPK,-9) = -9 AND NVL(e.testpatient, 0)=0 AND f.donortypeid in (3,4,5)

) B WHERE B.TRANSPLANTID=A.TRANSPLANTID(+) AND B.VITALSIGNID=A.VITALSIGNID(+) AND B.TIMELINE=A.TIMELINE(+) ORDER BY DECODE(b.vitalsignid, 11, 'Blood Pressure', 12, 'Blood Pressure', b.vitalsignname), b.transplantDate, b.transplantid, fullname, b.timeline, b.vitalsignid, a.timelinesort);

APPENDIX C

PKD DATABASE DATA DICTIONARY


TBLAREA AREA LOOKUP TABLE

Number of rows 56



AREAID NUMBER(5,0) N UNIQUE TBLAREA_P

AREANAME VARCHAR2(150) Y

AUTOMATCH VARCHAR2(1) Y


DATEDRIVEN VARCHAR2(1) Y BITMAP




PROJECTID NUMBER(10,0) Y NORMAL TBLAREA_F_TBLPROJECT

SORTBY NUMBER(5,0) Y NORMAL

UMLSCONFIRM NUMBER(2,0) Y

UMLSCUI VARCHAR2(10) Y

TBLCATEGORY CATEGORIES LOOKUP TABLE

Number of rows 14




CATEGORYDESCRIPTION VARCHAR2(100) Y

CATEGORYID NUMBER(5,0) N UNIQUE TBLCATEGORY_P

CATEGORYNAME VARCHAR2(25) N


DATEDRIVEN VARCHAR2(1) Y

DATELABEL VARCHAR2(20) Y


ISMAJOR VARCHAR2(1) Y BITMAP



PROJECTID NUMBER(10,0) Y NORMAL TBL_CAT_F_TBLPROJECT

SORTBY NUMBER(5,0) N NORMAL



TBLEVENT EVENTS LOOKUP TABLE

Number of rows 276



ADVERSEEVENTID NUMBER(10,0) Y TBLEVENT_F_TBLADVERSEEVENTID

AREAID NUMBER(5,0) N NORMAL TBLEVENT_F_TBLAREA



DATELABEL VARCHAR2(50) Y


EVENTID NUMBER(5,0) N UNIQUE TBLEVENT_P

EVENTNAME VARCHAR2(150) Y

EVENTTYPEID NUMBER(3,0) Y

ISADVERSEEVENT VARCHAR2(1) Y

ISMULTIPLE VARCHAR2(1) Y BITMAP



REPORTCOMPILATION VARCHAR2(1) Y

REPORTFIRSTOCCUR VARCHAR2(1) Y

REPORTLASTOCCUR VARCHAR2(1) Y

REPORTTOTALOCCUR VARCHAR2(1) Y

SCHEMATYPE VARCHAR2(1) Y BITMAP




YESNOTYPE VARCHAR2(1) Y BITMAP

TBLEVENTTYPE EVENT TYPE LOOKUP TABLE (ex. procedure, syndrome)

Number of rows 3





EVENTTYPE VARCHAR2(50) N

EVENTTYPEDESCRIPTION VARCHAR2(500) Y

EVENTTYPEID NUMBER(2,0) N UNIQUE TBLEVENTYPE_P

EVENTTYPEVERB VARCHAR2(20) Y



TBLFREQUENCYGROUP FREQUENCY GROUP LOOKUP TABLE

Number of rows 2





FREQUENCYGROUP VARCHAR2(50) N

FREQUENCYGROUPID NUMBER(2,0) N UNIQUE TBLFREQUENCYGROUP_P



TBLFREQUENCYTYPE LOOKUP TABLE FOR FREQUENCY TYPE WHICH EXISTS IN A FREQUENCY GROUP

Number of rows 10






FREQUENCYGROUPID NUMBER(2,0) N NORMAL TBLFREQTYPE_F_TBLFREQGRP

FREQUENCYTYPE VARCHAR2(50) Y

FREQUENCYTYPEID NUMBER(5,0) N UNIQUE TBLFREQUENCYTYPE_P




UMLSCONFIRM VARCHAR2(10) Y


TBLLINKCATEGORYAREA LINK BETWEEN CATEGORY AND AREA

Number of rows 55



AREAID NUMBER(5,0) N

TBLLINKCATEGORYAREA_P ,TBLLINKCAT_F_TBLAREA

CATEGORYID NUMBER(5,0) N UNIQUE TBLLINKCATEGORYAREA_P, TBLLINKCAT_F_TBLCATEGORY





TBLLINKPATIENTEVENT STORES EVENTS OCCURED OR NOT OCCURED TO A PATIENT






EVENTDATE DATE(7) Y NORMAL

EVENTID NUMBER(10,0) N NORMAL TBLLINKPATIENT_F_TBLEVENT

MODIFICATIONID NUMBER(10,0) Y



PATIENTEVENTID NUMBER(7,0) N UNIQUE TBLLINKPATIENTEVENT_P

PATIENTID NUMBER(10,0) N NORMAL TBLLINKPATIENT_F_TBLPATIENT

SCHEMAID NUMBER(10,0) Y NORMAL TBLLINKPATIENTEVENT_F_TBLSCHEM

TIMELINEID NUMBER(10,0) Y NORMAL TBLLINKPATIENT_F_TBLTIMELINE

VISITDATE DATE(7) N

YESNO NUMBER(2,0) Y BITMAP

TBLLINKPATIENTQUESTION STORES PATIENTS ANSWERS TO EVENT QUESTIONS




ACCESSIONNUMBER NUMBER(20,0) Y NORMAL

ANSWERTEXT VARCHAR2(1000) N



ISOTHEROPTION VARCHAR2(1) Y NORMAL




PATIENTEVENTID NUMBER(10,0) Y NORMAL TBLLINKPATIENTQUES_F_LINKPATEV

POINTID NUMBER(10,0) Y NORMAL TBLLINKPATQUES_F_TBLPOINT

QUESTIONID NUMBER(5,0) N NORMAL TBLLINKPATIENTQUES_F_TBLQUES

TBLLINKPATIENTTIMELINE STORES PATIENT VISIT DATE FOR EACH TIMELINE

Number of rows 447








PATIENTID NUMBER(10,0) N UNIQUE

TBLLINKPTTIMELINE_F_TBLPATIENT TBLLINKPTTIMELINE_U

SCHEDULEID NUMBER(5,0) Y TBLLINKPTTIMELINE_F_TBLSCHEDUL

SCHEMAID NUMBER(10,0) Y TBLLINKPTTIMELINE_F_TBLSCHEMA

TIMELINEID NUMBER(10,0) N NORMAL

TBLLINKPTTIMELINE_F_TBLTIMELIN TBLLINKPTTIMELINE_U

VISITDATE DATE(7) Y

TBLPATIENTINFO PATIENT DEMOGRAPHIC INFORMATION

Number of rows 178



ANONYMOUSID VARCHAR2(20) N NORMAL

BIRTHDATE DATE(7) Y NORMAL



DELETED VARCHAR2(1) Y BITMAP

ENDDATE DATE(7) Y

ENROLLMENTNUMBER NUMBER(10,0) Y

EXCLUDEDATA VARCHAR2(1) Y

GENDER VARCHAR2(1) Y BITMAP

HISPANICLATINO VARCHAR2(1) Y

INSTITUTIONID NUMBER(5,0) Y

ISDROPPED VARCHAR2(1) Y

MARITALSTATUS VARCHAR2(1) Y



PATIENTID NUMBER(7,0) N UNIQUE TBLPATIENTINFO_P

PATIENTSUMMARY VARCHAR2(1000) Y

PROJECTID NUMBER(10,0) Y NORMAL TBLPATENT_F_TBLPROJECT

RACEID NUMBER(5,0) Y TBLPATENT_F_TBLRACE

SCHEDULEID NUMBER(10,0) N

STARTDATE DATE(7) Y

TESTPATIENT VARCHAR2(1) Y

TBLPROJECT PROJECT LOOKUP TABLE

Number of rows 5





MEDICATIONS VARCHAR2(1) Y



NAMINGTYPE VARCHAR2(10) Y

PATIENTIDLENGTH NUMBER(2,0) Y

PROJECTFOLDER VARCHAR2(20) Y

PROJECTID NUMBER(5,0) N UNIQUE TBLPROJECT_P

PROJECTNAME VARCHAR2(50) N

PROJECTTYPE VARCHAR2(10) Y

PROTOCOLCODE VARCHAR2(20) Y

SORTBY NUMBER(22) Y NORMAL

TBLQUESTION QUESTION LOOKUP TABLE

Number of rows 1226






EVENTID NUMBER(5,0) N NORMAL TBLQUESTION_F_TBLEVENT

FORMOBJECTTYPEID NUMBER(5,0) N NORMAL TBLQUESTION_F_TBLFORMOBJECT

FREQUENCYGROUPID NUMBER(5,0) Y NORMAL TBLQUESTION_F_TBLFREQGROUP

ISACCESSIONLINKED VARCHAR2(1) Y BITMAP

ISOPTIONOTHER VARCHAR2(1) Y NORMAL

ISPOINTLINKED VARCHAR2(1) Y BITMAP

ISPRIMARY VARCHAR2(1) Y

ISREQUIRED VARCHAR2(1) Y

MAXSIZE NUMBER(3,0) Y



ORIENTATION VARCHAR2(1) Y

QUESTIONID NUMBER(5,0) N UNIQUE TBLQUESTION_P

QUESTIONTEXT VARCHAR2(150) N

QUESTIONTYPEID NUMBER(5,0) Y NORMAL TBLQUESTION_F_TBLQUESTYPE




UNITID NUMBER(5,0) Y NORMAL TBLQUESTION_F_TBLUNIT

UNITS VARCHAR2(20) Y

TBLQUESTIONTYPE QUESTION TYPE (TEXT,NUMBER...) LOOKUP TABLE

Number of rows 10







QUESTIONTYPE VARCHAR2(50) N

QUESTIONTYPEID NUMBER(5,0) N UNIQUE TBLQUESTIONTYPE_P

SPSSTYPE VARCHAR2(7) Y

YESNOTYPE VARCHAR2(1) Y

TBLRACE RACE LOOKUP TABLE

Number of rows 6







RACE VARCHAR2(150) N

RACEID NUMBER(5,0) N UNIQUE TBLRACE_P


TBLREPORTOPTION IN/OUT REPORT PARAMETERS LOOKUP TABLE

Number of rows 24





FORMOBJECTTYPEID NUMBER(2,0) Y



OPTIONDESCRIPTION VARCHAR2(30) Y

OPTIONID VARCHAR2(30) Y

OPTIONTABLENAME VARCHAR2(30) Y

REPORTOPTION VARCHAR2(50) N

REPORTOPTIONGROUPID NUMBER(3,0) Y

REPORTOPTIONID NUMBER(5,0) N UNIQUE TBLREPORTOPTION_P

REPORTOPTIONLABEL VARCHAR2(30) Y

REPORTOPTIONTYPE VARCHAR2(3) Y

TBLSTATUS STATUS OF A PHYSICAL CONDITION LOOKUP TABLE

Number of rows 4








STATUS VARCHAR2(150) N

STATUSID NUMBER(5,0) N UNIQUE TBLSTATUS_P


TBLTIMELINE TIMELINES DEFINED FOR EACH PROJECT

Number of rows 126







NOMINALKINETICTIME NUMBER(15,2) Y

PROJECTID NUMBER(10,0) N

SORTBY NUMBER(10,0) N NORMAL

TIMELINEID NUMBER(10,0) N UNIQUE TBLTIMELINE_P

TIMELINENAME VARCHAR2(30) N

TIMELINEVALUE NUMBER(5,0) Y

TIMEUNITID NUMBER(2,0) Y

TBLYESNO YES/NO OPTION LOOKUP TABLE

Number of rows 2







OPTIONID NUMBER(5,0) N UNIQUE TBLYESNO_P

OPTIONTEXT VARCHAR2(150) N


PKD Entity Relationship Diagram

Materialized View SQL

CREATE MATERIALIZED VIEW "RIADMIN"."MVSTAT" PCTFREE 10 PCTUSED 0 MAXTRANS 255 STORAGE ( INITIAL 64K NEXT 0K MINEXTENTS 1 MAXEXTENTS 2147483645 PCTINCREASE 0) LOGGING TABLESPACE "USERS" BUILD IMMEDIATE USING INDEX TABLESPACE "USERS" PCTFREE 10 INITRANS 2 MAXTRANS 255 STORAGE ( INITIAL 64K NEXT 0K MINEXTENTS 1 MAXEXTENTS 2147483645 ) REFRESH FORCE ON DEMAND ENABLE QUERY REWRITE AS SELECT a.areaName, a.eventName,a.questionText, a.dateDriven, a.categorySOrt, a.areaSort, a.eventSort, a.questionSort, a.reportTotalOccur, a.reportFirstOccur,a.reportLastOccur, a.reportCompilation, a.patientID, a.timeLineSort, a.optionSort, a.timeLineName, a.eventID, a.questionID, a.questionTypeID, a.questionType, a.optionID, a.optionText, a.isOptionOther, a.patientEventID,a.eventDate, a.answerText, a.isOtherOption,a.yesno, c.institutionid, c.startdate, e.timelineid, g.schedulename, c.anonymousid, c.gender, to_char(c.birthdate, 'mm/dd/yyyy') AS birthdate, DECODE(c.hispanicLatino,'H','Hispanic or Latino','N','Not Hispanic or Latino','Unknown') AS hispanicLatino, nvl(c.race, 'Unknown') AS race, to_char(e.visitdate, 'mm/dd/yyyy hh:mi:ss') AS firsttimelinedate FROM (SELECT a.areaname, a.eventname, a.questiontext, a.datedriven, a.categorysort, a.areasort, a.eventsort, a.questionsort, a.reporttotaloccur, a.reportfirstoccur, a.reportlastoccur, a.reportcompilation, a.patientid, a.timelinesort, a.optionsort, nvl(a.timelinename, '') AS timelinename, a.eventid, a.questionid, a.questiontypeid, a.questiontype, a.optionid, a.optiontext, a.isoptionother, b.patienteventid, to_char( b.eventdate, 'mm/dd/yyyy hh:mi:ss') AS eventdate, b.answertext, b.isotheroption, b.yesno FROM (SELECT e.areaname, e.eventname, e.questiontext, e.datedriven, e.categorysort, e.areasort, e.eventsort, e.questionsort, e.reporttotaloccur, e.reportfirstoccur, e.reportlastoccur, e.reportcompilation, b.patientid, d.timelineid, d.timelinename, d.sortby timelinesort, e.eventid, e.questionid, e.questiontypeid, e.questiontype, f.optionid, f.optiontext, decode(e.questiontypeid, 3, 1, f.sortby) AS optionsort, 0 AS isoptionother FROM tblpatientinfo b, tbltimeline d, vieweventquestion e, tbloption f WHERE b.projectid = 1 AND d.projectid = 1

AND e.projectid = 1 AND NVL(datedriven, 0)=1 AND decode(e.questiontypeid, 3, 0, e.questionid) = f.questionid (+) UNION ALL SELECT e.areaname, e.eventname, e.questiontext, e.datedriven, e.categorysort, e.areasort, e.eventsort, e.questionsort, e.reporttotaloccur, e.reportfirstoccur, e.reportlastoccur, e.reportcompilation, b.patientid, d.timelineid, d.timelinename, d.sortby timelinesort, e.eventid, e.questionid, 0 AS questiontypeid, 'Text' AS questiontype, 0 AS optionid, 'Other' AS optiontext, 1000 AS optionsort, 1 AS isoptionother FROM tblpatientinfo b, tbltimeline d, vieweventquestion e WHERE b.projectid = 1 AND d.projectid = 1 AND e.projectid = 1 AND NVL(datedriven, 0)=1 AND nvl(e.isoptionother, 0) = 1 AND (e.questiontypeid = 3 OR e.questiontypeid = 4)) a, (SELECT e.patientid, e.patienteventid, e.eventdate, e.timelineid, q.answertext, q.questionid, q.answertext optionid, q.isotheroption, e.yesno FROM tbllinkpatientevent e, tbllinkpatientquestion q WHERE e.patienteventid = q.patienteventid ) b WHERE a.patientid = b.patientid (+) AND a.timelineid = b.timelineid (+) AND a.questionid = b.questionid (+) AND trim(to_char(decode(a.isoptionother, 1, 1, decode( a.questiontypeid, 4, a.optionid, 0)))) = decode( a.isoptionother, 1, trim(to_char(b.isotheroption (+))), decode(a.questiontypeid, 4, b.optionid (+), '0')) UNION ALL SELECT a.areaname, a.eventname, a.questiontext, a.datedriven, a.categorysort, a.areasort, a.eventsort, a.questionsort, a.reporttotaloccur, a.reportfirstoccur, a.reportlastoccur, a.reportcompilation, a.patientid, a.timelinesort, a.optionsort, a.timelinename, a.eventid, a.questionid, a.questiontypeid, a.questiontype, a.optionid, a.optiontext, a.isoptionother, a.patienteventid, to_char(a.eventdate, 'mm/dd/yyyy hh:mi:ss')

AS eventdate, a.answertext, a.isotheroption, a.yesno FROM (SELECT a.areaname, a.eventname, a.questiontext, a.datedriven, a.categorysort, a.areasort, a.eventsort, a.questionsort, a.reporttotaloccur, a.reportfirstoccur, a.reportlastoccur, a.reportcompilation, a.patientid, a.eventid, a.questionid, a.questiontypeid, a.questiontype, a.optionid, a.optiontext, a.optionsort, a.isoptionother, b.patienteventid, b.eventdate, b.answertext, b.isotheroption, b.yesno, c.sortby timelinesort, c.timelinename FROM (SELECT e.areaname, e.eventname, e.questiontext, e.datedriven, e.categorysort, e.areasort, e.eventsort, e.questionsort, e.reporttotaloccur, e.reportfirstoccur, e.reportlastoccur, e.reportcompilation, b.patientid, e.eventid, e.questionid, e.questiontypeid, e.questiontype, 0 AS optionid, '' AS optiontext, 0 AS optionsort, 0 AS isoptionother FROM tblpatientinfo b, vieweventquestion e WHERE b.projectid = 1 AND e.projectid = 1 AND NVL(datedriven, 0)=0 AND (e.questiontypeid = 1 OR e.questiontypeid = 2 OR e.questiontypeid = 5) UNION ALL SELECT e.areaname, e.eventname, e.questiontext, e.datedriven, e.categorysort, e.areasort, e.eventsort, e.questionsort, e.reporttotaloccur, e.reportfirstoccur, e.reportlastoccur, e.reportcompilation, b.patientid, e.eventid, e.questionid, e.questiontypeid, e.questiontype, f.optionid, f.optiontext, f.sortby optionsort, 0 AS isoptionother FROM tblpatientinfo b, vieweventquestion e, tblyesno f WHERE b.projectid = 1

AND e.projectid = 1 AND NVL(datedriven, 0)=0 AND e.questiontypeid = 6 UNION ALL SELECT e.areaname, e.eventname, e.questiontext, e.datedriven, e.categorysort, e.areasort, e.eventsort, e.questionsort, e.reporttotaloccur, e.reportfirstoccur, e.reportlastoccur, e.reportcompilation, b.patientid, e.eventid, e.questionid, e.questiontypeid, e.questiontype, f.optionid, f.optiontext, f.sortby optionsort, 0 AS isoptionother FROM tblpatientinfo b, vieweventquestion e, tblnormalabnormal f WHERE b.projectid = 1 AND e.projectid = 1 AND NVL(datedriven, 0)=0 AND e.questiontypeid = 8 UNION ALL SELECT e.areaname, e.eventname, e.questiontext, e.datedriven, e.categorysort, e.areasort, e.eventsort, e.questionsort, e.reporttotaloccur, e.reportfirstoccur, e.reportlastoccur, e.reportcompilation, b.patientid, e.eventid, e.questionid, e.questiontypeid, e.questiontype, f.statusid optionid, f.status optiontext, f.sortby optionsort, 0 AS isoptionother FROM tblpatientinfo b, vieweventquestion e, tblstatus f WHERE b.projectid = 1 AND e.projectid = 1 AND NVL(datedriven, 0)=0 AND e.questiontypeid = 10 UNION ALL SELECT e.areaname, e.eventname, e.questiontext, e.datedriven, e.categorysort, e.areasort, e.eventsort, e.questionsort, e.reporttotaloccur, e.reportfirstoccur, e.reportlastoccur, e.reportcompilation,

b.patientid, e.eventid, e.questionid, e.questiontypeid, e.questiontype, f.frequencytypeid optionid, f.frequencytype optiontext, f.sortby optionsort, 0 AS isoptionother FROM tblpatientinfo b, vieweventquestion e, viewfrequencytype f WHERE b.projectid = 1 AND e.projectid = 1 AND NVL(datedriven, 0)=0 AND e.questiontypeid = 7 AND e.frequencygroupid = f.frequencygroupid UNION ALL SELECT e.areaname, e.eventname, e.questiontext, e.datedriven, e.categorysort, e.areasort, e.eventsort, e.questionsort, e.reporttotaloccur, e.reportfirstoccur, e.reportlastoccur, e.reportcompilation, b.patientid, e.eventid, e.questionid, e.questiontypeid, e.questiontype, f.optionid, f.optiontext, f.sortby optionsort, 0 AS isoptionother FROM tblpatientinfo b, vieweventquestion e, tbloption f WHERE b.projectid = 1 AND e.projectid = 1 AND NVL(e.datedriven, 0)=0 AND (e.questiontypeid = 3 OR e.questiontypeid = 4) AND e.questionid = f.questionid UNION ALL SELECT e.areaname, e.eventname, e.questiontext, e.datedriven, e.categorysort, e.areasort, e.eventsort, e.questionsort, e.reporttotaloccur, e.reportfirstoccur, e.reportlastoccur, e.reportcompilation, b.patientid, e.eventid, e.questionid, 0 AS questiontypeid, 'Text' AS questiontype, 0 AS optionid, 'Other' AS optiontext, 1000 AS optionsort, 1 AS isoptionother FROM tblpatientinfo b, vieweventquestion e WHERE b.projectid = 1 AND e.projectid = 1 AND NVL(e.datedriven, 0)=0 AND nvl(e.isoptionother, 0) = 1 AND (e.questiontypeid = 3 OR e.questiontypeid = 4) UNION ALL SELECT e.areaname, e.eventname, '' AS

questiontext, e.datedriven, e.categorysort, e.areasort, e.eventsort, 0 AS questionsort, e.reporttotaloccur, e.reportfirstoccur, e.reportlastoccur, e.reportcompilation, b.patientid, e.eventid, 0 AS questionid, 0 AS questiontypeid, '0' AS questiontype, 0 AS optionid, '0' AS optiontext, 0 AS optionsort, 0 AS isoptionother FROM tblpatientinfo b, viewcategoryevent e WHERE b.projectid = 1 AND e.projectid = 1 AND NVL(e.datedriven, 0)=0 ) a, (SELECT e.patienteventid, e.patientid, e.eventid, e.eventdate, q.answertext, q.questionid, q.answertext optionid, q.isotheroption, e.yesno FROM tbllinkpatientevent e, tbllinkpatientquestion q WHERE e.patienteventid = q.patienteventid AND nvl(e.yesno,0) = 1 UNION ALL SELECT e.patienteventid, e.patientid, e.eventid, e.eventdate, '0' AS answertext, 0 AS questionid, '0' AS optionid, '0' AS isotheroption, e.yesno FROM tbllinkpatientevent e) b, tbltimeline c WHERE a.patientid = b.patientid (+) AND decode(a.questionid, 0, a.eventid, a.questionid) = decode(a.questionid, 0, b.eventid (+), b.questionid (+)) AND decode(questiontypeid, 1, '1', 2, '1', 5, '1', trim(to_char(decode(a.isoptionother, 1, 1, a.optionid)))) = decode(questiontypeid, 1, '1', 2,

'1', 5, '1', decode(a.isoptionother, 1, trim(to_char(b.isotheroption (+))), b.optionid (+))) AND c.projectid = 1) a ) a, viewpatientinfo c, tblschedule g, tbllinkpatienttimeline e WHERE a.patientid = c.patientid AND nvl(c.testpatient, 0) = 0 AND nvl(c.excludedata, 0) = 0 AND nvl(c.deleted, 0) = 0 AND a.patientid = e.patientid (+) AND c.scheduleid = g.scheduleid AND c.projectid = 1

ViewEventQuestion

SELECT a.areaID, a.eventID, a.eventName, a.sortBy AS eventSort, a.eventtypeid, a.datelabel, a.yesnotype AS eventYesNoType, a.schematype, a.reportTotalOccur, a.reportFirstOccur, a.reportLastOccur, a.reportCompilation, a.umlscui as eventumlscui, a.umlsconfirm as eventumlsconfirm, a.automatch as eventautomatch, b.questionText, b.sortBy AS questionSort, b.isrequired, b.frequencyGroupID, b.formobjecttypeid, b.questiontypeid, b.isoptionother, b.maxsize, b.orientation, b.ispointlinked, NVL(b.questionID,0) AS questionID, b.umlscui as questionumlscui, b.umlsconfirm as questionumlsconfirm, b.automatch as questionautomatch, b.units, b.unitid, b.isAccessionLinked, b.isPrimary, d.questionType, d.spssType, e.formObjectType, f.areaName, f.sortBy AS areaSort, f.dateDriven, f.umlscui as areaumlscui, f.umlsconfirm as areaumlsconfirm, f.automatch as areaautomatch, h.dateLabel AS areaDateLabel, h.categoryID, h.categoryName,

h.categoryDescription, h.sortBy AS categorySort, h.projectID, h.isMajor AS isMajorCategory, h.umlscui as categoryumlscui, h.umlsconfirm as categoryumlsconfirm, h.automatch as categoryautomatch, i.unitname, a.isMultiple, j.eventtype, j.eventTypeDescription, j.eventTypeVerb FROM tblEvent a, tblQuestion b, tblQuestionType d, tblformObjectType e, tblArea f, tblLinkCategoryArea g, tblCategory h, tblUnit i, tblEventtype j WHERE a.eventID = b.eventID (+) AND b.questionTypeID = d.questionTypeID (+) AND b.formObjectTypeID = e.formObjectTypeID (+) AND a.areaID = f.areaID AND f.areaID = g.areaID AND h.categoryID = g.categoryID AND b.unitid=i.unitid(+) AND a.eventtypeid=j.eventtypeid(+)

ViewPatientInfo

SELECT a."PATIENTID",a."BIRTHDATE",a."GENDER",a."DATECREATED",a."DATEMODIFIED", a."MODIFIEDBYUSERID",a."MODIFIEDBYUSER",a."ANONYMOUSID",a."MARITALSTATUS", a."PROJECTID",a."PATIENTSUMMARY",a."HISPANICLATINO",a."RACEID",a."TESTPATIENT", a."DELETED",a."INSTITUTIONID",a.excludeData, a.isDropped, a.startDate,a.scheduleid, b.race FROM tblpatientInfo a, tblRace b WHERE a.raceID = b.raceID (+)

Original PKD Query Execution Plan COST Rows Plan ---------- ---------- ------------------------------------------------------------- 4014701 125 SELECT STATEMENT 4014701 125 SORT ORDER BY 4014676 125 HASH JOIN OUTER 4014672 95 HASH JOIN 4 1 NESTED LOOPS 3 1 NESTED LOOPS OUTER 2 1 TABLE ACCESS FULL TBLPATIENTINFO 1 1 TABLE ACCESS BY INDEX ROWID TBLRACE 1 INDEX UNIQUE SCAN TBLRACE_P 1 1 TABLE ACCESS BY INDEX ROWID TBLSCHEDULE 1 INDEX UNIQUE SCAN TBLSCHEDULE_P 4014242 981576 VIEW UNION-ALL 12799 523338 HASH JOIN OUTER 1625 523338 VIEW UNION-ALL 1218 464771 HASH JOIN OUTER 787 58567 MERGE JOIN CARTESIAN 21 383 NESTED LOOPS OUTER 21 383 HASH JOIN OUTER 18 383 NESTED LOOPS OUTER 18 383 HASH JOIN OUTER 13 80 NESTED LOOPS OUTER 13 80 HASH JOIN 9 69 HASH JOIN 6 69 HASH JOIN 3 69 MERGE JOIN CARTESIAN 2 1 TABLE ACCESS FULL TBLTIMELINE 1 55 BUFFER SORT 1 55 INDEX FULL SCAN TBLLINKCATEGORYAREA_P 2 14 TABLE ACCESS FULL TBLCATEGORY 2 57 TABLE ACCESS FULL TBLAREA 3 66 TABLE ACCESS FULL TBLEVENT 1 INDEX UNIQUE SCAN TBLFREQUENCYGROUP_P_1 4 1226 TABLE ACCESS FULL TBLQUESTION 1 INDEX UNIQUE SCAN TBLFORMOBJECTTYPE_P 2 10 TABLE ACCESS FULL TBLQUESTIONTYPE 1 INDEX UNIQUE SCAN TBLUNIT_P 787 153 BUFFER SORT 2 153 TABLE ACCESS FULL TBLPATIENTINFO 4 1976 TABLE ACCESS FULL TBLOPTION 662 58567 NESTED LOOPS OUTER 662 58567 NESTED LOOPS OUTER 662 58567 NESTED LOOPS OUTER 662 58567 NESTED LOOPS OUTER FILTER HASH JOIN OUTER 593 12277 HASH JOIN 2 14 TABLE ACCESS FULL TBLCATEGORY 589 12277 HASH JOIN 1 55 INDEX FULL SCAN TBLLINKCATEGORYAREA_P 586 12723 HASH JOIN 2 57 TABLE ACCESS FULL TBLAREA 583 12723 MERGE JOIN CARTESIAN 4 193 MERGE JOIN CARTESIAN 2 1 TABLE ACCESS FULL TBLTIMELINE 2 153 BUFFER SORT 2 153 TABLE ACCESS FULL TBLPATIENTINFO 581 66 BUFFER SORT

3 66 TABLE ACCESS FULL TBLEVENT 4 1226 TABLE ACCESS FULL TBLQUESTION 1 INDEX UNIQUE SCAN TBLFREQUENCYGROUP_P_1 1 INDEX UNIQUE SCAN TBLUNIT_P 1 INDEX UNIQUE SCAN TBLQUESTIONTYPE_P 1 INDEX UNIQUE SCAN TBLFORMOBJECTTYPE_P 122 50220 VIEW 122 50220 HASH JOIN 37 12992 TABLE ACCESS FULL TBLLINKPATIENTEVENT 43 50222 TABLE ACCESS FULL TBLLINKPATIENTQUESTION 4001443 458238 MERGE JOIN CARTESIAN 3274081 363681 NESTED LOOPS OUTER 952 363681 VIEW UNION-ALL 798 75356 NESTED LOOPS OUTER 798 75356 HASH JOIN OUTER 386 75356 NESTED LOOPS OUTER FILTER HASH JOIN OUTER 317 15797 NESTED LOOPS OUTER 317 15797 HASH JOIN 3 107 TABLE ACCESS FULL TBLEVENT 313 8415 HASH JOIN 2 57 TABLE ACCESS FULL TBLAREA 310 8415 HASH JOIN 1 55 INDEX FULL SCAN TBLLINKCATEGORYAREA_P 308 2142 MERGE JOIN CARTESIAN 2 153 TABLE ACCESS FULL TBLPATIENTINFO 306 14 BUFFER SORT 2 14 TABLE ACCESS FULL TBLCATEGORY 1 INDEX UNIQUE SCAN TBLFREQUENCYGROUP_P_1 4 1226 TABLE ACCESS FULL TBLQUESTION 1 INDEX UNIQUE SCAN TBLFORMOBJECTTYPE_P 2 10 TABLE ACCESS FULL TBLQUESTIONTYPE 1 INDEX UNIQUE SCAN TBLUNIT_P 539 39687 MERGE JOIN CARTESIAN 21 259 NESTED LOOPS OUTER 21 259 NESTED LOOPS OUTER 21 259 HASH JOIN OUTER 19 259 HASH JOIN 4 152 TABLE ACCESS FULL TBLQUESTION 14 206 NESTED LOOPS OUTER 14 206 HASH JOIN 10 110 HASH JOIN 7 110 HASH JOIN 4 110 MERGE JOIN CARTESIAN 2 2 TABLE ACCESS FULL TBLYESNO 2 55 BUFFER SORT 1 55 INDEX FULL SCAN TBLLINKCATEGORYAREA_P 2 14 TABLE ACCESS FULL TBLCATEGORY 2 57 TABLE ACCESS FULL TBLAREA 3 107 TABLE ACCESS FULL TBLEVENT 1 INDEX UNIQUE SCAN TBLFREQUENCYGROUP_P_1 1 1 TABLE ACCESS BY INDEX ROWID TBLQUESTIONTYPE 1 INDEX UNIQUE SCAN TBLQUESTIONTYPE_P 1 INDEX UNIQUE SCAN TBLFORMOBJECTTYPE_P 1 INDEX UNIQUE SCAN TBLUNIT_P 539 153 BUFFER SORT 2 153 TABLE ACCESS FULL TBLPATIENTINFO 106 6496 MERGE JOIN CARTESIAN 22 42 NESTED LOOPS OUTER 22 42 NESTED LOOPS OUTER 22 42 HASH JOIN OUTER 20 42 HASH JOIN 4 22 TABLE ACCESS FULL TBLQUESTION

15 206 NESTED LOOPS OUTER 15 206 HASH JOIN 11 110 HASH JOIN 8 110 HASH JOIN 6 28 MERGE JOIN CARTESIAN 2 2 TABLE ACCESS FULL TBLNORMALABNORMAL 4 14 BUFFER SORT 2 14 TABLE ACCESS FULL TBLCATEGORY 1 55 INDEX FULL SCAN TBLLINKCATEGORYAREA_P 2 57 TABLE ACCESS FULL TBLAREA 3 107 TABLE ACCESS FULL TBLEVENT 1 INDEX UNIQUE SCAN TBLFREQUENCYGROUP_P_1 1 1 TABLE ACCESS BY INDEX ROWID TBLQUESTIONTYPE 1 INDEX UNIQUE SCAN TBLQUESTIONTYPE_P 1 INDEX UNIQUE SCAN TBLFORMOBJECTTYPE_P 1 INDEX UNIQUE SCAN TBLUNIT_P 106 153 BUFFER SORT 2 153 TABLE ACCESS FULL TBLPATIENTINFO 945 70500 MERGE JOIN CARTESIAN 23 461 NESTED LOOPS OUTER 23 461 NESTED LOOPS OUTER 23 461 HASH JOIN OUTER 21 461 HASH JOIN 4 121 TABLE ACCESS FULL TBLQUESTION 16 413 HASH JOIN 2 14 TABLE ACCESS FULL TBLCATEGORY 13 413 NESTED LOOPS OUTER 13 413 HASH JOIN 3 107 TABLE ACCESS FULL TBLEVENT 9 220 HASH JOIN 6 220 MERGE JOIN CARTESIAN 2 4 TABLE ACCESS FULL TBLSTATUS 4 55 BUFFER SORT 1 55 INDEX FULL SCAN TBLLINKCATEGORYAREA_P 2 57 TABLE ACCESS FULL TBLAREA 1 INDEX UNIQUE SCAN TBLFREQUENCYGROUP_P_1 1 1 TABLE ACCESS BY INDEX ROWID TBLQUESTIONTYPE 1 INDEX UNIQUE SCAN TBLQUESTIONTYPE_P 1 INDEX UNIQUE SCAN TBLFORMOBJECTTYPE_P 1 INDEX UNIQUE SCAN TBLUNIT_P 945 153 BUFFER SORT 2 153 TABLE ACCESS FULL TBLPATIENTINFO 12 215 MERGE JOIN CARTESIAN 1 5 TABLE ACCESS BY INDEX ROWID TBLFREQUENCYTYPE 10 1 NESTED LOOPS 9 1 NESTED LOOPS 8 1 NESTED LOOPS 7 1 NESTED LOOPS 6 1 NESTED LOOPS OUTER 6 1 NESTED LOOPS 5 1 NESTED LOOPS OUTER 4 1 NESTED LOOPS 4 1 NESTED LOOPS OUTER 4 1 NESTED LOOPS OUTER 4 1 TABLE ACCESS FULL TBLQUESTION 1 INDEX UNIQUE SCAN TBLUNIT_P 1 INDEX UNIQUE SCAN TBLFORMOBJECTTYPE_P 1 INDEX UNIQUE SCAN TBLFREQUENCYGROUP_P 1 1 TABLE ACCESS BY INDEX ROWID TBLQUESTIONTYPE 1 INDEX UNIQUE SCAN TBLQUESTIONTYPE_P 1 1 TABLE ACCESS BY INDEX ROWID TBLEVENT 1 INDEX UNIQUE SCAN TBLEVENT_P 1 INDEX UNIQUE SCAN TBLFREQUENCYGROUP_P_1 1 1 TABLE ACCESS BY INDEX ROWID TBLAREA

1 INDEX UNIQUE SCAN TBLAREA_P 1 1 TABLE ACCESS BY INDEX ROWID TBLLINKCATEGORYAREA 1 INDEX RANGE SCAN TBLLINKCATEGORYAREA_I 1 1 TABLE ACCESS BY INDEX ROWID TBLCATEGORY 1 INDEX UNIQUE SCAN TBLCATEGORY_P 5 INDEX RANGE SCAN TBLFREQUENCYFREQGROUP_I 11 153 BUFFER SORT 2 153 TABLE ACCESS FULL TBLPATIENTINFO 864 95947 HASH JOIN 4 1976 TABLE ACCESS FULL TBLOPTION 798 75356 NESTED LOOPS OUTER 798 75356 HASH JOIN OUTER 386 75356 NESTED LOOPS OUTER FILTER HASH JOIN OUTER 317 15797 NESTED LOOPS OUTER 317 15797 HASH JOIN 3 107 TABLE ACCESS FULL TBLEVENT 313 8415 HASH JOIN 2 57 TABLE ACCESS FULL TBLAREA 310 8415 HASH JOIN 1 55 INDEX FULL SCAN TBLLINKCATEGORYAREA_P 308 2142 MERGE JOIN CARTESIAN 2 153 TABLE ACCESS FULL TBLPATIENTINFO 306 14 BUFFER SORT 2 14 TABLE ACCESS FULL TBLCATEGORY 1 INDEX UNIQUE SCAN TBLFREQUENCYGROUP_P_1 4 1226 TABLE ACCESS FULL TBLQUESTION 1 INDEX UNIQUE SCAN TBLFORMOBJECTTYPE_P 2 10 TABLE ACCESS FULL TBLQUESTIONTYPE 1 INDEX UNIQUE SCAN TBLUNIT_P 387 75356 NESTED LOOPS OUTER 387 75356 NESTED LOOPS OUTER 387 75356 NESTED LOOPS OUTER FILTER HASH JOIN OUTER 317 15797 NESTED LOOPS OUTER 317 15797 HASH JOIN 3 107 TABLE ACCESS FULL TBLEVENT 313 8415 HASH JOIN 2 57 TABLE ACCESS FULL TBLAREA 310 8415 HASH JOIN 1 55 INDEX FULL SCAN TBLLINKCATEGORYAREA_P 308 2142 MERGE JOIN CARTESIAN 2 153 TABLE ACCESS FULL TBLPATIENTINFO 306 14 BUFFER SORT 2 14 TABLE ACCESS FULL TBLCATEGORY 1 INDEX UNIQUE SCAN TBLFREQUENCYGROUP_P_1 4 1226 TABLE ACCESS FULL TBLQUESTION 1 INDEX UNIQUE SCAN TBLFORMOBJECTTYPE_P 1 INDEX UNIQUE SCAN TBLQUESTIONTYPE_P 1 INDEX UNIQUE SCAN TBLUNIT_P 7 184 MERGE JOIN CARTESIAN 5 1 NESTED LOOPS OUTER 5 1 NESTED LOOPS 4 1 NESTED LOOPS 3 1 NESTED LOOPS 2 1 TABLE ACCESS FULL TBLCATEGORY 1 5 TABLE ACCESS BY INDEX ROWID TBLLINKCATEGORYAREA 5 INDEX RANGE SCAN TBLLINKCATAREA_I 1 1 TABLE ACCESS BY INDEX ROWID TBLAREA 1 INDEX UNIQUE SCAN TBLAREA_P 1 2 TABLE ACCESS BY INDEX ROWID TBLEVENT

5 INDEX RANGE SCAN TBLEVENT_I 1 INDEX UNIQUE SCAN TBLFREQUENCYGROUP_P_1 7 153 BUFFER SORT 2 153 TABLE ACCESS FULL TBLPATIENTINFO 9 1 VIEW UNION-ALL PARTITION 2 4 TABLE ACCESS BY INDEX ROWID TBLLINKPATIENTQUESTION 168 344 NESTED LOOPS 10 79 TABLE ACCESS BY INDEX ROWID TBLLINKPATIENTEVENT 1 199 INDEX RANGE SCAN TBLLINKPATEVENT_I 1 4 INDEX RANGE SCAN TBLLINKPATQUESPATEVENT_I 10 157 TABLE ACCESS BY INDEX ROWID TBLLINKPATIENTEVENT 1 199 INDEX RANGE SCAN TBLLINKPATEVENT_I 4001434 1 BUFFER SORT 2 1 TABLE ACCESS FULL TBLTIMELINE 2 137 TABLE ACCESS FULL TBLLINKPATIENTTIMELINE

Documents

Medical Research Database Tuning