275
1 Advanced PL/SQL Optimizing for Better Performance Zohar Elkayam CTO, Brillix [email protected] www.realdbamagic.com Twitter: @realmgic

Advanced PL/SQL Optimizing for Better Performance 2016

Embed Size (px)

Citation preview

Page 1: Advanced PL/SQL Optimizing for Better Performance 2016

1

Advanced PL/SQLOptimizing for Better Performance

Zohar Elkayam CTO, Brillix

[email protected]

Twitter: @realmgic

Page 2: Advanced PL/SQL Optimizing for Better Performance 2016

2

Who am I?• Zohar Elkayam, CTO at Brillix

• Programmer, DBA, team leader, database trainer, public speaker, and a senior consultant for over 18 years

• Oracle ACE Associate

• Part of ilOUG – Israel Oracle User Group

• Blogger – www.realdbamagic.com and www.ilDBA.co.il

Page 3: Advanced PL/SQL Optimizing for Better Performance 2016

3

About Brillix• We offer complete, integrated end-to-end solutions based on

best-of-breed innovations in database, security and big data technologies

• We provide complete end-to-end 24x7 expert remote database services

• We offer professional customized on-site trainings, delivered by our top-notch world recognized instructors

Page 4: Advanced PL/SQL Optimizing for Better Performance 2016

4

Some of Our Customers

Page 5: Advanced PL/SQL Optimizing for Better Performance 2016

5

Agenda• Developing PL/SQL:

– Composite datatypes, advanced cursors, dynamic SQL, tracing, and more…

• Compiling PL/SQL: – dependencies, optimization levels, and DBMS_WARNING

• Tuning PL/SQL: – GTT, Result cache and Memory handling

• Oracle 11g, 12cR1 and 12cR2 new useful features• SQLcl – New replacement tool for SQL*Plus (if we

have time)

5

Page 6: Advanced PL/SQL Optimizing for Better Performance 2016

6

Our Goal Today• Learning new and old PL/SQL techniques

• We will not expert everything

• Getting to know new features (12cR1 and 12cR2)

• This is a starting point – don’t be afraid to try

Page 7: Advanced PL/SQL Optimizing for Better Performance 2016

7

The REAL Agenda

נשמח לקבל את חוות , יחולק טופס משוביום הסמינר בסיום •

!טאבלטממלאי המשוב יוגרל מידי יום בין . דעתכם

הפסקה10:30-10:45

י הכנס בגן המלוןארוחת צהריים לכל משתתפ12:30-13:30

הפסקה מתוקה במתחם קבלת הפנים15:00-15:15

הולכים הביתה16:30

Page 8: Advanced PL/SQL Optimizing for Better Performance 2016

Developing PL/SQL

8

Page 9: Advanced PL/SQL Optimizing for Better Performance 2016

9

Developing PL/SQL• Composite datatypes

• Advanced Cursors and Bulk operations

• Dynamic SQL and SQL Injection

• Autonomous Transactions

• 11g mentionable features

• 12cR1 new development features

• 12cR2 new features

9

Page 10: Advanced PL/SQL Optimizing for Better Performance 2016

Composite Datatypes

10

Page 11: Advanced PL/SQL Optimizing for Better Performance 2016

11

Composite Datatypes• Collections

– Nested table, varray

– Associative arrays/PLSQL tables

• Use collections methods

• Manipulate collections

• Distinguish between the different types of collections and when to use them

Page 12: Advanced PL/SQL Optimizing for Better Performance 2016

12

Understanding Collections• A collection is a group of elements, all of the

same type.• Collections work like arrays.• Collections can store instances of an object type and,

conversely, can be attributes of an object type.• Types of collections in PL/SQL:

– Associative arrays• String-indexed collections• INDEX BY pls_integer or BINARY_INTEGER

– Nested tables– Varrays

Page 13: Advanced PL/SQL Optimizing for Better Performance 2016

13

Collection Types

Nested table Varray

Associative array

1 2 3 4 5 6 a f i o t w

Index byPLS_INTEGER

Index byVARCHAR2

Page 14: Advanced PL/SQL Optimizing for Better Performance 2016

15

Using Associative ArraysAssociative arrays:

• That are indexed by strings can improve performance

• Are pure memory structures that are much faster than schema-level tables

• Provide significant additional flexibilityAssociative arrays

1 2 3 4 5 6 a f i o t w

Index byPLS_INTEGER

Index byVARCHAR2

Page 15: Advanced PL/SQL Optimizing for Better Performance 2016

17

Creating the ArrayAssociative array in PL/SQL (string-indexed):

TYPE type_name IS TABLE OF element_type

INDEX BY VARCHAR2(size)

CREATE OR REPLACE PROCEDURE report_credit

(p_last_name customers.cust_last_name%TYPE,

p_credit_limit customers.credit_limit%TYPE)

IS

TYPE typ_name IS TABLE OF customers%ROWTYPE

INDEX BY customers.cust_email%TYPE;

v_by_cust_email typ_name;

i VARCHAR2(30);

PROCEDURE load_arrays IS

BEGIN

FOR rec IN (SELECT * FROM customers WHERE cust_email IS NOT NULL)

LOOP

-- Load up the array in single pass to database table.

v_by_cust_email (rec.cust_email) := rec;

END LOOP;

END;

...

Create the string-indexed associative array type.

Create the string-indexed associative array variable.

Populate the string-indexed associative array variable.

Page 16: Advanced PL/SQL Optimizing for Better Performance 2016

18

Traversing the Array...

BEGIN

load_arrays;

i:= v_by_cust_email.FIRST;

dbms_output.put_line ('For credit amount of: ' || p_credit_limit);

WHILE i IS NOT NULL LOOP

IF v_by_cust_email(i).cust_last_name = p_last_name

AND v_by_cust_email(i).credit_limit > p_credit_limit

THEN dbms_output.put_line ( 'Customer '||

v_by_cust_email(i).cust_last_name || ': ' ||

v_by_cust_email(i).cust_email || ' has credit limit of: ' ||

v_by_cust_email(i).credit_limit);

END IF;

i := v_by_cust_email.NEXT(i);

END LOOP;

END report_credit;

/

EXECUTE report_credit('Walken', 1200)

For credit amount of: 1200

Customer Walken: [email protected] has credit limit of: 3600

Customer Walken: [email protected] has credit limit of: 3700

Page 17: Advanced PL/SQL Optimizing for Better Performance 2016

20

Using Nested TablesNested table characteristics:

– A table within a table

– Unbounded

– Available in both SQL and PL/SQL as well as the database

– Array-like access to individual rows

Nested table:

Page 18: Advanced PL/SQL Optimizing for Better Performance 2016

21

pOrder nested table:

Nested Table StorageNested tables are stored out-of-line in storage tables.

Storage table:

ORDID SUPPLIER REQUESTER ORDERED ITEMS

500 50 5000 30-OCT-07

800 80 8000 31-OCT-07

NESTED_TABLE_ID PRODID PRICE

55 555

56 566

57 577

NESTED_TABLE_ID PRODID PRICE

88 888

Page 19: Advanced PL/SQL Optimizing for Better Performance 2016

22

Creating Nested TablesTo create a nested table in the database:

To create a nested table in PL/SQL:

CREATE [OR REPLACE] TYPE type_name AS TABLE OF

Element_datatype [NOT NULL];

TYPE type_name IS TABLE OF element_datatype [NOT NULL];

Page 20: Advanced PL/SQL Optimizing for Better Performance 2016

23

Declaring Collections: Nested Table– First, define an object type:

– Second, declare a column of that collection type:

CREATE TYPE typ_item AS OBJECT --create object

(prodid NUMBER(5),

price NUMBER(7,2) )

/

CREATE TYPE typ_item_nst -- define nested table type

AS TABLE OF typ_item

/

CREATE TABLE pOrder ( -- create database table

ordid NUMBER(5),

supplier NUMBER(5),

requester NUMBER(4),

ordered DATE,

items typ_item_nst)

NESTED TABLE items STORE AS item_stor_tab

/

1

2

3

Page 21: Advanced PL/SQL Optimizing for Better Performance 2016

24

Using Nested TablesAdd data to the nested table:INSERT INTO pOrder

VALUES (500, 50, 5000, sysdate, typ_item_nst(

typ_item(55, 555),

typ_item(56, 566),

typ_item(57, 577)));

INSERT INTO pOrder

VALUES (800, 80, 8000, sysdate,

typ_item_nst (typ_item (88, 888)));

ORDID SUPPLIER REQUESTER ORDERED ITEMS

500 50 5000 30-OCT-07

800 80 8000 31-OCT-07

PRODID PRICE

55 555

56 566

57 577

PRODID PRICE

88 888

1

2

1

2

pOrder nested table

Page 22: Advanced PL/SQL Optimizing for Better Performance 2016

25

Using Nested Tables• Querying the results:

• Querying the results with the TABLE function:

SELECT * FROM porder;

ORDID SUPPLIER REQUESTER ORDERED

---------- ---------- ---------- ---------

ITEMS(PRODID, PRICE)

-----------------------------------------------------------------

500 50 5000 31-OCT-07

TYP_ITEM_NST(TYP_ITEM(55, 555), TYP_ITEM(56, 566), TYP_ITEM(57, 577))

800 80 8000 31-OCT-07

TYP_ITEM_NST(TYP_ITEM(88, 888))

SELECT p2.ordid, p1.*

FROM porder p2, TABLE(p2.items) p1;

ORDID PRODID PRICE

---------- ---------- ----------

800 88 888

500 57 577

500 55 555

500 56 566

Page 23: Advanced PL/SQL Optimizing for Better Performance 2016

26

Referencing Collection ElementsUse the collection name and a subscript to reference a collection element:

– Syntax:

– Example:

– To reference a field in a collection:

collection_name(subscript)

v_with_discount(i)

p_new_items(i).prodid

Page 24: Advanced PL/SQL Optimizing for Better Performance 2016

27

Using Nested Tables in PL/SQLCREATE OR REPLACE PROCEDURE add_order_items

(p_ordid NUMBER, p_new_items typ_item_nst)

IS

v_num_items NUMBER;

v_with_discount typ_item_nst;

BEGIN

v_num_items := p_new_items.COUNT;

v_with_discount := p_new_items;

IF v_num_items > 2 THEN

--ordering more than 2 items gives a 5% discount

FOR i IN 1..v_num_items LOOP

v_with_discount(i) :=

typ_item(p_new_items(i).prodid,

p_new_items(i).price*.95);

END LOOP;

END IF;

UPDATE pOrder

SET items = v_with_discount

WHERE ordid = p_ordid;

END;

Page 25: Advanced PL/SQL Optimizing for Better Performance 2016

28

Using Nested Tables in PL/SQL-- caller pgm:

DECLARE

v_form_items typ_item_nst:= typ_item_nst();

BEGIN

-- let's say the form holds 4 items

v_form_items.EXTEND(4);

v_form_items(1) := typ_item(1804, 65);

v_form_items(2) := typ_item(3172, 42);

v_form_items(3) := typ_item(3337, 800);

v_form_items(4) := typ_item(2144, 14);

add_order_items(800, v_form_items);

END;

PRODID PRICE

1804 65

3172 42

3337 800

2144 14

v_form_items variable

ORDID SUPPLIER REQUESTER ORDERED ITEMS

500 50 5000 30-OCT-07

800 80 8000 31-OCT-07

Resulting data in the pOrder nested table

PRODID PRICE

1804 61.75

3172 39.9

3337 760

2144 13.3

The prices are added after discounts.

Page 26: Advanced PL/SQL Optimizing for Better Performance 2016

29

Understanding Varrays• To create a varray in the database:

• To create a varray in PL/SQL:

Varray:

CREATE [OR REPLACE] TYPE type_name AS VARRAY (max_elements) OF element_datatype [NOT NULL];

TYPE type_name IS VARRAY (max_elements) OF

element_datatype [NOT NULL];

Page 27: Advanced PL/SQL Optimizing for Better Performance 2016

30

CREATE TABLE department ( -- create database table

dept_id NUMBER(2),

name VARCHAR2(25),

budget NUMBER(12,2),

projects typ_ProjectList) -- declare varray as column

/

Declaring Collections: Varray• First, define a collection type:

• Second, declare a collection of that type:

CREATE TYPE typ_Project AS OBJECT( --create object

project_no NUMBER(4),

title VARCHAR2(35),

cost NUMBER(12,2))

/

CREATE TYPE typ_ProjectList AS VARRAY (50) OF typ_Project

-- define VARRAY type

/

1

2

3

Page 28: Advanced PL/SQL Optimizing for Better Performance 2016

31

Using VarraysAdd data to the table containing a varray column:

INSERT INTO department

VALUES (10, 'Exec Admn', 30000000,

typ_ProjectList(

typ_Project(1001, 'Travel Monitor', 400000),

typ_Project(1002, 'Open World', 10000000)));

INSERT INTO department

VALUES (20, 'IT', 5000000,

typ_ProjectList(

typ_Project(2001, 'DB11gR2', 900000)));

1

2

DEPT_ID NAME BUDGET PROJECTS

PROJECT_NO TITLE COSTS

10 Exec Admn 30000000 1001 Travel Monitor 400000

1002 Open World 10000000

20 IT 5000000 2001 DB11gR2 900000

1

2

DEPARTMENT table

Page 29: Advanced PL/SQL Optimizing for Better Performance 2016

32

– Querying the results:

– Querying the results with the TABLE function:

Using Varrays

SELECT * FROM department;

DEPT_ID NAME BUDGET

---------- ------------------------- ----------

PROJECTS(PROJECT_NO, TITLE, COST)

-----------------------------------------------------------------

10 Executive Administration 30000000

TYP_PROJECTLIST(TYP_PROJECT(1001, 'Travel Monitor', 400000),

TYP_PROJECT(1002, 'Open World', 10000000))

20 Information Technology 5000000

TYP_PROJECTLIST(TYP_PROJECT(2001, 'DB11gR2', 900000))

SELECT d2.dept_id, d2.name, d1.*

FROM department d2, TABLE(d2.projects) d1;

DEPT_ID NAME PROJECT_NO TITLE COST

------- ------------------------ ---------- -------------- --------

10 Executive Administration 1001 Travel Monitor 400000

10 Executive Administration 1002 Open World 10000000

20 Information Technology 2001 DB11gR2 900000

Page 30: Advanced PL/SQL Optimizing for Better Performance 2016

33

Working with Collections in PL/SQL• You can declare collections as the formal parameters of procedures

and functions.• You can specify a collection type in the RETURN clause of a

function specification.• Collections follow the usual scoping and instantiation rules.

CREATE OR REPLACE PACKAGE manage_dept_projASPROCEDURE allocate_new_proj_list(p_dept_id NUMBER, p_name VARCHAR2, p_budget NUMBER);

FUNCTION get_dept_project (p_dept_id NUMBER)RETURN typ_projectlist;

PROCEDURE update_a_project(p_deptno NUMBER, p_new_project typ_Project,p_position NUMBER);

FUNCTION manipulate_project (p_dept_id NUMBER)RETURN typ_projectlist;

FUNCTION check_costs (p_project_list typ_projectlist)RETURN boolean;

END manage_dept_proj;

Page 31: Advanced PL/SQL Optimizing for Better Performance 2016

36

Initializing CollectionsThree ways to initialize:

– Use a constructor.

– Fetch from the database.

– Assign another collection variable directly.PROCEDURE allocate_new_proj_list

(p_dept_id NUMBER, p_name VARCHAR2, p_budget NUMBER)ISv_accounting_project typ_projectlist;

BEGIN-- this example uses a constructorv_accounting_project :=typ_ProjectList(typ_Project (1, 'Dsgn New Expense Rpt', 3250),typ_Project (2, 'Outsource Payroll', 12350),typ_Project (3, 'Audit Accounts Payable',1425));

INSERT INTO departmentVALUES(p_dept_id, p_name, p_budget, v_accounting_project);

END allocate_new_proj_list;

Page 32: Advanced PL/SQL Optimizing for Better Performance 2016

37

FUNCTION get_dept_project (p_dept_id NUMBER)RETURN typ_projectlist

ISv_accounting_project typ_projectlist;

BEGIN -- this example uses a fetch from the databaseSELECT projects INTO v_accounting_projectFROM department WHERE dept_id = p_dept_id;

RETURN v_accounting_project;END get_dept_project;

Initializing Collections

FUNCTION manipulate_project (p_dept_id NUMBER)RETURN typ_projectlist

ISv_accounting_project typ_projectlist;v_changed_list typ_projectlist;

BEGINSELECT projects INTO v_accounting_project

FROM department WHERE dept_id = p_dept_id; -- this example assigns one collection to anotherv_changed_list := v_accounting_project;RETURN v_changed_list;

END manipulate_project;

1

2

Page 33: Advanced PL/SQL Optimizing for Better Performance 2016

38

Referencing Collection Elements-- sample caller program to the manipulate_project functionDECLAREv_result_list typ_projectlist;

BEGINv_result_list := manage_dept_proj.manipulate_project(10);FOR i IN 1..v_result_list.COUNT LOOPdbms_output.put_line('Project #: '

||v_result_list(i).project_no);dbms_output.put_line('Title: '||v_result_list(i).title);dbms_output.put_line('Cost: ' ||v_result_list(i).cost);

END LOOP;

END;

Project #: 1001Title: Travel MonitorCost: 400000Project #: 1002Title: Open WorldCost: 10000000

Page 34: Advanced PL/SQL Optimizing for Better Performance 2016

39

Using Collection Methods• EXISTS

• COUNT

• LIMIT

• FIRST and LAST• PRIOR and NEXT• EXTEND

• TRIM

• DELETE

collection_name.method_name [(parameters)]

Page 35: Advanced PL/SQL Optimizing for Better Performance 2016

41

FUNCTION check_costs (p_project_list typ_projectlist)RETURN boolean

ISc_max_allowed NUMBER := 10000000;i INTEGER;v_flag BOOLEAN := FALSE;

BEGINi := p_project_list.FIRST ; WHILE i IS NOT NULL LOOPIF p_project_list(i).cost > c_max_allowed thenv_flag := TRUE;dbms_output.put_line (p_project_list(i).title || '

exceeded allowable budget.');RETURN TRUE;

END IF;i := p_project_list.NEXT(i); END LOOP;RETURN null;

END check_costs;

Using Collection MethodsTraverse collections with the following methods:

Page 36: Advanced PL/SQL Optimizing for Better Performance 2016

42

Using Collection Methods-- sample caller program to check_costsset serverout onDECLAREv_project_list typ_projectlist;

BEGINv_project_list := typ_ProjectList(typ_Project (1,'Dsgn New Expense Rpt', 3250),typ_Project (2, 'Outsource Payroll', 120000),typ_Project (3, 'Audit Accounts Payable',14250000));

IF manage_dept_proj.check_costs(v_project_list) THENdbms_output.put_line('Project rejected: overbudget');

ELSEdbms_output.put_line('Project accepted, fill out forms.');

END IF;END;

Audit Accounts Payable exceeded allowable budget.Project rejected: overbudget

PROJECT_NO TITLE COSTS

1 Dsgn New Expense Rpt 3250

2 Outsource Payroll 120000

3 Audit Accounts Payable 14250000

V_PROJECT_LIST variable:

Page 37: Advanced PL/SQL Optimizing for Better Performance 2016

43

PROCEDURE update_a_project(p_deptno NUMBER, p_new_project typ_Project, p_position NUMBER)

ISv_my_projects typ_ProjectList;

BEGINv_my_projects := get_dept_project (p_deptno);v_my_projects.EXTEND; --make room for new project/* Move varray elements forward */FOR i IN REVERSE p_position..v_my_projects.LAST - 1 LOOPv_my_projects(i + 1) := v_my_projects(i);

END LOOP;v_my_projects(p_position) := p_new_project; -- insert new oneUPDATE department SET projects = v_my_projectsWHERE dept_id = p_deptno;

END update_a_project;

Manipulating Individual Elements

Page 38: Advanced PL/SQL Optimizing for Better Performance 2016

44

Manipulating Individual Elements-- check the table prior to the update:SELECT d2.dept_id, d2.name, d1.*FROM department d2, TABLE(d2.projects) d1;

DEPT_ID NAME PROJECT_NO TITLE COST ------- ------------------------- ---------- ----------------------------- --

10 Executive Administration 1001 Travel Monitor 400000 10 Executive Administration 1002 Open World 10000000 20 Information Technology 2001 DB11gR2 900000

-- caller program to update_a_projectBEGINmanage_dept_proj.update_a_project(20,typ_Project(2002, 'AQM', 80000), 2);

END;

DEPT_ID NAME PROJECT_NO TITLE COST ------- ------------------------- ---------- ----------------------------- --

10 Executive Administration 1001 Travel Monitor 400000 10 Executive Administration 1002 Open World 10000000 20 Information Technology 2001 DB11gR2 90000020 Information Technology 2002 AQM 80000

-- check the table after the update:SELECT d2.dept_id, d2.name, d1.*FROM department d2, TABLE(d2.projects) d1;

Page 39: Advanced PL/SQL Optimizing for Better Performance 2016

45

Listing Characteristics for Collections

PL/SQL

Nested

Tables

DB

Nested

Tables

PL/SQL

Varrays

DB

Varrays

PL/SQL

Associative

Arrays

Maximum

size

No No Yes Yes Dynamic

Sparsity Can be No Dense Dense Yes

Storage N/A Stored out-of-

line

N/A Stored inline

(if < 4,000

bytes)

N/A

Ordering Does not retain ordering and subscripts

Does not

retain ordering

and subscripts

Retains ordering and subscripts

Retains

ordering and

subscripts

Retains

ordering and

subscripts

Page 40: Advanced PL/SQL Optimizing for Better Performance 2016

46

Guidelines for Using Collections Effectively• Varrays involve fewer disk accesses and are more efficient.

• Use nested tables for storing large amounts of data.

• Use varrays to preserve the order of elements in the collection column.

• If you do not have a requirement to delete elements in the middle of a collection, favor varrays.

• Varrays do not allow piecewise updates.

• After deleting the elements, release the unused memory with DBMS_SESSION.FREE_UNUSED_USER_MEMORY

Page 41: Advanced PL/SQL Optimizing for Better Performance 2016

Advanced Cursors

47

Page 42: Advanced PL/SQL Optimizing for Better Performance 2016

48

Cursor Design: Use Records• Fetch into a record when fetching from a cursor.

• Benefit

– No individual variables declaration is needed.

– You can automatically use the structure of the SELECT column list.

DECLARE

CURSOR cur_cust IS

SELECT customer_id, cust_last_name, cust_email

FROM customers WHERE credit_limit = 1200;

v_cust_record cur_cust%ROWTYPE;

BEGIN

OPEN cur_cust;

LOOP

FETCH cur_cust INTO v_cust_record;

...

Page 43: Advanced PL/SQL Optimizing for Better Performance 2016

49

Guidelines for Good Cursor Design• Reference implicit cursor attributes immediately after the SQL

statement executes.

• Benefit– Doing so ensures that you are dealing with the result of the

correct SQL statement.

BEGIN

UPDATE customers

SET credit_limit = p_credit_limit

WHERE customer_id = p_cust_id;

get_avg_order(p_cust_id); -- procedure call

IF SQL%NOTFOUND THEN

...

``

Page 44: Advanced PL/SQL Optimizing for Better Performance 2016

50

Use Cursor Parameters• Create cursors with parameters.

• Benefit– Parameters increase the cursor’s flexibility and reusability.– Parameters help avoid scoping problems.

CURSOR cur_cust(p_crd_limit NUMBER, p_acct_mgr NUMBER)

ISSELECT customer_id, cust_last_name, cust_emailFROM customersWHERE credit_limit = p_crd_limitAND account_mgr_id = p_acct_mgr;

BEGINOPEN cur_cust(p_crd_limit_in, p_acct_mgr_in);

...CLOSE cur_cust;

...OPEN cur_cust(v_credit_limit, 145);

...END;

Page 45: Advanced PL/SQL Optimizing for Better Performance 2016

51

Using the Cursor For Loop• Simplify coding with cursor FOR loops.

• Benefit

– Reduces the volume of code

– Automatically handles the open, fetch, and close operations, and defines a record type that matches the cursor definition

CREATE OR REPLACE PROCEDURE cust_pack(p_crd_limit_in NUMBER, p_acct_mgr_in NUMBER)ISv_credit_limit NUMBER := 1500;CURSOR cur_cust

(p_crd_limit NUMBER, p_acct_mgr NUMBER)ISSELECT customer_id, cust_last_name, cust_emailFROM customers WHERE credit_limit = p_crd_limitAND account_mgr_id = p_acct_mgr;

BEGINFOR cur_rec IN cur_cust (p_crd_limit_in, p_acct_mgr_in)LOOP -- implicit open and fetch

...

END LOOP; -- implicit close...

END;

Page 46: Advanced PL/SQL Optimizing for Better Performance 2016

52

CREATE OR REPLACE PROCEDURE cust_list

IS

CURSOR cur_cust IS

SELECT customer_id, cust_last_name, credit_limit*1.1

FROM customers;

cust_record cur_cust%ROWTYPE;

BEGIN

OPEN cur_cust;

LOOP

FETCH cur_cust INTO cust_record;

DBMS_OUTPUT.PUT_LINE('Customer ' ||

cust_record.cust_last_name || ' wants credit '

|| cust_record.(credit_limit * 1.1));

EXIT WHEN cur_cust%NOTFOUND;

END LOOP;

...

More Guidelines for Good Cursor Design• Make a DBA happy: Close a cursor when it is no longer needed.

• Use column aliases in cursors for calculated columns fetched into records declared with %ROWTYPE.

Use col. alias

Page 47: Advanced PL/SQL Optimizing for Better Performance 2016

53

Returning Result Sets From PL/SQL• A Ref Cursor is a Cursor variable

• It hold a pointer to the result set of a previously opened cursor

• The actual SQL statement of the cursor is dynamic and determined at execution time

• A single Ref Cursor can point to different result sets at different times

53

Page 48: Advanced PL/SQL Optimizing for Better Performance 2016

54

Memory

Cursor Variables: Overview

1 Southlake, Texas 1400

2 San Francisco 1500

3 New Jersey 1600

4 Seattle, Washington 1700

5 Toronto 1800

REF CURSORmemorylocator

Page 49: Advanced PL/SQL Optimizing for Better Performance 2016

55

Working with Cursor Variables

Define and declare the cursor variable.

Open the cursor variable.

Fetch rows from the result set.

Close the cursor variable.

1 2 3 4

Page 50: Advanced PL/SQL Optimizing for Better Performance 2016

56

Strong Versus Weak REF CURSOR Variables• Strong REF CURSOR:

– Is restrictive

– Specifies a RETURN type

– Associates only with type-compatible queries

– Is less error prone

• Weak REF CURSOR:– Is nonrestrictive

– Associates with any query

– Is very flexible

Page 51: Advanced PL/SQL Optimizing for Better Performance 2016

57

DECLARE

TYPE rt_cust IS REF CURSOR

RETURN customers%ROWTYPE;

...

Step 1: Defining a REF CURSOR TypeDefine a REF CURSOR type:

– ref_type_name is a type specified in subsequent declarations.

– return_type represents a record type.

– RETURN keyword indicates a strong cursor.

TYPE ref_type_name IS REF CURSOR

[RETURN return_type];

Page 52: Advanced PL/SQL Optimizing for Better Performance 2016

58

Step 1: Declaring a Cursor VariableDeclare a cursor variable of a cursor type:

– cursor_variable_name is the name of the cursor variable.

– ref_type_name is the name of a REF CURSOR type.

DECLARE

TYPE rt_cust IS REF CURSOR

RETURN customers%ROWTYPE;

cv_cust rt_cust;

cursor_variable_name ref_type_name;

Page 53: Advanced PL/SQL Optimizing for Better Performance 2016

59

Step 1: Declaring a REF CURSOR Return TypeOptions:

– Use %TYPE and %ROWTYPE.

– Specify a user-defined record in the RETURN clause.

– Declare the cursor variable as the formal parameter of a stored procedure or function.

Page 54: Advanced PL/SQL Optimizing for Better Performance 2016

60

Step 2: Opening a Cursor Variable• Associate a cursor variable with a multiple-row SELECT statement.

• Execute the query.

• Identify the result set:

– cursor_variable_name is the name of thecursor variable.

– select_statement is the SQL SELECT statement.

OPEN cursor_variable_name

FOR select_statement;

Page 55: Advanced PL/SQL Optimizing for Better Performance 2016

62

Step 3: Fetching from a Cursor Variable• Retrieve rows from the result set one at a time.

• The return type of the cursor variable must be compatible with the variables named in the INTOclause of the FETCH statement.

FETCH cursor_variable_name

INTO variable_name1

[,variable_name2,. . .]

| record_name;

Page 56: Advanced PL/SQL Optimizing for Better Performance 2016

63

Step 4: Closing a Cursor Variable• Disable a cursor variable.

• The result set is undefined.

• Accessing the cursor variable after it is closed raises the INVALID_CURSOR predefined exception.

CLOSE cursor_variable_name;

Page 57: Advanced PL/SQL Optimizing for Better Performance 2016

64

Passing Cursor Variables as ArgumentsYou can pass query result sets among PL/SQL-stored subprograms and various clients.

Pointer to the result

set

Access by a host variable on the client side

Page 58: Advanced PL/SQL Optimizing for Better Performance 2016

65

Passing Cursor Variables as Arguments

Page 59: Advanced PL/SQL Optimizing for Better Performance 2016

67

Using the SYS_REFCURSOR Predefined Type CREATE OR REPLACE PROCEDURE REFCUR

(p_num IN NUMBER)

IS

refcur sys_refcursor;

empno emp.empno%TYPE;

ename emp.ename%TYPE;

BEGIN

IF p_num = 1 THEN

OPEN refcur FOR SELECT empno, ename FROM emp;

DBMS_OUTPUT.PUT_LINE('Employee# Name');

DBMS_OUTPUT.PUT_LINE('----- -------');

LOOP

FETCH refcur INTO empno, ename;

EXIT WHEN refcur%NOTFOUND;

DBMS_OUTPUT.PUT_LINE(empno || ' ' || ename);

END LOOP;

ELSE

....

SYS_REFCURSOR is a built-in REF CURSOR type that allows any result set to be associated with it.

Page 60: Advanced PL/SQL Optimizing for Better Performance 2016

69

Rules for Cursor Variables• You cannot use cursor variables with remote

subprograms on another server.

• You cannot use comparison operators to test cursor variables.

• You cannot assign a null value to cursor variables.

• You cannot use REF CURSOR types in CREATE TABLE or VIEW statements.

• Cursors and cursor variables are not interoperable.

Page 61: Advanced PL/SQL Optimizing for Better Performance 2016

70

Comparing Cursor Variables with Static CursorsCursor variables have the following benefits:

– Are dynamic and ensure more flexibility

– Are not tied to a single SELECT statement

– Hold the value of a pointer

– Can reduce network traffic

– Give access to query work areas after ablock completes

Page 62: Advanced PL/SQL Optimizing for Better Performance 2016

Ref Cursor Demo

71

Page 63: Advanced PL/SQL Optimizing for Better Performance 2016

72

Using the For Update Clause• Use explicit locking to deny access to other

sessions for the duration of a transaction

• Lock the rows before the update or delete

• Syntax:

72

SELECT ...

FROM ...

FOR UPDATE [OF column_reference][NOWAIT | WAIT n];

Page 64: Advanced PL/SQL Optimizing for Better Performance 2016

73

Handling Locked Rows• Select for Update default is to wait for locked rows

• We can set the wait time to be limited with a time frame or fail if rows are already locked

• We can also ask the query to skip the locked rows and return the unlocked rows:

73

SELECT ...

FROM ...

FOR UPDATE SKIP LOCKED;

Page 65: Advanced PL/SQL Optimizing for Better Performance 2016

74

Using WHERE CURRENT OF Clause

• When using cursor with FOR UPDATE clause, we might want to change that same record

• We can use the WHERE CURRENT OF cursor to get a fast ROWID access to that row

74

UPDATE employees

SET salary = 12000

WHERE CURRENT OF emp_cursor;

Page 66: Advanced PL/SQL Optimizing for Better Performance 2016

75

Using the RETURNING Clause

• Include the RETURNING clause with an INSERT, UPDATE, and DELETE to return column values.

• Benefit:

– Eliminates the need to SELECT the row after DML

– Fewer network round trips

– Less server CPU time

– Fewer cursors

– Less server memory is required

75

Page 67: Advanced PL/SQL Optimizing for Better Performance 2016

76

Using RETURNING Clause Example

76

DECLARE

v_emp_sal employees.salary%type;

BEGIN

UPDATE employees e

set salary = salary * 1.2

WHERE e.employee_id = 202

RETURNING e.salary into v_emp_sal;

dbms_output.put_line('The new salary is ' || v_emp_sal);

END;

/

The new salary is 7200

Page 68: Advanced PL/SQL Optimizing for Better Performance 2016

77

Using Bulk BindingUse bulk binds to reduce context switches between the PL/SQL engine and the SQL engine.

SQL enginePL/SQL run-time engine

PL/SQL block

FORALL j IN 1..1000INSERT … (OrderId(j),

OrderDate(j), …);

SQL statement executor

Procedural statement executor

Page 69: Advanced PL/SQL Optimizing for Better Performance 2016

78

Using Bulk BindingBind whole arrays of values simultaneously, rather than looping to perform fetch, insert, update, and delete on multiple rows.

– Instead of:

– Use:

...FOR i IN 1 .. 50000 LOOP

INSERT INTO bulk_bind_example_tblVALUES(...);

END LOOP; ...

...FORALL i IN 1 .. 50000

INSERT INTO bulk_bind_example_tblVALUES(...);

...

Page 70: Advanced PL/SQL Optimizing for Better Performance 2016

80

Using Bulk BindingUse BULK COLLECT to improve performance:

CREATE OR REPLACE PROCEDURE process_customers(p_account_mgr customers.account_mgr_id%TYPE)

ISTYPE typ_numtab IS TABLE OF

customers.customer_id%TYPE;TYPE typ_chartab IS TABLE OF

customers.cust_last_name%TYPE;TYPE typ_emailtab IS TABLE OF

customers.cust_email%TYPE;v_custnos typ_numtab;v_last_names typ_chartab;v_emails typ_emailtab;

BEGINSELECT customer_id, cust_last_name, cust_email

BULK COLLECT INTO v_custnos, v_last_names, v_emailsFROM customersWHERE account_mgr_id = p_account_mgr;

...END process_customers;

Page 71: Advanced PL/SQL Optimizing for Better Performance 2016

81

Using Bulk BindingUse the RETURNING clause to retrieve information about the rows that are being modified:

DECLARE

TYPE typ_replist IS VARRAY(100) OF NUMBER;

TYPE typ_numlist IS TABLE OF

orders.order_total%TYPE;

repids typ_replist :=

typ_replist(153, 155, 156, 161);

totlist typ_numlist;

c_big_total CONSTANT NUMBER := 60000;

BEGIN

FORALL i IN repids.FIRST..repids.LAST

UPDATE orders

SET order_total = .95 * order_total

WHERE sales_rep_id = repids(i)

AND order_total > c_big_total

RETURNING order_total BULK COLLECT INTO Totlist;

END;

Page 72: Advanced PL/SQL Optimizing for Better Performance 2016

83

Using SAVE EXCEPTIONS• You can use the SAVE EXCEPTIONS keyword in your FORALL

statements:

• Exceptions raised during execution are saved in the %BULK_EXCEPTIONS cursor attribute.

• The attribute is a collection of records with two fields:

– Note that the values always refer to the most recently executed FORALL statement.

FORALL index IN lower_bound..upper_bound

SAVE EXCEPTIONS

{insert_stmt | update_stmt | delete_stmt}

Field Definition

ERROR_INDEX Holds the iteration of the FORALL statement where the

exception was raised

ERROR_CODE Holds the corresponding Oracle error code

Page 73: Advanced PL/SQL Optimizing for Better Performance 2016

84

Handling FORALL ExceptionsDECLARE

TYPE NumList IS TABLE OF NUMBER;

num_tab NumList :=

NumList(100,0,110,300,0,199,200,0,400);

bulk_errors EXCEPTION;

PRAGMA EXCEPTION_INIT (bulk_errors, -24381 );

BEGIN

FORALL i IN num_tab.FIRST..num_tab.LAST

SAVE EXCEPTIONS

DELETE FROM orders WHERE order_total < 500000/num_tab(i);

EXCEPTION WHEN bulk_errors THEN

DBMS_OUTPUT.PUT_LINE('Number of errors is: '

|| SQL%BULK_EXCEPTIONS.COUNT);

FOR j in 1..SQL%BULK_EXCEPTIONS.COUNT

LOOP

DBMS_OUTPUT.PUT_LINE (

TO_CHAR(SQL%BULK_EXCEPTIONS(j).error_index) ||

' / ' ||

SQLERRM(-SQL%BULK_EXCEPTIONS(j).error_code) );

END LOOP;

END;

/

Page 74: Advanced PL/SQL Optimizing for Better Performance 2016

Dynamic SQL and SQL Injection

85

Page 75: Advanced PL/SQL Optimizing for Better Performance 2016

86

Dynamic SQL• Used for two main reasons:

– Modify commands dynamically to retrieve or filter columns based on input

– Run DDL and DCL commands from PL/SQL

– PLEASE don’t use it for running DMLs!

• Two main ways to run dynamic commands:

– EXECUTE IMMDIATE

– DBMS_SQL

86

Page 76: Advanced PL/SQL Optimizing for Better Performance 2016

87

Advantages of Native Dynamic SQL• Easy syntax:

• Accepts all command types

• Accept bind variables with the USING clause

• Allows dynamic PL/SQL creation

87

EXECUTE_IMMEDIATE dynamic_string

{

INTO { define_variable [, define_variable ...] | record_name }

| BULK COLLECT INTO { collection_name [, collection_name ...] |

:host_array_name }

}

[ USING [ IN | OUT | IN OUT ] bind_argument

[, [ IN | OUT | IN OUT ] bind_argument] ... ] [ returning_clause ] ;

Page 77: Advanced PL/SQL Optimizing for Better Performance 2016

88

Advantages of DBMS_SQL

• DBMS_SQL supports statements with unknown number of inputs or outputs

• We can use DESCRIBE_COLUMNS procedure in the DBMS_SQL package to describe columns for a cursor opened/parsed through DBMS_SQL

• DBMS_SQL Supports SQL statements larger than 32 KB

• DBMS_SQL Lets you reuse SQL statements

88

Page 78: Advanced PL/SQL Optimizing for Better Performance 2016

89

Using Execute Immediate

89

CREATE OR REPLACE PROCEDURE del_rows

(p_condition varchar2, p_rows_deld out number)

IS

BEGIN

EXECUTE IMMEDIATE 'DELETE FROM employees ' || p_condition;

p_rows_deld:=sql%ROWCOUNT;

END;

/

DECLARE

cnt number;

BEGIN

del_rows(‘where employee_id = 201’, cnt);

END;

/

Page 79: Advanced PL/SQL Optimizing for Better Performance 2016

90

Using DBMS_SQL

90

CREATE OR REPLACE

PROCEDURE del_rows

(p_condition in varchar2, p_rows_del out number)

is

cursor_id integer;

BEGIN

cursor_id := dbms_sql.open_cursor;

dbms_sql.parse (cursor_id,'DELETE FROM employees ‘ ||p_condition,

dbms_sql.native);

p_rows_del := dbms_sql.execute (cursor_id);

dbms_sql.close_cursor (cursor_id);

END;

/

Page 80: Advanced PL/SQL Optimizing for Better Performance 2016

91

Transforming DBMS_SQLCursor into a REF CURSOR

• Starting Oracle 11g, we can transform a DBMS_SQL cursor into a PL/SQL REF CURSOR and vice versa

91

DBMS_SQL.TO_REFCURSOR (cursor_number IN INTEGER) RETURN SYS_REFCURSOR;

DBMS_SQL.TO_CURSOR_NUMBER (rc IN OUT SYS_REFCURSOR) RETURN INTEGER;

Page 81: Advanced PL/SQL Optimizing for Better Performance 2016

92

Disadvantages of Dynamic SQL• Dynamic SQL is not checked until runtime

– Syntax

– Structure validity

– Permissions

• Dynamic SQL can be used as SQL injection entry point

92

Page 82: Advanced PL/SQL Optimizing for Better Performance 2016

93

Understanding SQL InjectionSQL injection is a technique for maliciously exploiting applications that use client-supplied data in SQL statements.

– Attackers trick the SQL engine into executing unintended commands.

– SQL injection techniques may differ, but they all exploit a single vulnerability in the application.

– To immunize your code against SQL injection attacks, use bind arguments or validate and sanitize all input concatenated to dynamic SQL.

Page 83: Advanced PL/SQL Optimizing for Better Performance 2016

94

SQL Injection: Example-- First order attackCREATE OR REPLACE PROCEDURE GET_EMAIL

(p_last_name VARCHAR2 DEFAULT NULL)AS

TYPE cv_custtyp IS REF CURSOR;cv cv_custtyp;v_email customers.cust_email%TYPE;v_stmt VARCHAR2(400);

BEGINv_stmt := 'SELECT cust_email FROM customers

WHERE cust_last_name = '''|| p_last_name || '''';DBMS_OUTPUT.PUT_LINE('SQL statement: ' || v_stmt); OPEN cv FOR v_stmt;LOOP

FETCH cv INTO v_email;EXIT WHEN cv%NOTFOUND;DBMS_OUTPUT.PUT_LINE('Email: '||v_email);

END LOOP;CLOSE cv;

EXCEPTION WHEN OTHERS THENdbms_output.PUT_LINE(sqlerrm);dbms_output.PUT_LINE('SQL statement: ' || v_stmt);

END;

String literals that are incorrectly validated or not validated are concatenated into a dynamic SQL statement, and interpreted as code by the SQL engine.

Page 84: Advanced PL/SQL Optimizing for Better Performance 2016

95

Assessing VulnerabilityCode

Uses bind arguments for all

dynamic components

VulnerableSafe

Contains dynamic SQL?

Filters input correctly?

No

No

No

Yes

Yes

YesLateral injection vulnerability

No

Page 85: Advanced PL/SQL Optimizing for Better Performance 2016

96

Reducing the Attack Surface Use the following strategies to reduce attack surface:

– Use invoker’s rights.

– Reduce arbitrary inputs.

– Use Bind arguments.

– The Filter pitfall.

Page 86: Advanced PL/SQL Optimizing for Better Performance 2016

97

Avoiding Privilege Escalation• Give out privileges appropriately.

• Run code with invoker’s rights when possible.

• Ensure that the database privilege model is upheld when using definer’s rights.

Invoker’s rights

Definer’s rights

Page 87: Advanced PL/SQL Optimizing for Better Performance 2016

98

Using Invoker’s Rights• Using invoker’s rights:

– Helps to limit the privileges– Helps to minimize the security exposure.

• The following example does not use invoker's rights:CREATE OR REPLACEPROCEDURE change_password(p_username VARCHAR2 DEFAULT NULL,

p_new_password VARCHAR2 DEFAULT NULL)ISv_sql_stmt VARCHAR2(500);

BEGINv_sql_stmt := 'ALTER USER '||p_username ||' IDENTIFIED BY '

|| p_new_password;EXECUTE IMMEDIATE v_sql_stmt;

END change_password;

GRANT EXECUTE ON change_password to OE, HR, SH;

1

2

Note the use of dynamic SQL with concatenated input values.

Page 88: Advanced PL/SQL Optimizing for Better Performance 2016

99

Using Invoker’s Rights• OE is successful at changing the SYS password, because,

by default, CHANGE_PASSWORD executes with SYSprivileges:

• Add the AUTHID to change the privileges to the invokers:

EXECUTE sys.change_password ('SYS', 'mine')

CREATE OR REPLACEPROCEDURE change_password(p_username VARCHAR2 DEFAULT NULL,

p_new_password VARCHAR2 DEFAULT NULL)AUTHID CURRENT_USERISv_sql_stmt VARCHAR2(500);

BEGINv_sql_stmt := 'ALTER USER '||p_username ||' IDENTIFIED BY '

|| p_new_password;EXECUTE IMMEDIATE v_sql_stmt;

END change_password;

Page 89: Advanced PL/SQL Optimizing for Better Performance 2016

100

Reducing Arbitrary Inputs• Reduce the end-user interfaces to only those that are

actually needed. – In a Web application, restrict users to accessing specified Web

pages. – In a PL/SQL API, expose only those routines that are intended for

customer use.

• Where user input is required, make use of language features to ensure that only data of the intended type can be specified. – Do not specify a VARCHAR2 parameter when it will be used as a

number. – Do not use numbers if you need only positive integers; use

natural instead.

Page 90: Advanced PL/SQL Optimizing for Better Performance 2016

101

Understanding DBMS_ASSERTDBMS_ASSERT functions:

Function Description

ENQUOTE_LITERAL Encloses string literal in single quotes

SIMPLE_SQL_NAME Verifies that the string is a simple SQL name

Page 91: Advanced PL/SQL Optimizing for Better Performance 2016

102

Formatting Oracle Identifiers– Example 1: The object name used as an identifier:

SELECT count(*) records FROM orders;

– Example 2: The object name used as a literal:SELECT num_rows FROM user_tables

WHERE table_name = 'ORDERS';

– Example 3: The object name used as a quoted (normal format) identifier:

• The "orders" table referenced in example 3 is a different table compared to the orders table in examples 1 and 2.

• It is vulnerable to SQL injection.

SELECT count(*) records FROM "orders";

Page 92: Advanced PL/SQL Optimizing for Better Performance 2016

103

Working with Identifiers in Dynamic SQL• For your identifiers, determine:

1.Where will the input come from: user or data dictionary?

2.What verification is required?

3.How will the result be used, as an identifier or a literal value?

• These three factors affect:– What preprocessing is required (if any) prior to calling the

verification functions

– Which DBMS_ASSERT verification function is required

– What post-processing is required before the identifier can actually be used

Page 93: Advanced PL/SQL Optimizing for Better Performance 2016

104

Avoiding Injection by Using DBMS_ASSERT.ENQUOTE_LITERAL

CREATE OR REPLACE PROCEDURE Count_Rows(w in varchar2) authid definer asQuote constant varchar2(1) := '''';Quote_Quote constant varchar2(2) := Quote||Quote;Safe_Literal varchar2(32767) :=

Quote||replace(w,Quote,Quote_Quote)||Quote;Stmt constant varchar2(32767) :=

'SELECT count(*) FROM t WHERE a='||DBMS_ASSERT.ENQUOTE_LITERAL(Safe_Literal);

Row_Count number;BEGIN

EXECUTE IMMEDIATE Stmt INTO Row_Count;DBMS_OUTPUT.PUT_LINE(Row_Count||' rows');

END;/

Verify whether the literal is well-formed.

Page 94: Advanced PL/SQL Optimizing for Better Performance 2016

107

Avoiding Injection by Using DBMS_ASSERT.SIMPLE_SQL_NAME

CREATE OR REPLACE PROCEDURE show_col2 (p_colname varchar2, p_tablename varchar2)AStype t is varray(200) of varchar2(25);Results t; Stmt CONSTANT VARCHAR2(4000) :=

'SELECT '||dbms_assert.simple_sql_name( p_colname ) || ' FROM '|| dbms_assert.simple_sql_name( p_tablename ) ;

BEGINDBMS_Output.Put_Line ('SQL Stmt: ' || Stmt);

EXECUTE IMMEDIATE Stmt bulk collect into Results;for j in 1..Results.Count() loopDBMS_Output.Put_Line(Results(j));end loop;END show_col2;

Verify that the input string conforms to the basic characteristics of a simple

SQL name.

Page 95: Advanced PL/SQL Optimizing for Better Performance 2016

109

Using Bind Arguments• Most common vulnerability:

– Dynamic SQL with string concatenation

• Your code design must:

– Avoid input string concatenation in dynamic SQL

– Use bind arguments, whether automatically via static SQL or explicitly via dynamic SQL statements

Page 96: Advanced PL/SQL Optimizing for Better Performance 2016

110

Beware of Filter Parameters• Filter parameter:

– P_WHERE_CLAUSE is a filter.

– It is difficult to protect against SQL injection.

• Prevention methods:– Do not specify APIs that allow arbitrary query parameters

to be exposed.

– Any existing APIs with this type of functionality must be deprecated and replaced with safe alternatives.

stmt := 'SELECT session_id FROM sessions WHERE' || p_where_clause;

Page 97: Advanced PL/SQL Optimizing for Better Performance 2016

111

Autonomous Transactions• An Autonomous Transaction is an independent

Transaction started by another transaction.

• The main transaction will hold for the AT and wait until it is completed

• Uses PRAGMA compiler directive

• Allowed in individual routines

• Commonly used for loggers, progress bars, and concurrent operations

111

Page 98: Advanced PL/SQL Optimizing for Better Performance 2016

112

AT Example: Main Transaction

112

DECLARE

tmp NUMBER;

BEGIN

FOR i IN 1..10

LOOP

tmp := i;

INSERT INTO t

VALUES (TRUNC(i/2));

END LOOP;

EXCEPTION

WHEN DUP_VAL_ON_INDEX THEN

logger('Failed on ' || TO_CHAR(tmp));

ROLLBACK;

RAISE;

END;

/

Page 99: Advanced PL/SQL Optimizing for Better Performance 2016

113

AT Example: Logger (AT)

113

PROCEDURE logger (message IN VARCHAR2) IS

PRAGMA AUTONOMOUS_TRANSACTION

BEGIN

INSERT INTO logger VALUES (sysdate, message);

COMMIT;

END;

/

Page 100: Advanced PL/SQL Optimizing for Better Performance 2016

New and Useful Features for Developers

11gR2, 12cR1 and 12cR2

114

Page 101: Advanced PL/SQL Optimizing for Better Performance 2016

Features in 11gR2

115

Page 102: Advanced PL/SQL Optimizing for Better Performance 2016

116

Using the LISTAGG Function

• For a specified measure, LISTAGG orders data within each group specified in the ORDER BY clause and then concatenates the values of the measure column

• Limited to 4000 chars (in 11g, see 12cR2enhancement!)

LISTAGG(measure_expr [, 'delimiter'])WITHIN GROUP (order_by_clause) [OVER query_partition_clause]

Page 103: Advanced PL/SQL Optimizing for Better Performance 2016

117

Using LISTAGG: ExampleSELECT department_id "Dept", hire_date

"Date", last_name "Name", LISTAGG(last_name, ', ') WITHIN GROUP

(ORDER BY hire_date, last_name)OVER (PARTITION BY department_id) as

"Emp_list"FROM hr.employeesWHERE hire_date < '01-SEP-2003'ORDER BY "Dept", "Date", "Name";

Page 104: Advanced PL/SQL Optimizing for Better Performance 2016

118

The NTH_VALUE Analytic Function

• Returns the N-th values in an ordered set of values

• Different default window: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT

ROW

118

NTH_VALUE (measure_expr, n)[ FROM { FIRST | LAST } ][ { RESPECT | IGNORE } NULLS ] OVER (analytic_clause)

Page 105: Advanced PL/SQL Optimizing for Better Performance 2016

119

Using NTH_VALUE: Example

SELECT prod_id, channel_id, MIN(amount_sold),

NTH_VALUE ( MIN(amount_sold), 2) OVER (PARTITION BY

prod_id ORDER BY channel_id

ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED

FOLLOWING) nv

FROM sh.sales

WHERE prod_id BETWEEN 13 and 16

GROUP BY prod_id, channel_id;

Page 106: Advanced PL/SQL Optimizing for Better Performance 2016

120

Using NTH_VALUE: Example

SELECT prod_id, channel_id, MIN(amount_sold),

NTH_VALUE ( MIN(amount_sold), 2) OVER (PARTITION BY

prod_id ORDER BY channel_id

ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED

FOLLOWING) nv

FROM sh.sales

WHERE prod_id BETWEEN 13 and 16

GROUP BY prod_id, channel_id;

Page 107: Advanced PL/SQL Optimizing for Better Performance 2016

121

Virtual Columns• Virtual columns are dynamic (not stored) columns of a table

(no views required)• Virtual columns obtain their value be evaluating as

expression:– Columns from the same table– Constraints– Function Calls (user defined)

• Might be used for– Eliminate views– Control table partitioning– Manage “Binary” XMLType data– Index values (function based index)

121

Page 108: Advanced PL/SQL Optimizing for Better Performance 2016

122

Virtual Columns Example

122

column_name [datatype] [GENERATED ALWAYS] AS (expression) [VIRTUAL]

CREATE TABLE employees (id NUMBER,first_name VARCHAR2(10),last_name VARCHAR2(10),salary NUMBER(9,2),comm1 NUMBER(3),comm2 NUMBER(3),salary1 AS (ROUND(salary*(1+comm1/100),2)),salary2 NUMBER GENERATED ALWAYS AS (ROUND(salary*(1+comm2/100),2)) VIRTUAL,

CONSTRAINT employees_pk PRIMARY KEY (id));

Page 109: Advanced PL/SQL Optimizing for Better Performance 2016

Trigger Improvements

123

Page 110: Advanced PL/SQL Optimizing for Better Performance 2016

124

What Is a Compound Trigger? • A single trigger on a table that allows you to

specify actions for each of the following four timing points:

– Before the firing statement

– Before each row that the firing statement affects

– After each row that the firing statement affects

– After the firing statement

Page 111: Advanced PL/SQL Optimizing for Better Performance 2016

125

Working with Compound Triggers• The compound trigger body supports a common

PL/SQL state that the code for each timing point can access.

• The compound trigger common state is:

– Established when the triggering statement starts

– Destroyed when the triggering statement completes

• A compound trigger has a declaration section and a section for each of its timing points.

Page 112: Advanced PL/SQL Optimizing for Better Performance 2016

126

The Benefits of Using a Compound Trigger• You can use compound triggers to:

– Program an approach where you want the actions you implement for the various timing points to share common data.

– Accumulate rows destined for a second table so that you can periodically bulk-insert them

– Avoid the mutating-table error (ORA-04091)by allowing rows destined for a second table to accumulate and then bulk-inserting them

Page 113: Advanced PL/SQL Optimizing for Better Performance 2016

127

Timing-Point Sections of a Table Compound Trigger

• A compound trigger defined on a table has one or more of the following timing-point sections. Timing-point sections must appear in the order shown in the table.

Timing Point Compound Trigger Section

Before the triggering statement executes BEFORE statement

After the triggering statement executes AFTER statement

Before each row that the triggering statement affects BEFORE EACH ROW

After each row that the triggering statement affects AFTER EACH ROW

Page 114: Advanced PL/SQL Optimizing for Better Performance 2016

128

Compound Trigger Structure for TablesCREATE OR REPLACE TRIGGER schema.triggerFOR dml_event_clause ON schema.tableCOMPOUND TRIGGER

-- Initial section -- Declarations -- Subprograms

-- Optional section BEFORE STATEMENT IS ...;

-- Optional sectionBEFORE EACH ROW IS ...;

-- Optional section AFTER EACH ROW IS ...;

-- Optional section AFTER STATEMENT IS ...;

1

2

Page 115: Advanced PL/SQL Optimizing for Better Performance 2016

129

Trigger Restrictions on Mutating Tables• A mutating table is:

– A table that is being modified by an UPDATE, DELETE, or INSERTstatement, or

– A table that might be updated by the effects of a DELETE CASCADEconstraint

• The session that issued the triggering statement cannot query or modify a mutating table.

• This restriction prevents a trigger from seeing an inconsistent set of data.

• This restriction applies to all triggers that use the FOR EACHROW clause.

• Views being modified in the INSTEAD OF triggers are not considered mutating.

Page 116: Advanced PL/SQL Optimizing for Better Performance 2016

130

Using a Compound Trigger to Resolve the Mutating Table Error

CREATE OR REPLACE TRIGGER check_salaryFOR INSERT OR UPDATE OF salary, job_idON employeesWHEN (NEW.job_id <> 'AD_PRES')COMPOUND TRIGGER

TYPE salaries_t IS TABLE OF employees.salary%TYPE;min_salaries salaries_t;max_salaries salaries_t;

TYPE department_ids_t IS TABLE OF employees.department_id%TYPE;department_ids department_ids_t;

TYPE department_salaries_t IS TABLE OF employees.salary%TYPEINDEX BY VARCHAR2(80);

department_min_salaries department_salaries_t;department_max_salaries department_salaries_t;

-- example continues on next slide

Page 117: Advanced PL/SQL Optimizing for Better Performance 2016

131

Using a Compound Trigger to Resolve the Mutating Table Error

. . .BEFORE STATEMENT ISBEGINSELECT MIN(salary), MAX(salary), NVL(department_id, -1)BULK COLLECT INTO min_Salaries, max_salaries, department_idsFROM employeesGROUP BY department_id;FOR j IN 1..department_ids.COUNT() LOOPdepartment_min_salaries(department_ids(j)) := min_salaries(j);department_max_salaries(department_ids(j)) := max_salaries(j);

END LOOP;END BEFORE STATEMENT;

AFTER EACH ROW ISBEGINIF :NEW.salary < department_min_salaries(:NEW.department_id)OR :NEW.salary > department_max_salaries(:NEW.department_id) THENRAISE_APPLICATION_ERROR(-20505,'New Salary is out of acceptable

range');END IF;

END AFTER EACH ROW;END check_salary;

Page 118: Advanced PL/SQL Optimizing for Better Performance 2016

132

Compound Trigger Restrictions• A compound trigger must be a DML trigger and defined on

either a table or a view

• An exception that occurs in one section must be handled in that section. It cannot transfer control to another section

• :OLD and :NEW cannot appear in the declaration, BEFORE STATEMENT, or the AFTER STATEMENT sections

• Only the BEFORE EACH ROW section can change the value of :NEW

• The firing order of compound triggers is not guaranteed unless you use the FOLLOWS clause

Page 119: Advanced PL/SQL Optimizing for Better Performance 2016

133

FOLLOWS Clause• To ensure that a trigger fires after certain other

triggers on the same object, use the FOLLOWS clause

• Lets you order the executions of multiple triggers relative to each other

• Applies to both compound and simple triggers

• Applies only to the section of the compound trigger with the same timing point as the simple trigger

133

Page 120: Advanced PL/SQL Optimizing for Better Performance 2016

134

FOLLOWS Clause Example

• Consider two AFTER ROW ... FOR UPDATE triggers defined on the same table. One trigger needs to reference the :OLD value and the other trigger needs to change the :OLD value.

• In this case, you can use the FOLLOWS clause to order the firing sequence

134

CREATE OR REPLACE TRIGGER change_productAFTER UPDATE of product_id ON order_itemsFOR EACH ROWFOLLOWS oe1.compute_totalBEGIN

dbms_output.put_line ('Do processing here…');END;

Page 121: Advanced PL/SQL Optimizing for Better Performance 2016

Pivot and Unpivot

Turning things around!

Page 122: Advanced PL/SQL Optimizing for Better Performance 2016

136

PIVOT and UNPIVOT

• You can use the PIVOT operator of the SELECTstatement to write cross-tabulation queries that rotate the column values into new columns, aggregating data in the process.

• You can use the UNPIVOT operator of the SELECT statement to rotate columns into values of a column.

PIVOT UNPIVOT

Page 123: Advanced PL/SQL Optimizing for Better Performance 2016

137

Pivoting on the QUARTERColumn: Conceptual Example

30,000

40,000

60,000

30,000

40,000

20,000

AMOUNT_SOLD

2,500Q1IUSAKids Jeans

2,000Q2CJapanKids Jeans

2,000Q3SUSAShorts

I

P

C

CHANNEL

Kids Jeans

Shorts

Shorts

PRODUCT

1,000Q2Germany

1,500Q4USA

Q2

QUARTER

2,500Poland

QUANTITY_SOLD

COUNTRY

2,000

Q3

Kids Jeans

Shorts

PRODUCT

3,500

2,000

Q2

1,5002,500

Q4Q1

Page 124: Advanced PL/SQL Optimizing for Better Performance 2016

138

Pivoting Before Oracle 11g• Pivoting the data before 11g was a complex query

which required the use of the CASE or DECODEfunctions

select product,sum(case when quarter = 'Q1' then amount_sold else null end) Q1,sum(case when quarter = 'Q2' then amount_sold else null end) Q2,sum(case when quarter = 'Q3' then amount_sold else null end) Q3,sum(case when quarter = 'Q4' then amount_sold else null end) Q4from sales

group by product;

Page 125: Advanced PL/SQL Optimizing for Better Performance 2016

139

PIVOT Clause Syntaxtable_reference PIVOT [ XML ]

( aggregate_function ( expr ) [[AS] alias ][, aggregate_function ( expr ) [[AS] alias ] ]...

pivot_for_clausepivot_in_clause )

-- Specify the column(s) to pivot whose values are to -- be pivoted into columns.pivot_for_clause = FOR { column |( column [, column]... ) }

-- Specify the pivot column values from the columns you -- specified in the pivot_for_clause.pivot_in_clause = IN ( { { { expr | ( expr [, expr]... ) } [ [ AS] alias] }...

| subquery | { ANY | ANY [, ANY]...} } )

Page 126: Advanced PL/SQL Optimizing for Better Performance 2016

141

Creating a New View: Example

CREATE OR REPLACE VIEW sales_view ASSELECTprod_name AS product, country_name AS country, channel_id AS channel, SUBSTR(calendar_quarter_desc, 6,2) AS quarter,SUM(amount_sold) AS amount_sold, SUM(quantity_sold) AS quantity_sold

FROM sales, times, customers, countries, productsWHERE sales.time_id = times.time_id AND

sales.prod_id = products.prod_id ANDsales.cust_id = customers.cust_id ANDcustomers.country_id = countries.country_id

GROUP BY prod_name, country_name, channel_id,SUBSTR(calendar_quarter_desc, 6, 2);

Page 127: Advanced PL/SQL Optimizing for Better Performance 2016

143

Selecting the SALES VIEW DataSELECT product, country, channel, quarter, quantity_soldFROM sales_view;

PRODUCT COUNTRY CHANNEL QUARTER QUANTITY_SOLD------------ ------------ ---------- -------- -------------Y Box Italy 4 01 21Y Box Italy 4 02 17Y Box Italy 4 03 20. . .Y Box Japan 2 01 35Y Box Japan 2 02 39Y Box Japan 2 03 36Y Box Japan 2 04 46Y Box Japan 3 01 65. . .Bounce Italy 2 01 34Bounce Italy 2 02 43. . .9502 rows selected.

Page 128: Advanced PL/SQL Optimizing for Better Performance 2016

144

Pivoting the QUARTER Column in the SH Schema: Example

SELECT *FROM

(SELECT product, quarter, quantity_soldFROM sales_view) PIVOT (sum(quantity_sold) FOR quarter IN ('01', '02', '03', '04'))

ORDER BY product DESC;

. . .

Page 129: Advanced PL/SQL Optimizing for Better Performance 2016

146

Unpivoting the QUARTER Column: Conceptual Example

2,000

Q3

Kids Jeans

Shorts

PRODUCT

3,500

2,000

Q2

1,5002,500

Q4Q1

2,500Q1Kids Jeans

2,000Q2Kids Jeans

3,500Q2Shorts

1,500Q4Kids Jeans

Q3

QUARTER

2,000Shorts

SUM_OF_QUANTITYPRODUCT

Page 130: Advanced PL/SQL Optimizing for Better Performance 2016

147

Unpivoting Before Oracle 11g• Univoting the data before 11g requires multiple

queries on the table using the UNION ALL operator

SELECT *FROM (

SELECT product, '01' AS quarter, Q1_value FROM salesUNION ALLSELECT product, '02' AS quarter, Q2_value FROM salesUNION ALLSELECT product, '03' AS quarter, Q3_value FROM salesUNION ALLSELECT product, '04' AS quarter, Q4_value FROM sales);

Page 131: Advanced PL/SQL Optimizing for Better Performance 2016

148

Using the UNPIVOT Operator• An UNPIVOT operation does not reverse a PIVOT

operation; instead, it rotates data found in multiple columns of a single row into multiple rows of a single column.

• If you are working with pivoted data, UNPIVOTcannot reverse any aggregations that have been made by PIVOT or any other means.

UNPIVOT

Page 132: Advanced PL/SQL Optimizing for Better Performance 2016

149

Using the UNPIVOT Clause• The UNPIVOT clause rotates columns from a

previously pivoted table or a regular table into rows. You specify:– The measure column or columns to be unpivoted

– The name or names for the columns that result from the UNPIVOT operation

– The columns that are unpivoted back into values of the column specified in pivot_for_clause

• You can use an alias to map the column name to another value.

Page 133: Advanced PL/SQL Optimizing for Better Performance 2016

150

UNPIVOT Clause Syntaxtable_reference UNPIVOT [{INCLUDE|EXCLUDE} NULLS]-- specify the measure column(s) to be unpivoted.( { column | ( column [, column]... ) }

unpivot_for_clauseunpivot_in_clause )

-- Specify one or more names for the columns that will-- result from the unpivot operation.

unpivot_for_clause = FOR { column | ( column [, column]... ) }

-- Specify the columns that will be unpivoted into values of -- the column specified in the unpivot_for_clause.

unpivot_in_clause = ( { column | ( column [, column]... ) }

[ AS { constant | ( constant [, constant]... ) } ][, { column | ( column [, column]... ) }[ AS { constant | ( constant [, constant]...) } ] ]...)

Page 134: Advanced PL/SQL Optimizing for Better Performance 2016

151

Creating a New Pivot Table: Example

. . .

CREATE TABLE pivotedtable ASSELECT *FROM

(SELECT product, quarter, quantity_soldFROM sales_view) PIVOT (sum(quantity_sold) FOR quarter IN ('01' AS Q1, '02' AS Q2,

'03' AS Q3, '04' AS Q4));

SELECT * FROM pivotedtable ORDER BY product DESC;

Page 135: Advanced PL/SQL Optimizing for Better Performance 2016

152

Unpivoting the QUARTER Column : Example• Unpivoting the QUARTER Column in the SH Schema:

SELECT *FROM pivotedtableUNPIVOT (quantity_sold For Quarter IN (Q1, Q2, Q3, Q4))ORDER BY product DESC, quarter;

. . .

Page 136: Advanced PL/SQL Optimizing for Better Performance 2016

153

More Examples…• More information and examples could be found on

my Blog:

http://www.realdbamagic.com/he/pivot-a-table/

Page 137: Advanced PL/SQL Optimizing for Better Performance 2016

Oracle 12cR1

154

Page 138: Advanced PL/SQL Optimizing for Better Performance 2016

155

32K VARCHAR2/NVARCHAR2• Oracle 12c allows SQL VARCHAR2 to be the same size as

PL/SQL VARCHAR2

• This is not the default behavior and needs to be turned on:

• Create table with 32k varchar2:

155

ALTER SYSTEM set MAX_STRING_SIZE = EXTENDED scope = SPFILE;

CREATE TABLE Applicants(id NUMBER GENERATED AS IDENTITY,first_name varchar2(30),last_name varchar2(30),application date,CV varchar2(32767)

);

Page 139: Advanced PL/SQL Optimizing for Better Performance 2016

156

Invisible Columns• Columns might be marked “invisible”

• The invisible column will not be available unless explicitly named

• This is very useful when doing application migration

• The *_TAB_COLUMNS views see all of the columns (even invisible and unused)

156

Page 140: Advanced PL/SQL Optimizing for Better Performance 2016

157

Invisible Column Example

157

CREATE TABLE tab1 (id NUMBER,description VARCHAR2(50) INVISIBLE

);

INSERT INTO tab1 VALUES (1);

SELECT * FROM tab1;

ID----------

1

INSERT INTO tab1 (id, description) VALUES (2, 'TWO');COMMIT;

SELECT id, descriptionFROM tab1;

ID DESCRIPTION---------- ----------------------------------------------

1 2 TWO

Page 141: Advanced PL/SQL Optimizing for Better Performance 2016

158

Identity Column Type• In previous releases, there was no direct equivalent of

the AutoNumber or Identity functionality of other database engines

• This behavior had to be implemented using a combination of sequences and trigger

• Oracle 12c introduces the ability to define an identity clause against a table column defined using a numerictype

• User should have the create sequence privilege• Identity columns are always not null

Page 142: Advanced PL/SQL Optimizing for Better Performance 2016

159

Identity Column Type – Options• ALWAYS

– Forces the use of the identity. If an insert statement references the identity column, an error is produced

• BY DEFAULT– Allows using the identity if the column isn't referenced in

the insert statement. If the column is referenced, the specified value will be used in place of the identity

• BY DEFAULT ON NULL– Allows the identity to be used if the identity column is

referenced, but a value of NULL is specified

Identity.sql

Page 143: Advanced PL/SQL Optimizing for Better Performance 2016

160

Identity Column Type – Restrictions• You can specify only one identity column per table• When specifying identity clause, you must specify a

numeric data type for datatype in the column definition clause

• When specifying identity clause, you cannot specify the DEFAULT clause in the column definition clause

• When specifying identity clause, the NOT NULL constraint is implicitly specified

• CREATE TABLE AS SELECT will not inherit the identity property on a column

Page 144: Advanced PL/SQL Optimizing for Better Performance 2016

161

Default Value Using a Sequence• You can specify CURRVAL and NEXTVAL as default

values for a column

• Default value is used when the column is not referenced by the insert or when the DEFAULT keyword is used

• Gives you the ability to auto-populate master-detail relationships– Only makes sense if you can guarantee the inserts into the

detail table would always immediately follow the insert into the master table

Default_with_Sequence.sql

Page 145: Advanced PL/SQL Optimizing for Better Performance 2016

162

Default Value on Explicit Nulls• In an insert statement, when the column is

explicitly referenced, even when using the value NULL, the default value is not used

• Oracle database 12c allows you to modify this behavior using the ON NULL clause in the default definition

Default_with_Null.sql

Page 146: Advanced PL/SQL Optimizing for Better Performance 2016

163

Calling PL/SQL from SQL• You can define PL/SQL functions and procedures in

the WITH clause of a subquery and then use them as you would any other built-in or user-defined function

• The “;” does not work as a terminator to the SQL statement when the PL/SQL declaration is included in the WITH clause

• Functions defined in the PL/SQL declaration section of the WITH clause take precedence over objects with the same name defined at the schema level

• Provides better performance as compared with schema level functions

PLSQL_from_SQL.sql

Page 147: Advanced PL/SQL Optimizing for Better Performance 2016

164

Functions in the WITH Clause (12.1)

withfunction sumascii (str in varchar2) return number isx number := 0;

beginfor i in 1..length (str)loop

x := x + ascii (substr (str, i, 1)) ;end loop;return x;

end;select /*+ WITH_PLSQL */ h.EMPLOYEE_ID, h.last_name,

sumascii (h.last_name) from hr.employees h

Page 148: Advanced PL/SQL Optimizing for Better Performance 2016

Top-N and Paging QueriesIn Oracle 12c

Page 149: Advanced PL/SQL Optimizing for Better Performance 2016

166

Top-N Queries• A Top-N query is used to retrieve the top or

bottom N rows from an ordered set

• Combining two Top-N queries gives you the ability to page through an ordered set

• Oracle 12c has introduced the row limiting clause to simplify Top-N queries

Page 150: Advanced PL/SQL Optimizing for Better Performance 2016

167

Top-N in 12cR1

• This is ANSI syntax

• The default offset is 0

• Null values in offset, rowcount or percent will return no rows

[ OFFSET offset { ROW | ROWS } ][ FETCH { FIRST | NEXT } [ { rowcount | percent PERCENT } ]

{ ROW | ROWS } { ONLY | WITH TIES } ]

Page 151: Advanced PL/SQL Optimizing for Better Performance 2016

168

Top-N Examples

SELECT last_name, salaryFROM hr.employeesORDER BY salaryFETCH FIRST 4 ROWS ONLY;

SELECT last_name, salaryFROM hr.employeesORDER BY salaryFETCH FIRST 4 ROWS WITH TIES;

SELECT last_name, salaryFROM hr.employeesORDER BY salary DESCFETCH FIRST 10 PERCENT ROWS ONLY;

Page 152: Advanced PL/SQL Optimizing for Better Performance 2016

169

Paging Before 12c• Before 12c we had to use the rownum pseudo

column to filter out rows

• That will require sorting the entire rowset

SELECT valFROM (SELECT val, rownum AS rnum

FROM (SELECT valFROM rownum_order_testORDER BY val)

WHERE rownum <= 10)WHERE rnum >= 5;

Page 153: Advanced PL/SQL Optimizing for Better Performance 2016

170

Paging in Oracle 12c• After 12c we have a syntax improvement for

paging using the Top-N queries

• This will use ROW_NUMBER and RANK in the background – there is no real optimization improvements

SELECT valFROM rownum_order_testORDER BY valOFFSET 4 ROWS FETCH NEXT 5 ROWS ONLY;

Page 154: Advanced PL/SQL Optimizing for Better Performance 2016

171

More Examples• More information and examples could be found on

my blog:

http://www.realdbamagic.com/he/12c-top-n-query/

Page 155: Advanced PL/SQL Optimizing for Better Performance 2016

Pattern Matching inOracle 12c

Page 156: Advanced PL/SQL Optimizing for Better Performance 2016

173

What is Pattern Matching• Identify and group rows with consecutive values

• Consecutive in this regards – row after row

• Uses regular expression like syntax to find patterns

Page 157: Advanced PL/SQL Optimizing for Better Performance 2016

174

Common Business Challenges• Finding sequences of events in security

applications

• Locating dropped calls in a CDR listing

• Financial price behaviors (V-shape, W-shape U-shape, etc.)

• Fraud detection and sensor data analysis

Page 158: Advanced PL/SQL Optimizing for Better Performance 2016

175

MATCH_RECOGNIZE Syntax

SELECTFROM [row pattern input table]MATCH_RECOGNIZE( [ PARTITION BY <cols> ][ ORDER BY <cols> ][ MEASURES <cols> ][ ONE ROW PER MATCH | ALL ROWS PER MATCH ][ SKIP_TO_option]PATTERN ( <row pattern> )DEFINE <definition list>)

Page 159: Advanced PL/SQL Optimizing for Better Performance 2016

176

Example: Sequential Employee IDs• Our goal: find groups of users with sequences IDs

• This can be useful for detecting missing employees in a table, or to locate “gaps” in a group

FIRSTEMP LASTEMP---------- ----------

7371 74987500 75207522 75657567 76537655 76977699 77817783 77877789 7838

Page 160: Advanced PL/SQL Optimizing for Better Performance 2016

177

Pattern Matching Example

SELECT *

FROM Emps

MATCH_RECOGNIZE (

ORDER BY emp_id

PATTERN (STRT B*)

DEFINE B AS emp_id = PREV(emp_id)+1

ONE ROW PER MATCH

MEASURES

STRT.emp_id firstemp,

LAST(emp_id) lastemp

AFTER MATCH SKIP PAST LAST ROW

);

1. Define input

2. Pattern Matching

3. Order input

4. Process pattern

5. Using defined conditions

6. Output: rows per match

7. Output: columns per row

8. Where to go after match?

Page 161: Advanced PL/SQL Optimizing for Better Performance 2016

178

Pattern Matching Example (Actual Syntax)

SELECT *

FROM Emps

MATCH_RECOGNIZE (

ORDER BY emp_id

MEASURES

STRT.emp_id firstemp,

LAST(emp_id) lastemp

ONE ROW PER MATCH

AFTER MATCH SKIP PAST LAST ROW

PATTERN (STRT B*)

DEFINE B AS emp_id = PREV(emp_id)+1

);

1. Define input

2. Pattern Matching

3. Order input

4. Process pattern

5. Using defined conditions

6. Output: rows per match

7. Output: columns per row

8. Where to go after match?

Page 162: Advanced PL/SQL Optimizing for Better Performance 2016

179

Oracle 11g Analytic Function Solutionselect firstemp, lastempFrom (select nvl (lag (r) over (order by r), minr) firstemp, q lastemp

from (select emp_id r,lag (emp_id) over (order by emp_id) q,min (emp_id) over () minr,max (emp_id) over () maxr

from emps e1)where r != q + 1 -- groups including lower endunionselect q,

nvl (lead (r) over (order by r), maxr)from ( select emp_id r,

lead (emp_id) over (order by emp_id) q,min (emp_id) over () minr,max (emp_id) over () maxr

from emps e1)where r + 1 != q -- groups including higher end

);

Page 163: Advanced PL/SQL Optimizing for Better Performance 2016

180

Supported Regular Expression Patterns• Concatenation: No operator between elements.• Quantifiers:

– * 0 or more matches.– + 1 or more matches– ? 0 or 1 match.– {n} Exactly n matches.– {n,} n or more matches.– {n, m} Between n and m (inclusive) matches.– {, m} Between 0 an m (inclusive) matches.

• Alternation: |• Grouping: ()

Page 164: Advanced PL/SQL Optimizing for Better Performance 2016

181

Few Last Tips• Test all cases: pattern matching can be very tricky

• Don’t forget to test your data with no matches

• There is no LISTAGG and no DISTINCT when using match recognition

• Pattern variables cannot be used as bind variables

Page 165: Advanced PL/SQL Optimizing for Better Performance 2016

182

More 12c Developers’ Features…• Session-specific sequence

• Truncate CASCADE command

• Temporal Validity

• Temporary Undo

• Online DML Operations

• And tons of new features for DBAs too!

182

Page 166: Advanced PL/SQL Optimizing for Better Performance 2016

183

More Examples…• More information and examples could be found on

my Blog:

http://www.realdbamagic.com/he/pivot-a-table/

http://www.realdbamagic.com/he/12c-top-n-query/

http://www.realdbamagic.com/he/with-pl-sql-oracle-12c/

http://www.realdbamagic.com/he/session-level-sequence-12c/

183

Page 167: Advanced PL/SQL Optimizing for Better Performance 2016

Oracle 12cR2

184

Page 168: Advanced PL/SQL Optimizing for Better Performance 2016

185

Object Names Length• Up to Oracle 12cR2, objects name length (tables,

columns, indexes, constraints etc.) were limited to 30 chars

• Starting Oracle 12cR2, length is now limited to 128 bytes

create table with_a_really_really_really_really_really_long_name (and_lots_and_lots_and_lots_and_lots_and_lots_of int,really_really_really_really_really_long_columns int

);

Page 169: Advanced PL/SQL Optimizing for Better Performance 2016

186

LISTAGG in Oracle 12c• Limited to output of 4000 chars or 32000 with

extended column sizes

• Oracle 12cR2 provides overflow handling:

• Example:

listagg (measure_expr, ','[ on overflow (truncate|error) ][ text ] [ (with|without) count ]

) within group (order by cols)

select listagg(table_name, ',' on overflow truncate) within group (order by table_name) table_names

from dba_tables

Page 170: Advanced PL/SQL Optimizing for Better Performance 2016

187

Verify Data Type Conversions (12.2)• If we try to validate using regular conversion we

might hit an error: ORA-01858: a non-numeric character was found where a numeric

was expected

• Use validate_conversion to validate the data without an error

select t.*from dodgy_dates twhere validate_conversion(is_this_a_date as date) = 1;

select t.*from dodgy_dates twhere validate_conversion(is_this_a_date as date, 'yyyymmdd') = 1;

Page 171: Advanced PL/SQL Optimizing for Better Performance 2016

188

Handle Casting Conversion Errors (12.2)• Let’s say we convert the value of a column using

cast. What happens if some of the values doesn’t fit?

• The cast function can now handle conversion errors:select cast (

'not a date' as datedefault date'0001-01-01' on conversion error

) dtfrom dual;

Page 172: Advanced PL/SQL Optimizing for Better Performance 2016

189

JSON in 12.2.0.1• JSON in 12cR1 used to work with JSON documents

stored in the database

• 12cR2 brought the ability to create and modify JSON:– JSON_object

– JSON_objectagg

– JSON_array

– JSON_arrayagg

Page 173: Advanced PL/SQL Optimizing for Better Performance 2016

190

Handling JSON Documents from PL/SQL• 12.2.0.1 also introduced PL/SQL object to handle JSON.

The key object types are:

– json_element_t – a supertype for json docs and arrays

– json_document_t – for working with JSON documents

– json_array_t – for working with JSON arrays

• The treat function casts elements to the right typeemps := treat(doc.get('employees') as json_array_t);

for i in 0 .. emps.get_size - 1 loopemp := treat(emps.get(i) as json_object_t);emp.put('title', '');emp.put('name', upper(emp.get_String('name')));

end loop;

Page 174: Advanced PL/SQL Optimizing for Better Performance 2016

191

More 12c Developers’ Features…• Approximate Query Enhancements

• PL/SQL Code Coverage using new DBMS package: dbms_plsql_code_coverage

• Partitions enhancements

– List partition major changes: Auto-list, multi-column

– Read only partitions

– More…

• Currently available only in the cloud environment

Page 175: Advanced PL/SQL Optimizing for Better Performance 2016

Compiling PL/SQL

192

Page 176: Advanced PL/SQL Optimizing for Better Performance 2016

193

Compiling PL/SQL• Native and Interpreted Compilation

• Deprecating Functions (12cR2)

• Code inlining

• Fine-grain Dependency management

193

Page 177: Advanced PL/SQL Optimizing for Better Performance 2016

194

Native and Interpreted CompilationTwo compilation methods:

• Interpreted compilation

– Default compilation method

– Interpreted at run time

• Native compilation

– Compiles into native code

– Stored in the SYSTEM tablespace

Page 178: Advanced PL/SQL Optimizing for Better Performance 2016

195

Deciding on a Compilation Method• Use the interpreted mode when (typically during

development):– You are using a debugging tool, such as SQL Developer– You need the code compiled quickly

• Use the native mode when (typically post development):– Your code is heavily PL/SQL based– You are looking for increased performance in production

Native

Interpreted

Page 179: Advanced PL/SQL Optimizing for Better Performance 2016

196

Setting the Compilation Method• PLSQL_CODE_TYPE: Specifies the compilation mode for the

PL/SQL library units

• PLSQL_OPTIMIZE_LEVEL: Specifies the optimization level to be used to compile the PL/SQL library units

• In general, for fastest performance, use the following setting:

PLSQL_CODE_TYPE = { INTERPRETED | NATIVE }

PLSQL_OPTIMIZE_LEVEL = { 0 | 1 | 2 | 3}

PLSQL_CODE_TYPE = NATIVEPLSQL_OPTIMIZE_LEVEL = 2

Page 180: Advanced PL/SQL Optimizing for Better Performance 2016

198

Viewing the Compilation Settings• Use the USER|ALL|DBA_PLSQL_OBJECT_SETTINGS data

dictionary views to display the settings for a PL/SQL object:

DESCRIBE ALL_PLSQL_OBJECT_SETTINGS

Name Null? Type------------------------- -------- --------------------OWNER NOT NULL VARCHAR2(30)NAME NOT NULL VARCHAR2(30)TYPE VARCHAR2(12)PLSQL_OPTIMIZE_LEVEL NUMBERPLSQL_CODE_TYPE VARCHAR2(4000)PLSQL_DEBUG VARCHAR2(4000)PLSQL_WARNINGS VARCHAR2(4000)NLS_LENGTH_SEMANTICS VARCHAR2(4000)PLSQL_CCFLAGS VARCHAR2(4000)PLSCOPE_SETTINGS VARCHAR2(4000)

Page 181: Advanced PL/SQL Optimizing for Better Performance 2016

199

Viewing the Compilation SettingsSELECT name, plsql_code_type, plsql_optimize_levelFROM user_plsql_object_settings;

NAME PLSQL_CODE_TYP PLSQL_OPTIMIZE_LEVEL-------------------- -------------- --------------------ACTIONS_T INTERPRETED 2ACTION_T INTERPRETED 2ACTION_V INTERPRETED 2ADD_ORDER_ITEMS INTERPRETED 2CATALOG_TYP INTERPRETED 2CATALOG_TYP INTERPRETED 2CATALOG_TYP INTERPRETED 2CATEGORY_TYP INTERPRETED 2CATEGORY_TYP INTERPRETED 2COMPOSITE_CATEGORY_TYP INTERPRETED 2...

Page 182: Advanced PL/SQL Optimizing for Better Performance 2016

200

Setting Up a Database for Native Compilation• This requires DBA privileges.

• The PLSQL_CODE_TYPE compilation parameter must be set to NATIVE.

• The benefits apply to all the built-in PL/SQL packages that are used for many database operations.

ALTER SYSTEM SET PLSQL_CODE_TYPE = NATIVE;

Page 183: Advanced PL/SQL Optimizing for Better Performance 2016

201

Compiling a Program Unit for Native Compilation

SELECT name, plsql_code_type, plsql_optimize_levelFROM user_plsql_object_settingsWHERE name = 'ADD_ORDER_ITEMS';

NAME PLSQL_CODE_T PLSQL_OPTIMIZE_LEVEL---------------------- ------------ --------------------ADD_ORDER_ITEMS INTERPRETED 2ALTER SESSION SET PLSQL_CODE_TYPE = 'NATIVE';

ALTER PROCEDURE add_order_items COMPILE;

SELECT name, plsql_code_type, plsql_optimize_levelFROM user_plsql_object_settingsWHERE name = 'ADD_ORDER_ITEMS';

NAME PLSQL_CODE_T PLSQL_OPTIMIZE_LEVEL---------------------- ------------ --------------------ADD_ORDER_ITEMS NATIVE 2

1

2

3

4

Page 184: Advanced PL/SQL Optimizing for Better Performance 2016

202

PL/SQL Compile-Time Warnings• We can turn on checking for certain warning

conditions• Warning messages can be issued during compilation

of PL/SQL subprograms (not for anonymous blocks )• Use the SQL*Plus SHOW ERRORS command or query

the USER_ERRORS data dictionary view, to see any warnings generated during compilation

• PL/SQL warning messages use the prefix PLW• Use PLSQL_WARNINGS initialization parameter, or the DBMS_WARNING package

202

Page 185: Advanced PL/SQL Optimizing for Better Performance 2016

203

PL/SQL Warning Categories• SEVERE: Messages for conditions that might cause

unexpected behavior or wrong results, such as aliasing problems with parameters

• PERFORMANCE: Messages for conditions that might cause performance problems, such as passing a VARCHAR2 value to a NUMBER column in an INSERT statement.

• INFORMATIONAL: Messages for conditions that do not have an effect on performance or correctness, but that you might want to change to make the code more maintainable, such as unreachable code that can never be executed.

• All: refer to all warning messages

203

Page 186: Advanced PL/SQL Optimizing for Better Performance 2016

204

PLSQL_WARNINGS Parameter• Can be set at

– System level

– Session level

– Single compilation level

204

ALTER SYSTEM SET PLSQL_WARNINGS='ENABLE:PERFORMANCE';ALTER SESSION SET PLSQL_WARNINGS='DISABLE:ALL';ALTER SESSION SET PLSQL_WARNINGS='ENABLE:SEVERE','DISABLE:PERFORMANCE', 'ERROR:07204';

ALTER PROCEDURE query_emp COMPILEPLSQL_WARNINGS='ENABLE:ALL';

Page 187: Advanced PL/SQL Optimizing for Better Performance 2016

205

PLW-06009 Warning Message

• This warning means that the OTHERS handler of PL/SQL subroutine can exit without executing some form of RAISE or a call to the standard RAISE_APPLICATION_ERROR procedure.

• Good programming practices suggest that the OTHERS handler should pass an exception upward to avoid the risk of having exceptions go unnoticed

205

Page 188: Advanced PL/SQL Optimizing for Better Performance 2016

206

Pragma Deprecate (12.2)• Mark a function as deprecated

alter session set plsql_warnings = 'enable:(6019,6020,6021,6022)';

create or replace procedure your_old_code ispragma deprecate ( your_old_code,

'This is deprecated. Use new_code instead!' );begin

-- old code herenull;

end your_old_code;/show error

Warning(2,3): PLW-06019: entity YOUR_OLD_CODE is deprecated

Page 189: Advanced PL/SQL Optimizing for Better Performance 2016

207

Pragma Deprecate (cont.)• Errors will show when compiling calling code:

alter session set plsql_warnings = 'error:6020';create or replace procedure calling_old_code isbegin

your_old_code();end calling_old_code;/

SQL> show errorErrors for PROCEDURE CALLING_OLD_CODE:

LINE/COL ERROR-------- ---------------------------------------------------------------4/3 PLS-06020: reference to a deprecated entity: YOUR_OLD_CODE

declared in unit YOUR_OLD_CODE[1,11]. This is deprecated. Usenew_code instead!

Page 190: Advanced PL/SQL Optimizing for Better Performance 2016

Inlining

208

Page 191: Advanced PL/SQL Optimizing for Better Performance 2016

209

Intra Unit Inlining• Definition:

– Inlining is defined as the replacement of a call to subroutine with a copy of the body of the subroutine that is called.

– The copied procedure generally runs faster than the original.

– The PL/SQL compiler can automatically find the calls that should be inlined.

• Benefits:– Inlining can provide large performance gains when applied

judiciously by a factor of 2–10 times.

Page 192: Advanced PL/SQL Optimizing for Better Performance 2016

210

Use of Inlining• Influence implementing inlining via two methods:

– Oracle parameter PLSQL_OPTIMIZE_LEVEL

– PRAGMA INLINE

• Recommend that you:– Inline small programs

– Inline programs that are frequently executed

• Use performance tools to identify hot spots suitable for inline applications:– plstimer

Page 193: Advanced PL/SQL Optimizing for Better Performance 2016

211

Inlining Concepts• Noninlined program:

CREATE OR REPLACE PROCEDURE small_pgmISa NUMBER;b NUMBER;

PROCEDURE touch(x IN OUT NUMBER, y NUMBER) ISBEGINIF y > 0 THENx := x*x;

END IF;END;

BEGINa := b;FOR I IN 1..10 LOOPtouch(a, -17);a := a*b;

END LOOP;END small_pgm;

Page 194: Advanced PL/SQL Optimizing for Better Performance 2016

212

Inlining Concepts• Examine the loop after inlining:

...BEGIN

a := b;FOR i IN 1..10 LOOP

IF –17 > 0 THENa := a*a;

END IF;a := a*b;

END LOOP;END small_pgm;...

Page 195: Advanced PL/SQL Optimizing for Better Performance 2016

213

Inlining Concepts• The loop is transformed in several steps:

a := b;FOR i IN 1..10 LOOP ...

IF false THENa := a*a;

END IF;a := a*b;

END LOOP;

a := b;FOR i IN 1..10 LOOP ...

a := a*b;END LOOP;

a := b;a := a*b;FOR i IN 1..10 LOOP ...END LOOP;

a := b*b;FOR i IN 1..10 LOOP ...

END LOOP;

Page 196: Advanced PL/SQL Optimizing for Better Performance 2016

214

Inlining: Example• Set the PLSQL_OPTIMIZE_LEVEL session-level

parameter to a value of 2 or 3:

– Setting it to 2 means no automatic inlining is attempted.– Setting it to 3 means automatic inlining is attempted and no

pragmas are necessary.

• Within a PL/SQL subroutine, use PRAGMAINLINE– NO means no inlining occurs regardless of the level and

regardless of the YES pragmas.– YES means inline at level 2 of a particular call and increase the

priority of inlining at level 3 for the call.

ALTER PROCEDURE small_pgm COMPILE PLSQL_OPTIMIZE_LEVEL = 3 REUSE SETTINGS;

Page 197: Advanced PL/SQL Optimizing for Better Performance 2016

215

Inlining: Example• After setting the PLSQL_OPTIMIZE_LEVEL

parameter, use a pragma:

CREATE OR REPLACE PROCEDURE small_pgmIS

a PLS_INTEGER;FUNCTION add_it(a PLS_INTEGER, b PLS_INTEGER) RETURN PLS_INTEGER ISBEGIN

RETURN a + b;END;

BEGINpragma INLINE (small_pgm, 'YES');a := add_it(3, 4) + 6;

END small_pgm;

Page 198: Advanced PL/SQL Optimizing for Better Performance 2016

216

Inlining: Guidelines• Pragmas apply only to calls in the next statement

following the pragma.• Programs that make use of smaller helper subroutines

are good candidates for inlining.• Only local subroutines can be inlined.• You cannot inline an external subroutine.• Cursor functions should not be inlined.• Inlining can increase the size of a unit.• Be careful about suggesting to inline functions that

are deterministic.

Page 199: Advanced PL/SQL Optimizing for Better Performance 2016

Fine Grain Dependency Management

217

Page 200: Advanced PL/SQL Optimizing for Better Performance 2016

218

Invalidation of Dependent Objects

• Procedure A is a direct dependent of View B. View B is a direct dependent of Table C. Procedure A is an indirect dependent of Table C.

• Direct dependents are invalidated only by changes to the referenced object that affect them.

• Indirect dependents can be invalidated by changes to the reference object that do not affect them.

View B Table CProcedure A

Page 201: Advanced PL/SQL Optimizing for Better Performance 2016

219

More Precise Dependency Metadata • Before 11g, adding column D to table T invalidated the

dependent objects.• Oracle Database 11g records additional, finer-grained

dependency management:– Adding column D to table T does not impact view V and does

not invalidate the dependent objects

Procedure P Function FView VColumns: A,B

Table TColumns: A,B

Add column D

Page 202: Advanced PL/SQL Optimizing for Better Performance 2016

220

Fine-Grained Dependency Management• In Oracle Database 11g, dependencies are now

tracked at the level of element within unit.

• Element-based dependency tracking covers the following:– Dependency of a single-table view on its base table

– Dependency of a PL/SQL program unit (package specification, package body, or subprogram) on the following:

• Other PL/SQL program units

• Tables

• Views

Page 203: Advanced PL/SQL Optimizing for Better Performance 2016

221

Fine-Grained Dependency Management: Example 1

CREATE TABLE t2 (col_a NUMBER, col_b NUMBER, col_c NUMBER);

CREATE VIEW v AS SELECT col_a, col_b FROM t2;

ALTER TABLE t2 ADD (col_d VARCHAR2(20));

SELECT ud.name, ud.type, ud.referenced_name,

ud.referenced_type, uo.status

FROM user_dependencies ud, user_objects uo

WHERE ud.name = uo.object_name AND ud.name = 'V';

SELECT ud.name, ud.type, ud.referenced_name,

ud.referenced_type, uo.status

FROM user_dependencies ud, user_objects uo

WHERE ud.name = uo.object_name AND ud.name = 'V';

Page 204: Advanced PL/SQL Optimizing for Better Performance 2016

222

Fine-Grained Dependency Management: Example 1

ALTER TABLE t2 MODIFY (col_a VARCHAR2(20));

SELECT ud.name, ud.referenced_name, ud.referenced_type, uo.status

FROM user_dependencies ud, user_objects uo

WHERE ud.name = uo.object_name AND ud.name = 'V';

Page 205: Advanced PL/SQL Optimizing for Better Performance 2016

223

Fine-Grained Dependency Management:Example 2

CREATE PACKAGE pkg ISPROCEDURE proc_1;

END pkg; /CREATE OR REPLACE PROCEDURE p IS BEGIN

pkg.proc_1(); END p;/CREATE OR REPLACE PACKAGE pkgIS

PROCEDURE proc_1;PROCEDURE unheard_of;

END pkg; /

Page 206: Advanced PL/SQL Optimizing for Better Performance 2016

224

Guidelines for Reducing Invalidation• To reduce invalidation of dependent objects:

Add new items to theend of the package

Reference each tablethrough a view

Page 207: Advanced PL/SQL Optimizing for Better Performance 2016

225

Object Revalidation• An object that is not valid when it is referenced

must be validated before it can be used.

• Validation occurs automatically when an object is referenced; it does not require explicit user action.

• If an object is not valid, its status is either COMPILED WITH ERRORS, UNAUTHORIZED, or INVALID.

Page 208: Advanced PL/SQL Optimizing for Better Performance 2016

Tuning PL/SQL

226

Page 209: Advanced PL/SQL Optimizing for Better Performance 2016

227

Tuning PL/SQL• Memory handing in PL/SQL

• Global Temporary Tables (GTT)

• PL/SQL result cache

• Tips and Tricks

227

Page 210: Advanced PL/SQL Optimizing for Better Performance 2016

228

Packages: Memory Issues• Create packages that contain logically related

program units

• Reserve space for large allocations:

– Set the SHARED_POOL_RESERVED_SIZE initialization parameter

• Prevent large or frequently used objects from being aged out:

– Use the DBMS_SHARED_POOL package

228

ORA-04031: unable to allocate 4160 bytes of shared memory..

Page 211: Advanced PL/SQL Optimizing for Better Performance 2016

229

Pinning Objects• Use dbms_shared_pool package:

• Flags:

– P – Package, Procedure or Function

– T – Type

– R – Trigger

– Q – Sequence

229

DBMS_SHARED_POOL.KEEP(object_name, flag)DBMS_SHARED_POOL.UNKEEP(object_name, flag)

Page 212: Advanced PL/SQL Optimizing for Better Performance 2016

230

Reusing Package MemoryPragma SERIALLY_REUSABLE

• Memory is used more efficiently for scalability (more users consume more memory)

• Package global memory is kept in the SGA (instead of the UGA) and is reused for different users

• Package global memory is only used within a unit of work (a client-server call or a server to different server call)

• Memory can be released and reused by another user

230

Page 213: Advanced PL/SQL Optimizing for Better Performance 2016

231

SERIALLY_REUSABLE - Example

231

CREATE OR REPLACE PACKAGE maintain_stateISpragma serially_reusable;num1 number:= 0;

END maintain_state;/

CREATE OR REPLACE PACKAGE regular_stateISnum1 number:= 0;

END regular_state;/

Page 214: Advanced PL/SQL Optimizing for Better Performance 2016

232

SERIALLY_REUSABLE - Example

232

BEGINdbms_output.put_line(chr(10) || 'THE MAINTAIN PACKAGE');dbms_output.put_line('Original Value: ' || maintain_state.num1);maintain_state.num1 := maintain_state.num1 + 10;dbms_output.put_line('New Value: ' || maintain_state.num1 || chr(10));dbms_output.put_line('THE REGULAR PACKAGE');dbms_output.put_line('Original Value: ' || regular_state.num1);regular_state.num1 := regular_state.num1 + 10;dbms_output.put_line('New Value: ' || regular_state.num1 || chr(10));dbms_output.put_line(chr(10)||chr(10));dbms_output.put_line('THE MAINTAIN PACKAGE');dbms_output.put_line('Original Value: ' || maintain_state.num1);maintain_state.num1 := maintain_state.num1 + 10;dbms_output.put_line('New Value: ' || maintain_state.num1 || chr(10));dbms_output.put_line('THE REGULAR PACKAGE');dbms_output.put_line('Original Value: ' || regular_state.num1);regular_state.num1 := regular_state.num1 + 10;dbms_output.put_line('New Value: ' || regular_state.num1 || chr(10));

END;/

Page 215: Advanced PL/SQL Optimizing for Better Performance 2016

233

SERIALLY_REUSABLE - Example

First Run Second Run

233

THE MAINTAIN PACKAGEOriginal Value: 0New Value: 10

THE REGULAR PACKAGEOriginal Value: 0New Value: 10

THE MAINTAIN PACKAGEOriginal Value: 10New Value: 20

THE REGULAR PACKAGEOriginal Value: 10New Value: 20

THE MAINTAIN PACKAGEOriginal Value: 0New Value: 10

THE REGULAR PACKAGEOriginal Value: 20New Value: 30

THE MAINTAIN PACKAGEOriginal Value: 10New Value: 20

THE REGULAR PACKAGEOriginal Value: 30New Value: 40

Page 216: Advanced PL/SQL Optimizing for Better Performance 2016

234

SERIALLY_REUSABLE – Side Effect• Since we’re giving up state managing, we can now

avoid ORA-4068 when compiling

• For more information, visit my blog:

http://www.realdbamagic.com/he/solving-ora-04068/

234234234

Page 217: Advanced PL/SQL Optimizing for Better Performance 2016

235

Passing Data Between PL/SQL Programs• The flexibility built into PL/SQL enables you to

pass:

– Simple scalar variables

– Complex data structures

• You can use the NOCOPY hint to improve performance with the IN OUT parameters.

Page 218: Advanced PL/SQL Optimizing for Better Performance 2016

236

NOCOPY Hint• The hint enables the PL/SQL compiler to pass OUT

and IN OUT parameters by reference, as opposed to passing by value

• Enhances performance by reducing overhead when passing parameters since less memory is being used

• The Compiler will ignore the hint if it is not possible to reference the original structure (type conversion, constraints, for loop variable, etc.)

236

Page 219: Advanced PL/SQL Optimizing for Better Performance 2016

237

NOCOPY - Example

237

CREATE OR REPLACE PACKAGE show_emp_pkgIS

TYPE EmpTabTyp IS TABLE OF emp%ROWTYPE INDEX BY BINARY_INTEGER;PROCEDURE show_emp (p_Deptno IN NUMBER,

p_EmpTab OUT NOCOPY EmpTabTyp);END;/

Page 220: Advanced PL/SQL Optimizing for Better Performance 2016

238

Using the PARALLEL_ENABLE Hint• Can be used in functions as an optimization hint

• Indicates that a function can be used in a parallelized query or parallelized DML statement

CREATE OR REPLACE FUNCTION f2 (p_p1 NUMBER) RETURN NUMBER PARALLEL_ENABLE IS

BEGIN RETURN p_p1 * 2;

END f2;

Page 221: Advanced PL/SQL Optimizing for Better Performance 2016

239

Global Temporary Tables• Large result sets can be stored in Table Variables or Temporary Tables

• Temporary tables can be created to hold session-private data that exists only for the duration of a transaction or session.

• Each session sees its own separate set of rows

• DML locks are not acquired on the data

• We can create indexes, views, and triggers on temporary tables

• Using temporary tables instead of program variables for large

• record sets, can reduce memory consumption

239

CREATE GLOBAL TEMPORARY TABLE hr.employees_tempON COMMIT PRSERVE ROWS;

Page 222: Advanced PL/SQL Optimizing for Better Performance 2016

Result Cache

240

Page 223: Advanced PL/SQL Optimizing for Better Performance 2016

241

What Is Result Caching?• The result cache allows SQL query and PL/SQL function

results to be stored in cache memory. • Subsequent executions of the same query or function can

be served directly out of the cache, improving response times.

• This technique can be especially effective for SQL queries and PL/SQL functions that are executed frequently.

• Cached query results become invalid when the database data accessed by the query is modified.

Data dictionarycache

Librarycache

SGA

Resultcache

Shared pool

Page 224: Advanced PL/SQL Optimizing for Better Performance 2016

242

Increasing Result Cache Memory Size• You can increase the small, default result cache memory

size by using the RESULT_CACHE_MAX_SIZEinitialization parameter.

SGA

Default resultcache

Shared pool

Increasedresultcache

Page 225: Advanced PL/SQL Optimizing for Better Performance 2016

243

Setting Result_Cache_Max_Size• Set Result_Cache_Max_Size from the

command line or in an initialization file created by a DBA.

• The cache size is dynamic and can be changed either permanently or until the instance is restarted.

SQL> ALTER SYSTEM SET result_cache_max_size = 2M SCOPE = MEMORY;

System altered.

SQL> SELECT name, value2 FROM v$parameter3 WHERE name = 'result_cache_max_size';

NAME VALUE---------------------------------------- ------------------result_cache_max_size 2097152

1 row selected.

Page 226: Advanced PL/SQL Optimizing for Better Performance 2016

244

Enabling Query Result Cache• Use the RESULT_CACHE_MODE initialization

parameter in the database initialization parameter file.

• RESULT_CACHE_MODE can be set to:

– MANUAL (default): You must add the RESULT_CACHEhint to your queries for the results to be cached.

– FORCE: Results are always stored in the result cache memory, if possible.

Page 227: Advanced PL/SQL Optimizing for Better Performance 2016

245

SQL Query Result Cache• Definition:

– Cache the results of the current query or query fragment in memory, and then use the cached results in future executions of the query or query fragments.

– Cached results reside in the result cache memory portion of the SGA.

• Benefits:

– Improved performance

Page 228: Advanced PL/SQL Optimizing for Better Performance 2016

246

SQL Query Result Cache• Scenario:

– You need to find the greatest average value of credit limit grouped by state over the whole population.

– The query returns a large number of rows being analyzed to yield a few or one row.

– In your query, the data changes fairly slowly (say every hour) but the query is repeated fairly often (say every second).

• Solution:– Use the new optimizer hint /*+ result_cache */ in your query:

SELECT /*+ result_cache */ AVG(cust_credit_limit), cust_state_province

FROM sh.customersGROUP BY cust_state_province;

Page 229: Advanced PL/SQL Optimizing for Better Performance 2016

247

Clearing the Shared Pool and Result Cache

--- flush.sql--- Start with a clean slate. Flush the cache and shared pool. --- Verify that memory was released.SET ECHO ONSET FEEDBACK 1SET SERVEROUTPUT ON

execute dbms_result_cache.flushalter system flush shared_pool/

Page 230: Advanced PL/SQL Optimizing for Better Performance 2016

248

PL/SQL Function Result Cache• Definition:

– Enables data that is stored in cache to be shared across sessions

– Stores the function result cache in an SGA, making it available to any session that runs your application

• Benefits:

– Improved performance

– Improved scalability

Page 231: Advanced PL/SQL Optimizing for Better Performance 2016

249

Marking PL/SQL Function Results to Be Cached• Scenario:

– You need a PL/SQL function that derives a complex metric.

– The data that your function calculates changes slowly, but the function is frequently called.

• Solution:– Use the new RESULT_CACHE clause in your function

definition.

– You can also have the cache purged when a dependent table experiences a DML operation, by using the RELIES_ON clause.

Page 232: Advanced PL/SQL Optimizing for Better Performance 2016

250

CREATE OR REPLACE FUNCTION ORD_COUNT(cust_no number)

RETURN NUMBER

RESULT_CACHE RELIES_ON (orders)

IS

V_COUNT NUMBER;

BEGIN

SELECT COUNT(*) INTO V_COUNT

FROM orders

WHERE customer_id = cust_no;

return v_count;

end;

Creating a PL/SQL Function Using the RESULT_CACHE Clause

• Include the RESULT_CACHE option in the function definition. • Optionally, include the RELIES_ON clause.

Specifies that the result should be cached

Specifies the table upon which the function relies

(not needed in 11.2+)

Page 233: Advanced PL/SQL Optimizing for Better Performance 2016

251

Using the DETERMINISTIC Clause with Functions

• Specify DETERMINISTIC to indicate that the function returns the same result value whenever it is called with the same values for its arguments.

• This helps the optimizer avoid redundant function calls.

• If a function was called previously with the same arguments, the optimizer can elect to use the previous result.

• Do not specify DETERMINISTIC for a function whose result depends on the state of session variables or schema objects.

Page 234: Advanced PL/SQL Optimizing for Better Performance 2016

252

Calling the PL/SQL Function Inside a Query

select cust_last_name, ord_count(customer_id) no_of_orders

from customers

where cust_last_name = 'MacGraw'

Page 235: Advanced PL/SQL Optimizing for Better Performance 2016

253

Verifying Memory Allocation--- Establish the cache contentset serveroutput onexecute dbms_result_cache.memory_report

Page 236: Advanced PL/SQL Optimizing for Better Performance 2016

254

Viewing Cache Results Created

col name format a55select * from v$result_cache_statistics/

Page 237: Advanced PL/SQL Optimizing for Better Performance 2016

255

Calling the PL/SQL Function Again

select cust_last_name, ord_count(customer_id) no_of_orders

from customers

where cust_last_name = 'MacGraw'

Page 238: Advanced PL/SQL Optimizing for Better Performance 2016

256

Viewing Cache Results Foundcol name format a55select * from v$result_cache_statistics/

Page 239: Advanced PL/SQL Optimizing for Better Performance 2016

257

Confirming That the Cached Result Was Used

select type, namespace,status, scan_count,namefrom v$result_cache_objects/

Page 240: Advanced PL/SQL Optimizing for Better Performance 2016

258

PL/SQL Result Cache Pitfall• Beware of result caching of timed actions:

DBMS_LOCK.SLEEP will also be cached overriding the sleep

• Cannot be used with invoker's rights or in an anonymous block

• Cannot be used with pipelined table function

• Cannot be used with OUT or IN OUT parameters.

258

Page 241: Advanced PL/SQL Optimizing for Better Performance 2016

Tips and Tricks

259

Page 242: Advanced PL/SQL Optimizing for Better Performance 2016

260

Tuning PL/SQL CodeYou can tune your PL/SQL code by:

– Identifying the data type and constraint issues• Data type conversion

• The NOT NULL constraint

• PLS_INTEGER

• SIMPLE_INTEGER

– Writing smaller executable sections of code

– Comparing SQL with PL/SQL

– Rephrasing conditional statements

Page 243: Advanced PL/SQL Optimizing for Better Performance 2016

261

DECLAREn NUMBER;

BEGINn := n + 15; -- convertedn := n + 15.0; -- not converted...

END;

Avoiding Implicit Data Type Conversion– PL/SQL performs implicit conversions between

structurally different data types.

– Example: When assigning a PLS_INTEGER variable to a NUMBER variable

strings

dates

numbers

Page 244: Advanced PL/SQL Optimizing for Better Performance 2016

262

Understanding the NOT NULL Constraint

PROCEDURE calc_m IS

m NUMBER; --no constraint

...

BEGIN

m := a + b;

IF m IS NULL THEN

-- raise error

END IF;

END;

PROCEDURE calc_m IS

m NUMBER NOT NULL:=0;

a NUMBER;

b NUMBER;

BEGIN

m := a + b;

END;

The value of the expression a + b is assigned to a temporary variable,

which is then tested for nullity.

A better way to check nullity; no performance overhead

Page 245: Advanced PL/SQL Optimizing for Better Performance 2016

263

Using the PLS_INTEGER Data Type for Integers

Use PLS_INTEGER when dealing with integer data.

– It is an efficient data type for integer variables.

– It requires less storage than INTEGER or NUMBER.

– Its operations use machine arithmetic, which is faster than library arithmetic.

Page 246: Advanced PL/SQL Optimizing for Better Performance 2016

264

Using the SIMPLE_INTEGER Data Type• Definition:

– Is a predefined subtype– Has the range –2147483648 .. 2147483648– Does not include a null value– Is allowed anywhere in PL/SQL where the PLS_INTEGER data

type is allowed

• Benefits:– Eliminates the overhead of overflow

checking– Is estimated to be 2–10 times faster

when compared with the PLS_INTEGERtype with native PL/SQL compilation

Page 247: Advanced PL/SQL Optimizing for Better Performance 2016

265

Comparing SQL with PL/SQLEach has its own benefits:

• SQL:

– Accesses data in the database

– Treats data as sets

• PL/SQL:

– Provides procedural capabilities

– Has more flexibility built into the language

Page 248: Advanced PL/SQL Optimizing for Better Performance 2016

266

Comparing SQL with PL/SQL• Some simple set processing is markedly faster than the equivalent

PL/SQL.

• Avoid using procedural code when it may be better to use SQL.

...FOR I IN 1..5600 LOOP

counter := counter + 1;

SELECT product_id, warehouse_id

INTO v_p_id, v_wh_id

FROM big_inventories WHERE v_p_id = counter;

INSERT INTO inventories2 VALUES(v_p_id, v_wh_id);

END LOOP;...

BEGIN

INSERT INTO inventories2

SELECT product_id, warehouse_id

FROM main_inventories;

END;

Page 249: Advanced PL/SQL Optimizing for Better Performance 2016

267

Rephrasing Conditional Control Statements

If your business logic results in one condition being true, use the ELSIF syntax for mutually exclusive clauses:

IF v_acct_mgr = 145 THEN

process_acct_145;

END IF;

IF v_acct_mgr = 147 THEN

process_acct_147;

END IF;

IF v_acct_mgr = 148 THEN

process_acct_148;

END IF;

IF v_acct_mgr = 149 THEN

process_acct_149;

END IF;

IF v_acct_mgr = 145

THEN

process_acct_145;

ELSIF v_acct_mgr = 147 THEN

process_acct_147;

ELSIF v_acct_mgr = 148 THEN

process_acct_148;

ELSIF v_acct_mgr = 149 THEN

process_acct_149;

END IF;

Page 250: Advanced PL/SQL Optimizing for Better Performance 2016

SQL Developer Command Line (SQLcl)

The Next Generation of SQL*Plus?

268

Page 251: Advanced PL/SQL Optimizing for Better Performance 2016

269

SQL*Plus• Introduced in Oracle 5 (1985)

• Looks very simple but has tight integration with other Oracle infrastructure and tools

• Very good for reporting, scripting, and automation

• Replaced old CLI tool called …

UFI (“User Friendly Interface”)

Page 252: Advanced PL/SQL Optimizing for Better Performance 2016

270

What’s Wrong With SQL*Plus?• Nothing really wrong with SQL*Plus – it is being

updated constantly but it is missing a lot of functionality

• SQL*Plus forces us to use GUI tools to complete some basic tasks

• Easy to understand, a bit hard to use

• Not easy for new users or developers

Page 253: Advanced PL/SQL Optimizing for Better Performance 2016

271

Using SQL Developer• SQL Developer is a free GUI tool to handle common

database operations• Comes with Oracle client installation starting Oracle

11g• Good for development and management of databases

– Developer mode– DBA mode– Modeling mode

• Has a Command Line interface (SDCLI) – but it’s not interactive

Page 254: Advanced PL/SQL Optimizing for Better Performance 2016

272

SQL Developer Command Line (SQLcl)• The SQL Developer Command Line (SQLcl, priv.

SDSQL) is a new command line interface (CLI) for SQL developers, report users, and DBAs

• It is part of the SQL Developer suite – developed by the same team: Oracle Database Development Tools Team

• Does (or will do) most of what SQL*Plus can do, and much more

• Main focus: making life easier for CLI users• Minimal installation, minimal requirements

Page 255: Advanced PL/SQL Optimizing for Better Performance 2016

273

Current Status (November 2016)• Production as of September 2016

– current version: 4.2.0.16.308.0750, November 3, 2016

• New version comes out every couple of months

– Adding support for existing SQL*Plus commands/syntax

– Adding new commands and functionality

• The team is accepting bug reports and enhancement requestsfrom the public

• Active community on OTN forums!

Page 256: Advanced PL/SQL Optimizing for Better Performance 2016

274

Prerequisites• Very small footprint: 16 MB

• Tool is Java based so it can run on Windows, Linux, and OS/X

• Java 7/8 JRE (runtime environment - no need for JDK)

• No need for installer or setup

• No need for any other additional software or special license

• No need for an Oracle Client

Page 257: Advanced PL/SQL Optimizing for Better Performance 2016

275

Installing• Download from: SQL Developer Command Line

OTN Page

• Unzip the file

• Run it

Page 258: Advanced PL/SQL Optimizing for Better Performance 2016

276

Running SQLcl

Page 259: Advanced PL/SQL Optimizing for Better Performance 2016

What Can It Do?

Page 260: Advanced PL/SQL Optimizing for Better Performance 2016

278

Connecting to the Database• When no Oracle Client - using thin connection:

EZConnect connect style out of the box

connect host:port/service

• Support TNS, Thick and LDAP connection when Oracle home detected

• Auto-complete connection strings from last connections AND tnsnames.ora

Page 261: Advanced PL/SQL Optimizing for Better Performance 2016

279

Object Completion and Easy Edit• Use the tab key to complete commands

• Can be used to list tables, views or other queriableobjects

• Can be used to replace the * with actual column names

• Use the arrow keys to move around the command

• Use CTRL+W and CTRL+S to jump to the beginning/end of commands

Page 262: Advanced PL/SQL Optimizing for Better Performance 2016

280

Command History• 100 command history buffer• Commands are persistent between sessions (watch out for

security!)• Use UP and DOWN arrow keys to access old commands• Usage:

historyhistory usageHistory scripthistory fullHistory clear [session?]

• Load from history into command buffer:history <number>

Page 263: Advanced PL/SQL Optimizing for Better Performance 2016

281

Describe, Information and Info+• Describe lists the column of the tables just like

SQL*Plus

• Information shows column names, default values, indexes and constraints.

• In 12c database information shows table statistics and In memory status

• Works for table, views, sequences, and code objects

• Info+ shows additional information regarding column statistics and column histograms

Page 264: Advanced PL/SQL Optimizing for Better Performance 2016

282

SHOW ALL and SHOW ALL+• The show all command is familiar from SQL*Plus –

it will show all the parameters for the SQL*Plus settings

• The show all+ command will show the show all command and some perks: available tns entries, list of pdbs, connection settings, instance settings, nls settings, and more!

Page 265: Advanced PL/SQL Optimizing for Better Performance 2016

283

Pretty Input• Using the SQL Developer formatting rules, it will

change our input into well formatted commands.

• Use the SQLFORMATPATH to point to the SQL Developer rule file (XML)

SQL> select * from dual;

D-X

SQL> format buffer;1 SELECT2 *3 FROM4* dual

Page 266: Advanced PL/SQL Optimizing for Better Performance 2016

284

SQL*Plus Output• SQL*Plus output is generated as text tables

• We can output the data as HTML but the will take over everything we do in SQL*Plus (i.e. describe command)

• We can’t use colors in our output

• We can’t generate other types of useful outputs (CSV is really hard for example)

Page 267: Advanced PL/SQL Optimizing for Better Performance 2016

285

Generating Pretty Output• Outputting query results becomes easier with the “set

sqlformat” command (also available in SQL Developer)

• We can create a query in the “regular” way and then switch between the different output styles:– ANSIConsole

– Fixed column size output

– XML or JSON output

– HTML output generates a built in search field and a responsive html output for the result only

Page 268: Advanced PL/SQL Optimizing for Better Performance 2016

286

Generating Other Useful Outputs• We can generate loader ready output (with “|” as

a delimiter)

• We can generate insert commands

• We can easily generate CSV output

• Usage:set sqlformat { csv,html,xml,json,ansiconsole,insert,loader,fixed,default}

Page 269: Advanced PL/SQL Optimizing for Better Performance 2016

287

Load Data From CSV File• Loads a comma separated value (csv) file into a

table

• The first row of the file must be a header row and the file must be encoded UTF8

• The load is processed with 50 rows per batch

• Usage:LOAD [schema.]table_name[@db_link] file_name

Page 270: Advanced PL/SQL Optimizing for Better Performance 2016

288

SCRIPT – Client Side Scripting• SQLcl exposes JavaScript scripting with nashorn to

make things very scriptable on the client side

• This means we can create our own commands inside SQLcl using JavaScript

• Kris Rice’s from the development team published multiple example on his blog http://krisrice.blogspot.com/ and in GitHub, for example the autocorrect example demo.

Page 271: Advanced PL/SQL Optimizing for Better Performance 2016

289

Summary• We talked about Developing PL/SQL: composite

datatypes, cursors, dynamic SQL

• We went over Oracle 11g and Oracle 12c important features

• We looked at the compilation process of PL/SQL and how to make it more efficient

• We talked about tuning our PL/SQL code and some dedicated features for that

• SQLcl – new replacement tool for SQL*Plus

289

Page 272: Advanced PL/SQL Optimizing for Better Performance 2016

290

What Did We Not Talk About?• There are a lot of changes in Oracle 12c. We only

looked on a few developer oriented features

• New XML and JSON handling functions

• Edition Based Redefinition

• Pipeline Table Functions and Parallel executions

• SQL Performance and new Optimizer changes

• Exceptions and Error handling

290

Page 273: Advanced PL/SQL Optimizing for Better Performance 2016

Q&AAny Questions? Now will be the

time!

291

Page 274: Advanced PL/SQL Optimizing for Better Performance 2016

Zohar Elkayamtwitter: @[email protected]

www.ilDBA.co.ilwww.realdbamagic.com

292

Page 275: Advanced PL/SQL Optimizing for Better Performance 2016

293293293293