1 Copyright © 2011, Oracle and/or its affiliates. All rights Insert … · Insert Information Protection Policy Classification from Slide 8 Java in the Database – The Sky’s the

1 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.

Insert Information Protection Policy Classification from Slide 8


reserved.


Java in the Database – The Sky’s the Limit: Lessons

from the Largest Deployment Kuassi Mensah

Group Product Manager

Oracle Corporation

Rune Lilleng

Database Manager

Norwegian Labor and Welfare Admin..


reserved.


reserved.

Latin America 2011 December 6–8, 2011

Tokyo 2012 April 4–6, 2012


reserved.

Oracle OpenWorld Bookstore

• Visit the Oracle OpenWorld Bookstore for a fabulous

selection of books on many of the conference topics

and more!

• Bookstore located at Moscone West, Level 2

• All Books at 20% Discount


reserved.

Who is Kuassi Mensah …

• Group Product Manager for Database Access Frameworks (Net

Services, DRCP, etc) and database APIs (Java, C/C++, PHP, Ruby,

Python, Perl).

• Hold a MS in Computer Sciences from the Programming Institute

(University of Paris VI).

• Has published several articles and a book @

http://www.amazon.com/exec/obidos/ASIN/1555583296

• Is a frequent speaker at Oracle and IT events and maintain a blog @

http://db360.blogspot.com, as well as Facebook, Linkedin, and Twitter

(http://twitter.com/kmensah) pages.


reserved.

Who is Rune Lilleng …

• Database Manager @ Norwegian Labour and Welfare

Administration (NAV)

• Worked with Oracle products for 11 years

– Database

– Weblogic

– Exadata

• NAV’s IT department

– 700 employees (IT Operations and IT Development)


reserved.

Program Agenda

• Why Java in the database

• Architecture & best practices

• Use Cases

• Lessons from the largest deployment

• Q/A


reserved.


Java in the Database

Why?


reserved.

Java

JDBC Calls

SQL

Data Shipping/Caching vs Function Shipping

Java

Multiple Roundtrips

Java SE/EE in Mid-Tier

SQL


Up to 10 times faster!

Java + any language

1 Roundtrip


reserved.

Rationales for Java in the Database

• Why Java?

– #1 programming language (TIOBE index)

• Java VM

– Ensures Java Portability

– Not just Java, any language that compiles to Java byte codes

• http://www.is-research.de/info/vmlanguages/

• Why in the database

– In-place data processing

– Java methods as triggers and stored procedures

– Ability to extend database capabilities with Java libraries


reserved.

Decide When to Adopt Java in the Database

• Java EE belongs to Middle-tier

• Java SE with little/no SQL works better in the mid-tier

• Java SE with SQL (JDBC) works better/faster in the

database

– Avoid the round trip cost for massive data processing

– Extreme Scalability: system and user defined application classes

shared across the instance

– Relatively simple to move code from client to database and back


reserved.

Why Not PL/SQL

• PL/SQL is better for direct manipulation of SQL types

• Java more effective for algorithmic (data logic)

• There are more Java developers

• Java has superior built-in functionalities

• Java has a wide range of libraries

• There are things you cannot do (easily) in PL/SQL


reserved.



Architecture & Best Practices


reserved.

Tight Integration with the Database The Sky is the limit

UGA

Newspace Newspace Newspace

Oldspace Oldspace Oldspace

Class Objmems

UGA UGA Sessionspace Sessionspace Sessionspace

Internspace

PGA

SGA


reserved.

Specialized Heaps

• Newspace & Oldspace

– GC’d space

– Holds Java objects and most internal VM structures

– All Java objects are allocated from here

• Sessionspace

– Database session duration: objects which survive call boundary

• SGA class objmems

– Shared heaps for each loaded class

– Lifetime tied to duration for which class is in use

• Other internal heaps - UGA class objmems

– Call/session duration unscanned spaces

– Intern space


reserved.

JIT Compilation

• Automatic upon class loading

• Data dictionary views allow monitoring methods that

have been compiled SQL> select method_name, method_index, is_compiled from user_java_methods

2 where name = 'java/lang/String' and method_name = 'substring';

METHOD_NAME METHOD_INDEX IS_

---------------- ------------ ---

substring 48 YES

substring 49 YES


reserved.

Manual Compilation

Shouldn’t be necessary, but …

SQL> select dbms_java.compile_method('Test','foo','()V') from dual;

DBMS_JAVA.COMPILE_METHOD('TEST','FOO','()V')

--------------------------------------------

1

Total of 1 methods compiled in 25 ms.

SQL> select method_name, is_compiled from user_java_methods

where name='Test';

METHOD_NAME IS_

---------------- ---

<init> NO

foo YES


reserved.

Memory Management

• It’s better to handle an OutOfMemoryError than to get an ORA-

exception that can’t be handled in Java

• Find a good value for max oldspace size through experimentation

• To get or set the oldspace memory limit equivalent to –Xmx <value>

Static long oracle.aurora.OracleRuntime.getMaxMemorySize()

Static void oracle.aurora.OracleRuntime.setMaxMemorySize(long value)

If it’s huge it may mask a leak, but if it is too small you will get out of

memory errors.


reserved.

Dealing With Out of Memory Errors

import oracle.aurora.vm.OracleRuntime; ...

while (keepGoing) {

try {

run();

}

catch (OutOfMemoryError e) {

long currentMax = OracleRuntime.getMaxMemorySize();

resetAndCleanup();

if (currentMax < MAX_MEMORY_THRESHOLD)

OracleRuntime.setMaxMemorySize(currentMax +

MAX_MEMORY_INCREMENT);

else

keepGoing = false;

}

}


reserved.

Monitoring Memory Usage through JMX

• Built-in MBeans monitor each heap area


reserved.


• Set notification thresholds, per heap


reserved.


• Monitor finer-grained GC statistics


reserved.

Monitoring through the Data Dictionary SQL> select s.sid, n.name p_name, st.value

from v$session s, v$sesstat st, v$statname n

where s.sid=102 and s.sid=st.sid and n.statistic# = st.statistic# and n.name like

'java%';

SID P_NAME VALUE

--- ---------------------------------------- ----------

102 java call heap total size 6815744

102 java call heap total size max 6815744

102 java call heap used size 668904

102 java call heap used size max 846920

102 java call heap live size 667112

102 java call heap live size max 704312

102 java call heap object count 13959

102 java call heap object count max 17173

102 java call heap live object count 13907

102 java call heap live object count max 14916

102 java call heap gc count 432433

102 java call heap collected count 123196423


reserved.

Memory Usage Best Practices

• Configure appropriate heap size via: oracle.aurora.vm.OracleRuntime.setMaxMemorySize(long)

• Avoid retaining unnecessary state across calls

• Avoid very very large contiguous allocations

• Avoid repeated use of user-defined classloaders

• Typical Java best-practices

– Use finalizers sparingly

– Avoid System.gc()


reserved.



Use Cases


reserved.

Java in the Database: What for?

• Portable Data Logic

• Custom Alert applications that monitor business data

• Parsers for various File Formats (txt, zip, xml, binary)

• Custom Md5 CRC

• Trigger-based Notification System using RMI

• Sending emails with attachment from within the database

• Execute external OS commands and external procedures

• Implement image transformation and

format conversion (GIF, PNG, JPEG,

etc)

• Implement database-resident CMS

• HTTP Call-Out

• JDBC Call-Out to Non-Oracle RDBMS

• RMI Call-Out to SAP

• Web Services Call-Out

• RESTful Database Web Services

• Lucene Domain Index


reserved.

JDBC Callout to Non-Oracle RDBMS


reserved.

SAP Callout


reserved.

Database as Web Services Consumer


reserved.


reserved.

Java in the Database Summary

• Business Logic Runs Directly in the Database and

Process Data In Situ

– Code Shipping in Cheaper/Faster than Data Shipping

– Eliminates Network Roundtrip

– Less Moving Parts

• Perfect for Data-Intensive Problems

• Good for Data-and-Compute intensive


reserved.



Lessons from the largest

deployment


reserved.

Agenda

• About NAV

• About ARENA

– History

– Technical Architecture

– System Architecture

– Experiences

• Q & A


reserved.

About

• Rune Lilleng • Database Manager @ NAV

– Oracle, DB2, MS SQL – 9 DBA

• Worked with Oracle products for 11 years – Database – Weblogic – Exadata

• NAV’s IT department – 700 employees (IT Operations and IT Development)


reserved.

About NAV

• The Norwegian Labour and Welfare Administration (NAV) was established on 1 July 2006.

• Handles a broad range of state services – Pension services – Sickness benefits – Unemployment benefits – Occupational rehabilitation – Social Services

• 450 local offices all across Norway – 20.000 Employees

• Administers one third of the national budget – 300+ billion NOK

• Has the entire population as its users


reserved.

About ARENA

– Development started in late 1999 and the system was delivered in 2003.

– It replaced several old mainframe systems, and at the time it’s main functions were related

to unemployment and work related services.

– 2003: 1500 concurrent users, estimated and tested for a maximum load of 2500 concurrent

users. Processed about 14 Billion NOK in benefits.

– 2006: Aetat (work) and RTV (pensions and social security) merger into NAV. HW upgrades

and application changes made to handle 5500 concurrent users.

– 2010: Social reform, and system changes made to handle a new major benefit (AAP) and

potentially 3000 new users.

– 2011: Close to 8000 concurrent users, processing 75 Billion NOK.


reserved.

System Architecture


reserved.

Technical Architecture

– Oracle database 11g for all data and as JAVA VM for the system’s “beans framework”

– OAS 10g for the runtime environment.

– User Interface is developed in Oracle Developer/Forms Builder.


reserved.

Technical Architecture Oracle Application Server 10g Oracle Forms PL/SQL

Workbench GUI

Validation logic

Business logic

Presentation

Presentation-

logic

Oracle database 11g EE PL/SQL JAVA/XML

Frame Solution

Beans Facade Managers

Workflow Processes

Organization Rules

Document Process Data

JAVA

Business Data

Business Logic Batch

Batch $

Oracle Weblogic

11g

Integration Components


reserved.

Experiences

- Highly Scalable

- From 1500 concurrent users (2003) to 8000 (2011)

- Application performance remains unchanged

- Added more CPU cores and memory over the years (12 to 46)

- Highly Reliable

- Prior to 11g upgrade < 1h unplanned downtime due to non human errors (eg. bugs)

- 11g

- Issues with mutex contention

- Application accesses certain objects (PL/SQL packages JAVA) very frequently which again causes a high

degree of mutex contention.

- Oracle Forms

- Decreased in popularity, difficult to recruit

- Memory hungry on the database server, one connection per users

- Looking into connection pooling options


reserved.

Future

- Today this runs on an HP Superdome

- Was about to be replaced with a new one due to age and high support costs, but that is no longer a viable

option.

- Migration project is planned for 2012, and Oracle Exadata is an alternative.

- Done some tests already, on one of our X2-2 machines, but not full scale.

- Forms

- Although the application is closing on 12 years, it’s projected to live another 10 to 15 years

- Would like to migrate away from Forms, to eg. Weblogic


reserved.


Q&A


reserved.

For More Information

search.oracle.com

or oracle.com/technetwork/database/enterprise-edition/index-098579.html

Oracle Net Services


reserved.


reserved.

The preceding is intended to outline our general product direction.

It is intended for information purposes only, and may not be

incorporated into any contract. It is not a commitment to deliver

any material, code, or functionality, and should not be relied upon

in making purchasing decisions. The development, release, and

timing of any features or functionality described for Oracle’s

products remains at the sole discretion of Oracle.


reserved.


Documents

1 Copyright © 2011, Oracle and/or its affiliates. All rights Insert … · Insert Information Protection Policy Classification from Slide 8 Java in the Database – The Sky’s the