New Generation of OpenEdge ® RDBMS Advanced Storage Architecture II Tomáš Kučera Principal Solution Engineer / EMEA Power Team

New Generation of OpenEdge® RDBMS

Advanced Storage Architecture II

Tomáš KučeraPrincipal Solution Engineer / EMEA Power Team

© 2007 Progress Software Corporation2

> whoami

Started with Progress Software Czech Republic – Dec 19, 1994

2 years @ Technical Support (1996 at ETSC) Since Dec 1996 – Consultant, Presales, Trainer,

Project Manager Currently – GFS Mgmt responsibilities for Cze&Pol,

member of the EMEA Power Team – primary responsibility for SAND

Product Areas: OpenEdge, Sonic, some Apama and Actional

Contact: [email protected], Skype: tomas_kucera

TKU


Audience Survey

Release When Age

7.2A long ago ancient



9.0A Dec 1998 over 8 years



10.1A Jan 2006 over 1 year

10.1B Jan 2007 3 months


Audience Survey

Release When Age







10.1A Jan 2006 over 1 year

10.1B Jan 2007 3 months


Agenda

Summary of Advanced Storage Architecture Advanced Storage Architecture II Migrating to ASA II Bonus


What‘s Advanced Storage Architecture II

ASA I• Implemented in Progress V9• Data Storage Areas• RECID unique per area

ASA II• Implemented in OpenEdge 10• A brand new space allocation mechanism• RECID unique per area and object block

OE10 RDBMS can have objects from both ASAI&II

Summary


Agenda

Summary of Advanced Storage Architecture Advanced Storage Architecture II Migrating to ASA II


Advanced Storage Architecture II

Data Clusters Database „Objects“ Storage Area Type II

What‘s in the box?


Data Clusters

A group of 8 or more adjacent blocks – fixed size for area

A unit of space allocation for objects A unit of space allocation for extents All blocks in cluster belong to same object Entire cluster can be read in a single IO operation


Database „Objects“

Composed of one or more clusters Object types:

• Table

• Index

• Sequence generators

• Large objects: binary - BLOB, character - CLOB, character – XML

• Empty


Storage Area Type II

Data Storage Area is made up of one or more extents and contains one or more database objects

Extent contains one or more data clusters Data clusters have fixed size of 8, 64 or 512

database blocks and are unit of space allocation

Blocks in a cluster are contiguous and belong to the same database object


Areas, Extents, Clusters, Blocks

Area root cluster

A Table

An Index

Another Table

Free clusters

Empty clusters

Extent header

Free blocks

Extent 1

Extent 0

Area

cl 0cl 1cl 2cl 3cl 4cl 5cl 6cl 7

cl 8cl 9cl 10cl 11cl 12cl 13cl 14cl 15


New Database Block Types

Block Type Contents

Extent Header Extent type, current extent size, maximum extent size, cluster size, area id, block size, control file name

Area Root cluster size, link to free cluster mapcluster highwater mark

Free Cluster Map bitmap of clusters in use or free,next and prev links

Object List list of object id’s and their root cluster id’s

Object Root object id, object type, link to free map, link to row space map

Cluster List list of clusters belonging to an object


ASA Performance

Feature Benefit

• Unnecessary elimination of table and index fragmentation

• Reduced need for data reorganization providing less frequent and shorter downtimes

• Allocated space for objects in contiguous chunks that can be read (and written) using large i/o operations

• Enhanced overall performance

• Better utilization of available disk bandwidth

• Enhanced overall database i/o performance

• Enhanced maintenance utility performance for “bulk” operations

• Full object scans possible • Enhanced online application performance


Agenda

Summary of Advanced Storage Architecture Advanced Storage Architecture II Migrating to ASA II


Migrating to ASA II

Backup your database Keep dbanalys & promon data for reference Install OpenEdge 10 (no need to delete /

overwrite Progress V9) Upgrade database to OE10 Run UPDATE STATISTICS for SQL Backup your database again! Recompile application code Ready ... Steady ... GO!!!

General Upgrade Strategy


Migrating to ASA II

Backup your database Keep dbanalys & promon data for reference Install OpenEdge 10 (no need to delete /

overwrite Progress V9) Upgrade database to OE10 Run UPDATE STATISTICS for SQL Backup your database again! Recompile application code Ready ... Steady ... GO!!!

General Upgrade Strategy


Upgrading Database to OpenEdge 10

The 5-Minute Rule We Want More Rule

You have a choice



proutil <db> -C conv910 -B 512

Conversion utility runs in place, in 5 minutes or less

Mostly we upgrade the schema tables No changes to records or indexes No changes to physical structures

The 5-Minute Rule



Pros:• It is extremely quick and easy

• You can run the database

• It will very likely run better than before

• Most of your DBA scripts should work fine

Cons:• Fragmentation may still exist

• All data in Type I data storage areas

The 5-Minute Rule



Do some more work – move data to Data Storage Areas Type II

Upgrade with conv910 utility Move schema tables Create Data Storage Areas Type II Move tables and indexes Truncate old data storage areas Delete old data storage areas

We Want More Rule



Moving schema tables• proutil <db> -C mvsch

• Renumbers existing schema area

• Creates new schema area #6

• Copies schema tables

• Deletes old schema tables

We Want More Rule



Create Data Storage Areas Type II• prostrct add <db>

• Use database block size 4K or 8K• BI & AI block size should match each other• Type II Data Storage Area Cluster sizes:

– Tables areas: 512– Index areas: 512

• Note: D&L required to change database block size

We Want More Rule



Moving tables and indexes• proutil <db> -C tablemove | idxmove

• Can move just table, or table and its indexes

We Want More Rule



Truncate and delete old data storage areas• proutil <db> -C truncate bi• proutil <db> -C truncate area <area>

• After that data storage area is logically truncated and its extents can now be removed

• prostrct remove <db> <extent-type> <area-name>

We Want More Rule



If you don‘t like this approach, you still can do the famous Dump & Load

You will have to do it if you are changing database block size

D&L options:• Dictionary D&L• Custom D&L• Custom BUFFER-COPY• Binary D&L

We Want More Rule


Agenda

Summary of Advanced Storage Architecture Advanced Storage Architecture II Migrating to ASA II Bonus


What‘s new in OpenEdge 10.1B

Implemented INT64 data type 64-bit Database Keys – applies to Data

Storage Areas Type II only• Removes 2 billion row limit – pushed to more

than 9 quintillion• ROWID format has to have 8 additional

characters (0x0000000000000000) Database Limits pushed farther again

• 31994 areas * 1 petabyte per area =~32 exabytes of data

Summary



64-bit Sequences Large Index Key Entries

• Removes ~192 byte limit

• Allows ~1970 bytes of user data

1000 Area Limit• 32000 areas assumed

• Can be restricted by using -maxAreas startup parameter

Summary



proutil <db> -C describe proutil <db> -C enableseq64 proutil <db> -C enablelargekeys proutil <db> -C updatevst proutil <db> -C revert

Summary



Shared memory segment size on 32-bit platforms

Summary

Shared Memory 2GB

Max shmsegsize 128MB

Max #Segments 21

Shared Memory 4GB

Max shmsegsize 4GB

#Segments 32

Prior 10.1B 10.1B



Enable After-Imaging Online• probkup online <db> enableai <target>

• AI files must already exist (but can be added online too!)

• Online backup used as basis for roll-forward

• AI writers and replication must be enabled manually

Enable AI Online with AI Management

Summary


Question?

Tomáš Kuč[email protected]: tomas_kucera


Documents

New Generation of OpenEdge ® RDBMS Advanced Storage Architecture II Tomáš Kučera Principal Solution Engineer / EMEA Power Team