Adam Gwosdof, Applimation

1

WelcomeArchiving, Subsetting, Scrambling –Oracle Apps, PeopleSoft, Siebel, and Custom Applications

2

Welcome

• Let’s make today’s presentation interactive

• Who are you? And why are you Here?

– Functional?

– Technical?

• DBA

• Developer

• Superuser

• Applications and Modules causing “growing pains”

3

About me

• Adam Gwosdof – agwosdof@applimation.com

• Worked at Oracle Corporation in the Applications Division, starting in 1988

– Oracle Assets

– Oracle Payables

• Worked with Oracle Consulting through the early and mid 1990’s, doing Apps Implementations

• Now manage Applimation’s Oracle relationships

4

Applimation Overview

• Application Archiving

• Database Subsetting

• Data Masking

• Database Reorg

• Change Management

• SOX Compliance

• Segregation of Duties

• Security Reporting

CompanyBackground

Data GrowthManagement

IT Controls & Compliance

INFORMIA INTEGRA

• Headquartered in NYC

• 375+ Customers

• 105 Employees

• Investors: JMI Equity Fund & Battery Ventures

• Oracle, PeopleSoft & Siebel

5

Applimation Lifecycle Management

• Archive and Purge

• Combined access to historical and current transactions

• Repeatable Policy-based data movement

• Shrink test environments

• Mask sensitive data

…throughout the lifecycle

Challenge: Data Growth

Solution:

INFORMIA

• Enhance internal controls

• Manage impact of change

• Ensure application integrity

• Enforce Segregation of Duties rules

• Automate SOX compliance

…throughout the lifecycle

Challenge: Managing Change

Solution:

INTEGRA

6

Data Management ChallengesBenefits of archiving

7

The challenge of increasing data growth

Active data

Inactive data

Performance

Time

Dat

abas

e S

ize

BEFORE INFORMIA

• Growing storage costs

• Diminishing performance

• Increasing maintenance & compliance work

Benefits: AFTER INFORMIA

• Lower, stable storage costs

• Improved, stable performance

• Reduced maintenance & compliance work

Red

uced

costs

AFTER INFORMIA

• Reduced Total Cost of Ownership

Escalatin

g co

sts

8

Business reasons to manage growth

• Many copies of production

– Project teams require many testing and development environments

• Compliance requirements

– SOX regulating data retention

– Data must be retained for potential audits

• Application Upgrades

– Increased growth due to larger footprint

9

Data Management Options

• Purchase additional hardware

• DBAs tune application standard code

• Purge using Vendor provided purge routines if available

THE PROBLEM: constantly spending on hardware, tuning only does so much, purging doesn’t archive

10

Benefits of 3rd party solution

• Improve production environment performance

• Fully retire inactive data to offline storage devices

• Reduce size and resource requirements of the production environment

• Archive and purge inactive data

• Improve backup and recovery time requirements

• Maintain access to application data through familiar screens & reports

11

Key Strengths

• Speed and Scalability of Archiving / Purging Process– Faster than approaches based on vendor-proprietary purge

– Archive and purge are unified and optimized

• Applimation metadata-driven technology:– Does not mine archive logs

– Never requires specific Oracle patches or patch sets

– Data is never unloaded then reloaded to database!

• PeopleSoft and Siebel Certified

• Oracle Integration Validation

• Table/Index/Tablespace Reorg built in

• MetaStudio – productized developer’s edition

12

Over 115 Informia Customers

City of Chicago

13

Applimation InformiaSolution Overview

14

Informia-Enterprise Solution

Applimation Informia

InformiaArchive

InformiaSubset

InformiaReorgProduct

MetaStudio

DataGrowthMonitor

ApplimationDashboardPlatform

Database Support

OracleDatabase

Application Support

Oracle Apps(E-Business Suite)

PeopleSoftCustom or 3rd

Party ApplicationsSiebel

DB2/UDB&

DB2/zOS

SQL Server

15

• Users access ALL data seamlessly!

• Always Accessible• No Developer Modifications Needed• No Customizations Required

Seldom-used

Meeting the Challenge - Archive

Solution: Applimation Informia• Use Archive to identify inactive data, honoring retention

policy and business rules.

PRODUCTIONDATABASE

Actively-used data

ONLINE ARCHIVE DATABASE

• Then relocate data to archives:

Retire

Seldom-used

ENCAPSULATED XML ARCHIVES

Retired data

Onl

ine

Arc

hive

M

odul

e

XM

L A

rchi

ve

Mod

ule

16

Meeting the Challenge – Subset & Data Masking

Solution: Applimation Informia• Use Subset to flexibly create database subsets

Data for Module C

Data for Module B

Data for Module A

PRODUCTIONDATABASE

CLONE OF PRODUCTION DATABASE TO SUBSET

Subset for Module C

Subset for Module B

Subset for Module A

SUBSET OFPRODUCTION DATABASE

DATA MASK OFDATABASE SUBSET

• Optionally mask sensitive data in non-production databases

CLONE OFDATABASE SUBSETCLONE OF MASKEDDATABASE SUBSET

17

DATAFILE 1

DATAFILE 2

TABLE APart 1 of 2

TABLE B

TABLE CPart 1 of 2

TABLE APart 2 of 2

TABLE CPart 2 of 2

TABLE D

Free

DATAFILE 1

DATAFILE 2

TABLE APart 1 of 2

TABLE B

TABLE CPart 1 of 2

TABLE APart 2 of 2

TABLE CPart 2 of 2

TABLE D

Free

Meeting the Challenge - Reorg

Solution: Applimation Informia• Use Reorg to optimize tables, indexes, and tablespaces after archiving:

TABLESPACETABLESPACE

DATAFILE 1

Free

Free

TABLE A

TABLE B

TABLE C

TABLE D

Temporary DATAFILE

Temporary TABLESPACE

FreeCopy of TABLE A

Copy of TABLE B

Copy of TABLE C

Copy of TABLE D

18

MetaStudio – Developer’s Edition for Custom Archiving

Archive Engine

• Developer Toolkit to build and enhance metadata

• Identify tables to archive/purge

• Discover relationships

• Create or modify business rules

… for pre-supported business applications, or for entirely custom/in-house applications

Archive Engine

MetaStudio

Ora

cle

App

licat

ions

Peo

pleS

oft

Sie

bel

Cus

tom

/3rd

Par

ty

19

Online Archive

AM_AGENTSchema that contains Informia Archive engine

AM_HISTORYSchema that contains permanent repository of Online Archive data

ApplimationRepository (TOOLS)

AM_HOME

Schema that contains Informia seed data and programs

???

AM_RELOCTemporary relocation area – exists only during archive/purge process

APPS orSYSADM or

SIEBEL

BusinessApplication(s)

Production

AM_HISTORY

ApplimationRepository (TOOLS)

AM_HOME

Production

APPS orSYSADM or

SIEBEL

20

AM_RELOC

2003

AM_HISTORY

Online Archive Process

2004

2005

2006

Production Archive / History

2003

1. Archive 2003 Data to Relocation Area

2. Purge 2003 Data from Production

3. Merge 2003 Data to Online Archive

4. Validate; Drop Relocation Area

5. Resize tables/indexes based on purged data

6. Reclaim additional space from non-purged data

7. Right-size the underlying datafiles with revised growing room based on forward-looking retention policy

Optional process following archive

2004

2005

2006

21

Seamless Data Access

Archive DataProductionData

• Auto-generated• No user re-training• Access to:

• Current • Combined (current + history)

• No modifications to code base• Support for any/all query tools

22

Data Growth Monitor

On-demand view

• Current size

• Growth rate

Data distribution

• Tablespace

• Application

• Instance

• Server

• Storage device

23

1. Choose Entities

Create and Schedule “Policies”

2. Specify Parameters

3. Generate Candidates

24

Candidate exception reporting

All records within date range

Business rule exceptions are not archiveable

All archiveablerecords

25

1. Choose Entities

Online Archive Workbench

2. Specify Parameters

3. Generate Candidates

5. Relocate History Data

4. Process Archive/Purge

26

Oracle Integration Validation

Certified Archive Solution

27

PeopleSoft Certified

Certified Archive Solution

28

Siebel Validated Solution

Certified Archive Solution

29

Application and Customer Aware

• For your Oracle packaged modules

– Application-aware pre-configured “Entities”

– Oracle validated

– Policies compatible with your calendar periods

– Faster and more scalable than alternative approaches

• For your custom tables and applications

– MetaStudio – a complete workbench for managing custom data

30

Role-Based Security

• Supports Compliance & Segregation of Duties

• Roles include: Archive Administrator (DBA), Archive User (Policy Manager), Archive Scheduler (Operations), ArchiveDeveloper

• Keeps Administrator and Policy Managers apart

• “Who can Define” versus “Who can Do”

31

Oracle Applications 11i Archiving

Financials

Payables

Receivables

iReceivables

Cash Management

Fixed Assets

General Ledger

HR/Payroll

Collections

CRM

Contracts Core

Service Contracts

Field Service

Order Capture/Quoting/Sales On-Line

Telesales

Customer On-Line

iSupport

Supply Chain

Inventory

Order Management

Purchasing

Shipping

Workflow

Manufacturing

Cost/BOM

Work in Process

Project Accounting

32

PeopleSoft 7.5 through 8.9 Archiving

Human Resources

Payroll NA

Base Benefits

Benefit Administration

Human Capital Management

Financials

General Ledger

Accounts Payable

Accounts Receivable

Purchasing

Asset Management

Billing

Project Costing

Inventory

Time and Labor

33

Siebel 7.5, 7.7, 7.8 Archiving

Horizontals

Activities

Invoices

Messages

Opportunities

Orders

Quality – Change Requests

Quotes

Service Requests

Verticals

Activities

Invoices

Orders

Quality – Change Requests

Service Requests

34

Sample Retention Policies

Module Production Data Retention Archiving Frequency

Inventory 2 Months + Current Month Monthly

Cost History 2 Months + Current Month Monthly

WIP 2 Months + Current Month Monthly

Supplier Schedules 2 Months + Current Month Monthly

AP/Procurement 2-3 Months after PO gets finally closed Monthly

Cash Management 6 Months + Current Month Monthly

General Ledger 12+2 Months of adjustment period, after a period is permanently closed

Monthly

Quote Expired Quotes, older than 9 months will get deleted

Monthly

Workflow Delete workflows older than 15 days as part of regular Oracle workflow purge

Monthly

35

DATAFILE 1

DATAFILE 2

TABLE APart 1 of 2

TABLE B

TABLE CPart 1 of 2

TABLE APart 2 of 2

TABLE CPart 2 of 2

TABLE D

Free

DATAFILE 1

DATAFILE 2

TABLE APart 1 of 2

TABLE B

TABLE CPart 1 of 2

TABLE APart 2 of 2

TABLE CPart 2 of 2

TABLE D

Free

Informia Reorg

Solution: Applimation Informia• Use Reorg to optimize tables, indexes, and tablespaces after archiving:

TABLESPACETABLESPACE

DATAFILE 1

Free

Free

TABLE A

TABLE B

TABLE C

TABLE D

Temporary DATAFILE

Temporary TABLESPACE

FreeCopy of TABLE A

Copy of TABLE B

Copy of TABLE C

Copy of TABLE D

36

Data Masking

• Scramble or encrypt any data FK/PK surrogate keys

• Masking Methods– Built-in encryption

– Scrambling algorithm

– Easily extend to include one of your own

• De-identify sensitive data – Employees

– Customers

– Suppliers

• Perfect for training, testing, development databases

• Good for offshore development

37

Kick-Off

Kick-OffTechnicalSessions

ValidateDGA

2 Weeks

Archive Sample Project Plan

Project Timeline

Install Run Sample CyclesCreateData

Access

PlanNext

Steps

Module 1

Module 2

Module 3

TrainArchive User

ReviewBusiness

Rules

Conference Room Pilot

Installation and Training

1 Week

Refine

CycleTesting

ApproveBusiness

Rules

~ 6 Weeks

Simulate

DataAccessVerif.

Prod.Sims

Hardware/SoftwareConfigs

Setups

~ 6 Weeks

Deploy

GO-LIVE

Go-Live

38

Customer Support

• 24x7Availability

• Product Download Center

– Products

– Patches

– Documentation

– Tools

• Customer Support Center

– Submit Cases

– View and Comment on Open/Closed Cases

– Solutions Center – Knowledge Base

• Patch Development performed within Customer Support

• Interface with Product Development for enhancement requests and long-term problem resolution

39

Client: Electronic Arts

• Industry: Multimedia and Graphics Software

• Modules Archived: AR, INV, OE/ONT Business Rules Modified: None

Customizations Accommodated: Added 27 custom tables

• Environment: Oracle Apps 11.5.9, DB Version 9.2.0.5 RAC

• Implementation timeline: – Phase 1 1/24/05, Testing in February/March then 3 week delay until UAT, UAT

testing April/May.

– Archive Production Cycles started June, 2005 • Key project highlights: Archives will run every weekend from June to

November until data retention policies are caught up

40

Client: AT&T Wireless

• Industry: Telecommunications

• Modules Archived: FA, INV, AP, PO, OE/ONT Business Rules Modified: Upgrade to 11i affected last_update_date rules

Customizations Accommodated: Added custom tables to INV entity

• Environment: Oracle Apps 11.5.7/11.5.8 mixed, DB Version 9.2.0.4

• Implementation timeline: – Phase 1 7/6/04, Changed priority of modules to put FA first due to end of month

close issues

– Archive Production Cycles started November, 2004 • Key project highlights: Archived out 1998 through 2002 data to reduce

database by 50%. Performed reorg to reclaim space. Monthly archives planned.

41

Client: IKON

• Industry: Office Solutions

• Modules Archived: OM, PO, INV Business Rules Modified: Created rule to check open payment schedules for OM

Customizations Accommodated: Added custom tables

• Environment: Oracle Apps 11.5.9, DB Version 9.2.0.5 RAC

• Implementation timeline: – Phase 1 January 2004, Implementation of Subset and Archive concurrent so

priorities shifted often, Delayed to prioritize upgrade from 11.5.8 to 11.5.9 in November, 2004

– Archive Production Cycles started May, 2005 • Key project highlights: Archived out 540GB in first cycle, Archive will

run quarterly, Field Service and Contracts are next to Go Live

42

Customer Quotes

“Thank you - The superb support staff at Applimation who made me feel like I was their only customer and my issue was their most important concern” - Peter Plackowski

“Applimation's Service and Support team surpassed any support team we've ever worked with.  They were on-call 24/7 and addressed all our questions and concerns so we could achieve a smooth archive and go-live. We have not only saved substantial costs is disk storage, we have improved the overall performance of our system” - Bob Peery

“Excellent service. Always available to help” - Chris Carriero

43

Summary

• Anticipation of performance and storage issues might actually be your “ticket” towards proactively addressing data retention and compliance issues.

• Data Growth Management is an interdisciplinary challenge – you need to build consensus among technical, functional, and management

• Start Soon – with budgeting and implementation timeframes, your time horizon is now