Embed Size (px)
DESCRIPTION
Citation preview
1
WelcomeArchiving, Subsetting, Scrambling –Oracle Apps, PeopleSoft, Siebel, and Custom Applications
2
Welcome
• Let’s make today’s presentation interactive
• Who are you? And why are you Here?
– Functional?
– Technical?
• DBA
• Developer
• Superuser
• Applications and Modules causing “growing pains”
3
About me
• Adam Gwosdof – [email protected]
• Worked at Oracle Corporation in the Applications Division, starting in 1988
– Oracle Assets
– Oracle Payables
• Worked with Oracle Consulting through the early and mid 1990’s, doing Apps Implementations
• Now manage Applimation’s Oracle relationships
4
Applimation Overview
• Application Archiving
• Database Subsetting
• Data Masking
• Database Reorg
• Change Management
• SOX Compliance
• Segregation of Duties
• Security Reporting
CompanyBackground
Data GrowthManagement
IT Controls & Compliance
INFORMIA INTEGRA
• Headquartered in NYC
• 375+ Customers
• 105 Employees
• Investors: JMI Equity Fund & Battery Ventures
• Oracle, PeopleSoft & Siebel
5
Applimation Lifecycle Management
• Archive and Purge
• Combined access to historical and current transactions
• Repeatable Policy-based data movement
• Shrink test environments
• Mask sensitive data
…throughout the lifecycle
Challenge: Data Growth
Solution:
INFORMIA
• Enhance internal controls
• Manage impact of change
• Ensure application integrity
• Enforce Segregation of Duties rules
• Automate SOX compliance
…throughout the lifecycle
Challenge: Managing Change
Solution:
INTEGRA
6
Data Management ChallengesBenefits of archiving
7
The challenge of increasing data growth
Active data
Inactive data
Performance
Time
Dat
abas
e S
ize
BEFORE INFORMIA
• Growing storage costs
• Diminishing performance
• Increasing maintenance & compliance work
Benefits: AFTER INFORMIA
• Lower, stable storage costs
• Improved, stable performance
• Reduced maintenance & compliance work
Red
uced
costs
AFTER INFORMIA
• Reduced Total Cost of Ownership
Escalatin
g co
sts
8
Business reasons to manage growth
• Many copies of production
– Project teams require many testing and development environments
• Compliance requirements
– SOX regulating data retention
– Data must be retained for potential audits
• Application Upgrades
– Increased growth due to larger footprint
9
Data Management Options
• Purchase additional hardware
• DBAs tune application standard code
• Purge using Vendor provided purge routines if available
THE PROBLEM: constantly spending on hardware, tuning only does so much, purging doesn’t archive
10
Benefits of 3rd party solution
• Improve production environment performance
• Fully retire inactive data to offline storage devices
• Reduce size and resource requirements of the production environment
• Archive and purge inactive data
• Improve backup and recovery time requirements
• Maintain access to application data through familiar screens & reports
11
Key Strengths
• Speed and Scalability of Archiving / Purging Process– Faster than approaches based on vendor-proprietary purge
– Archive and purge are unified and optimized
• Applimation metadata-driven technology:– Does not mine archive logs
– Never requires specific Oracle patches or patch sets
– Data is never unloaded then reloaded to database!
• PeopleSoft and Siebel Certified
• Oracle Integration Validation
• Table/Index/Tablespace Reorg built in
• MetaStudio – productized developer’s edition
12
Over 115 Informia Customers
City of Chicago
13
Applimation InformiaSolution Overview
14
Informia-Enterprise Solution
Applimation Informia
InformiaArchive
InformiaSubset
InformiaReorgProduct
MetaStudio
DataGrowthMonitor
ApplimationDashboardPlatform
Database Support
OracleDatabase
Application Support
Oracle Apps(E-Business Suite)
PeopleSoftCustom or 3rd
Party ApplicationsSiebel
DB2/UDB&
DB2/zOS
SQL Server
15
• Users access ALL data seamlessly!
• Always Accessible• No Developer Modifications Needed• No Customizations Required
Seldom-used
Meeting the Challenge - Archive
Solution: Applimation Informia• Use Archive to identify inactive data, honoring retention
policy and business rules.
PRODUCTIONDATABASE
Actively-used data
ONLINE ARCHIVE DATABASE
• Then relocate data to archives:
Retire
Seldom-used
ENCAPSULATED XML ARCHIVES
Retired data
Onl
ine
Arc
hive
M
odul
e
XM
L A
rchi
ve
Mod
ule
16
Meeting the Challenge – Subset & Data Masking
Solution: Applimation Informia• Use Subset to flexibly create database subsets
Data for Module C
Data for Module B
Data for Module A
PRODUCTIONDATABASE
CLONE OF PRODUCTION DATABASE TO SUBSET
Subset for Module C
Subset for Module B
Subset for Module A
SUBSET OFPRODUCTION DATABASE
DATA MASK OFDATABASE SUBSET
• Optionally mask sensitive data in non-production databases
CLONE OFDATABASE SUBSETCLONE OF MASKEDDATABASE SUBSET
17
DATAFILE 1
DATAFILE 2
TABLE APart 1 of 2
TABLE B
TABLE CPart 1 of 2
TABLE APart 2 of 2
TABLE CPart 2 of 2
TABLE D
Free
DATAFILE 1
DATAFILE 2
TABLE APart 1 of 2
TABLE B
TABLE CPart 1 of 2
TABLE APart 2 of 2
TABLE CPart 2 of 2
TABLE D
Free
Meeting the Challenge - Reorg
Solution: Applimation Informia• Use Reorg to optimize tables, indexes, and tablespaces after archiving:
TABLESPACETABLESPACE
DATAFILE 1
Free
Free
TABLE A
TABLE B
TABLE C
TABLE D
Temporary DATAFILE
Temporary TABLESPACE
FreeCopy of TABLE A
Copy of TABLE B
Copy of TABLE C
Copy of TABLE D
18
MetaStudio – Developer’s Edition for Custom Archiving
Archive Engine
• Developer Toolkit to build and enhance metadata
• Identify tables to archive/purge
• Discover relationships
• Create or modify business rules
… for pre-supported business applications, or for entirely custom/in-house applications
Archive Engine
MetaStudio
Ora
cle
App
licat
ions
Peo
pleS
oft
Sie
bel
Cus
tom
/3rd
Par
ty
19
Online Archive
AM_AGENTSchema that contains Informia Archive engine
AM_HISTORYSchema that contains permanent repository of Online Archive data
ApplimationRepository (TOOLS)
AM_HOME
Schema that contains Informia seed data and programs
???
AM_RELOCTemporary relocation area – exists only during archive/purge process
APPS orSYSADM or
SIEBEL
BusinessApplication(s)
Production
AM_HISTORY
ApplimationRepository (TOOLS)
AM_HOME
Production
APPS orSYSADM or
SIEBEL
20
AM_RELOC
2003
AM_HISTORY
Online Archive Process
2004
2005
2006
Production Archive / History
2003
1. Archive 2003 Data to Relocation Area
2. Purge 2003 Data from Production
3. Merge 2003 Data to Online Archive
4. Validate; Drop Relocation Area
5. Resize tables/indexes based on purged data
6. Reclaim additional space from non-purged data
7. Right-size the underlying datafiles with revised growing room based on forward-looking retention policy
Optional process following archive
2004
2005
2006
21
Seamless Data Access
Archive DataProductionData
• Auto-generated• No user re-training• Access to:
• Current • Combined (current + history)
• No modifications to code base• Support for any/all query tools
22
Data Growth Monitor
On-demand view
• Current size
• Growth rate
Data distribution
• Tablespace
• Application
• Instance
• Server
• Storage device
23
1. Choose Entities
Create and Schedule “Policies”
2. Specify Parameters
3. Generate Candidates
24
Candidate exception reporting
All records within date range
Business rule exceptions are not archiveable
All archiveablerecords
25
1. Choose Entities
Online Archive Workbench
2. Specify Parameters
3. Generate Candidates
5. Relocate History Data
4. Process Archive/Purge
26
Oracle Integration Validation
Certified Archive Solution
27
PeopleSoft Certified
Certified Archive Solution
28
Siebel Validated Solution
Certified Archive Solution
29
Application and Customer Aware
• For your Oracle packaged modules
– Application-aware pre-configured “Entities”
– Oracle validated
– Policies compatible with your calendar periods
– Faster and more scalable than alternative approaches
• For your custom tables and applications
– MetaStudio – a complete workbench for managing custom data
30
Role-Based Security
• Supports Compliance & Segregation of Duties
• Roles include: Archive Administrator (DBA), Archive User (Policy Manager), Archive Scheduler (Operations), ArchiveDeveloper
• Keeps Administrator and Policy Managers apart
• “Who can Define” versus “Who can Do”
31
Oracle Applications 11i Archiving
Financials
Payables
Receivables
iReceivables
Cash Management
Fixed Assets
General Ledger
HR/Payroll
Collections
CRM
Contracts Core
Service Contracts
Field Service
Order Capture/Quoting/Sales On-Line
Telesales
Customer On-Line
iSupport
Supply Chain
Inventory
Order Management
Purchasing
Shipping
Workflow
Manufacturing
Cost/BOM
Work in Process
Project Accounting
32
PeopleSoft 7.5 through 8.9 Archiving
Human Resources
Payroll NA
Base Benefits
Benefit Administration
Human Capital Management
Financials
General Ledger
Accounts Payable
Accounts Receivable
Purchasing
Asset Management
Billing
Project Costing
Inventory
Time and Labor
33
Siebel 7.5, 7.7, 7.8 Archiving
Horizontals
Activities
Invoices
Messages
Opportunities
Orders
Quality – Change Requests
Quotes
Service Requests
Verticals
Activities
Invoices
Orders
Quality – Change Requests
Service Requests
34
Sample Retention Policies
Module Production Data Retention Archiving Frequency
Inventory 2 Months + Current Month Monthly
Cost History 2 Months + Current Month Monthly
WIP 2 Months + Current Month Monthly
Supplier Schedules 2 Months + Current Month Monthly
AP/Procurement 2-3 Months after PO gets finally closed Monthly
Cash Management 6 Months + Current Month Monthly
General Ledger 12+2 Months of adjustment period, after a period is permanently closed
Monthly
Quote Expired Quotes, older than 9 months will get deleted
Monthly
Workflow Delete workflows older than 15 days as part of regular Oracle workflow purge
Monthly
35
DATAFILE 1
DATAFILE 2
TABLE APart 1 of 2
TABLE B
TABLE CPart 1 of 2
TABLE APart 2 of 2
TABLE CPart 2 of 2
TABLE D
Free
DATAFILE 1
DATAFILE 2
TABLE APart 1 of 2
TABLE B
TABLE CPart 1 of 2
TABLE APart 2 of 2
TABLE CPart 2 of 2
TABLE D
Free
Informia Reorg
Solution: Applimation Informia• Use Reorg to optimize tables, indexes, and tablespaces after archiving:
TABLESPACETABLESPACE
DATAFILE 1
Free
Free
TABLE A
TABLE B
TABLE C
TABLE D
Temporary DATAFILE
Temporary TABLESPACE
FreeCopy of TABLE A
Copy of TABLE B
Copy of TABLE C
Copy of TABLE D
36
Data Masking
• Scramble or encrypt any data FK/PK surrogate keys
• Masking Methods– Built-in encryption
– Scrambling algorithm
– Easily extend to include one of your own
• De-identify sensitive data – Employees
– Customers
– Suppliers
• Perfect for training, testing, development databases
• Good for offshore development
37
Kick-Off
Kick-OffTechnicalSessions
ValidateDGA
2 Weeks
Archive Sample Project Plan
Project Timeline
Install Run Sample CyclesCreateData
Access
PlanNext
Steps
Module 1
Module 2
Module 3
TrainArchive User
ReviewBusiness
Rules
Conference Room Pilot
Installation and Training
1 Week
Refine
CycleTesting
ApproveBusiness
Rules
~ 6 Weeks
Simulate
DataAccessVerif.
Prod.Sims
Hardware/SoftwareConfigs
Setups
~ 6 Weeks
Deploy
GO-LIVE
Go-Live
38
Customer Support
• 24x7Availability
• Product Download Center
– Products
– Patches
– Documentation
– Tools
• Customer Support Center
– Submit Cases
– View and Comment on Open/Closed Cases
– Solutions Center – Knowledge Base
• Patch Development performed within Customer Support
• Interface with Product Development for enhancement requests and long-term problem resolution
39
Client: Electronic Arts
• Industry: Multimedia and Graphics Software
• Modules Archived: AR, INV, OE/ONT Business Rules Modified: None
Customizations Accommodated: Added 27 custom tables
• Environment: Oracle Apps 11.5.9, DB Version 9.2.0.5 RAC
• Implementation timeline: – Phase 1 1/24/05, Testing in February/March then 3 week delay until UAT, UAT
testing April/May.
– Archive Production Cycles started June, 2005 • Key project highlights: Archives will run every weekend from June to
November until data retention policies are caught up
40
Client: AT&T Wireless
• Industry: Telecommunications
• Modules Archived: FA, INV, AP, PO, OE/ONT Business Rules Modified: Upgrade to 11i affected last_update_date rules
Customizations Accommodated: Added custom tables to INV entity
• Environment: Oracle Apps 11.5.7/11.5.8 mixed, DB Version 9.2.0.4
• Implementation timeline: – Phase 1 7/6/04, Changed priority of modules to put FA first due to end of month
close issues
– Archive Production Cycles started November, 2004 • Key project highlights: Archived out 1998 through 2002 data to reduce
database by 50%. Performed reorg to reclaim space. Monthly archives planned.
41
Client: IKON
• Industry: Office Solutions
• Modules Archived: OM, PO, INV Business Rules Modified: Created rule to check open payment schedules for OM
Customizations Accommodated: Added custom tables
• Environment: Oracle Apps 11.5.9, DB Version 9.2.0.5 RAC
• Implementation timeline: – Phase 1 January 2004, Implementation of Subset and Archive concurrent so
priorities shifted often, Delayed to prioritize upgrade from 11.5.8 to 11.5.9 in November, 2004
– Archive Production Cycles started May, 2005 • Key project highlights: Archived out 540GB in first cycle, Archive will
run quarterly, Field Service and Contracts are next to Go Live
42
Customer Quotes
“Thank you - The superb support staff at Applimation who made me feel like I was their only customer and my issue was their most important concern” - Peter Plackowski
“Applimation's Service and Support team surpassed any support team we've ever worked with. They were on-call 24/7 and addressed all our questions and concerns so we could achieve a smooth archive and go-live. We have not only saved substantial costs is disk storage, we have improved the overall performance of our system” - Bob Peery
“Excellent service. Always available to help” - Chris Carriero
43
Summary
• Anticipation of performance and storage issues might actually be your “ticket” towards proactively addressing data retention and compliance issues.
• Data Growth Management is an interdisciplinary challenge – you need to build consensus among technical, functional, and management
• Start Soon – with budgeting and implementation timeframes, your time horizon is now
![The acquisition of simple sentences. One-word utterances / holophrases Daddy.[Adam 1;4] Mommy.[Adam 1;4] Doggy.[Adam 1;5] Goodbye.[Adam 1;5] Allgone.[Adam](https://img.pdfslide.us/doc/110x75/56649d005503460f949d29e8/the-acquisition-of-simple-sentences-one-word-utterances-holophrases-daddyadam.jpg)
