Auditing: Do You Care Who Did What, When, Where and How? (OpenEdge™ 10.1)
Angelo TracannaSenior Manager, OpenEdge Data Management
2 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Agenda Slide
Why auditing? Auditing in OpenEdge® overview Using auditing in OpenEdge Auditing best practices Future thinking
3 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
D I S C L A I M E R
Under Development
This talk includes information about potential future products and/or product enhancements.
What I am going to say reflects our current thinking, but the information contained herein is preliminary and subject to change. Any future products we ultimately deliver may be materially different from what is described here.
D I S C L A I M E R
4 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Core Business Services Vision Statement
“Provide a comprehensive set of
common business services that provide
the core feature support
of a modern SOA based application
modeled on the
OpenEdge Reference Architecture”
5 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Core Service: AuditingMission Statement
“Provide an auditing framework that can
supply an uninterrupted
trail of an application client’s access
to its operations and data”
6 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
2. Secure auditing to support non-repudiation of audit data
3. High performance, scalable and efficient storage of audit data
1. Compliance with international Government regulations, e.g.
Sarbanes-Oxley Act, CFR Part 11, HIPAA, European Union’s Annex 11, European Union Data Protection Directive, etc.
Customer Key Drivers for Auditing
4. Consistency across SQL, 4GL and database utilities
7 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
The Regulatory Compliance ChallengeOrganizations today face a growing number of regulations that
mandate the accuracy and reliability of information – not just SOX!
21 CFR Part 11
Basel II
EU Data ProtectionDirective
Sarbanes-Oxley Act
Gramm-Leach-BlileyAct
HIPAA
8 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Regulatory Compliance:Sarbanes-Oxley (SOX)
Enacted July 30, 2002 Designed to protect investors Senior management and advisors
personally accountable Financial information must
remain confidential and privileged
Data integrity and quality are key
What is it?
Section 404 — Internal Controls
annual evaluation and documentation of the internal controls and procedures in place to produce financial information
Section 302 — Certifying Results
CEO and CFO to certify the existence of controls and sign-off on the organization’s financial statements
Section 409 — Reporting
Sarbanes-Oxley Act
9 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Regulatory Compliance:Example Auditor Objectives
Example auditor objectives Logging and reporting
Determine who modified the database schema and when
List all schema changes by date by user
Determine who has access to particular systems
List of resources and the users who are authorized to access them
Ensure user accountability For a particular user, list their actions over a period of time
Ensure terminated employees access rights are terminated in a timely manner
For users who are terminated, list access rights and when they were revoked
11 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Compliance doesn’t impact me…?
Do you supply accounting system software? Do you supply software for the healthcare
industry? Do you provide solutions to the government? Do your applications contain confidential data? Might somebody in the state of California buy
your application? Have you any growth aspirations?
Think again!
12 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Think about the Opportunities!
Competitive advantage– Places a in the compliance box
Enhanced functionality sells apps– Security
– Audit trails
– BI reporting New product / service offerings
14 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Secure Auditing is Key to Compliance
Audit trails can tell you who did what, when, where and how
Must reflect the verifiable identify of the real application user
Must be complete, accurate and non-reputable– Prove audit policy and data has not been
tampered with
15 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Auditing Has a Cost!
Audit trails can generate large volumes of data – quickly!
Audit trails always add overhead– The more you audit – the bigger the
performance overhead! Must audit all methods of access to the
application and data– Compromises integration otherwise
16 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Introducing…Auditing in OpenEdge®Surviving the Auditor!
17 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
OpenEdge Database Schema-Trigger Based Auditing
4GL
Clie
nt
Audit PolicyTools
Application Code
Ap
pli
cati
on
D
ata
App DB
OfflineAuditData
OfflineAuditData
ArchiveDaemon
Arc
hiv
eM
anag
erAudit Data
Archive DB
Audit EventManager(schema triggers)
Audit Data
Audit Data Manager
Audit Policy ManagerA
PI
Policy Data
Sec
uri
ty M
anag
er
AuditReport
Rep
ort
Man
ager
SQ
L C
lien
t
Application Code
18 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Auditing Architecture Overview4G
L C
lien
t
DB Tools & Utilities
Open Tools
Audit Policy Tools (APMT)
Application Code
SQ
L C
lien
t
Application Code
Audit Data A
pp
lica
tio
n
Dat
a
Policy Data
App DB
Audit Data
Archive DB
Audit EventSubsystem
Audit EventSubsystem
Dat
abas
e
Inte
rnal
Ap
pli
cati
on
Sec
uri
ty S
ub
syst
emS
ecu
rity
Su
bsy
stem
Audit Data Subsystem
Audit Data Subsystem
OfflineAuditData
OfflineAuditData
AuditReport
Audit Policy Subsystem
Audit Policy SubsystemA
PI
ArchiveDaemon A
rch
ivin
g S
ub
syst
em
Arc
hiv
ing
Su
bsy
stem
Rep
orti
ng
Su
bsy
stem
19 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Audit Data A
pp
lica
tio
n
Dat
a
includes
includes
includes
record reads on
record deletes on
record updates onrecord creates on
is controlled by
_aud-audit-policy
_Audit-policy-guid
_Audit-policy-name (AK1.1)_Audit-policy-description (IE1.1)_Audit-data-security-level_Audit-custom-detail-level_Audit-policy-active (IE2.1)
_aud-file-policy
_Audit-policy-guid (FK)_File-Name (IE1.1)_Owner (IE1.2)
_Audit-create-level_Audit-create-criteria_Audit-update-level_Audit-update-criteria_Audit-delete-level_Audit-delete-criteria_Audit-read-level_Audit-read-criteria_Create-event-id (FK) (IE2.1)_Update-event-id (FK) (IE3.1)_Delete-event-id (FK) (IE4.1)_Read-event-id (FK) (IE5.1)
_aud-field-policy
_Audit-policy-guid (FK)_File-Name (FK) (IE1.1)_Owner (FK) (IE1.2)_Field-Name (IE1.3)
_Audit-create-level_Audit-update-level_Audit-delete-level_Audit-read-level_Audit-identifying-field
_aud-event-policy
_Audit-policy-guid (FK)_Event-id (FK) (IE1.1)
_Event-level_Event-criteria
_aud-event
_Event-id
_Event-type (IE1.1)_Event-name (IE1.2)_Event-description (IE2.1)
Multiple active policies
Control by table / CUD operation
Audit Policy MetaSchema
Override individual fieldsInternal & application defined audit events
Control by event Id
AuditPolicy
FilePolicy Field
Policy
EventPolicy
AuditEvent
Policy Data
20 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Audit Policy MetaSchema
includes
includes
includes
record reads on
record deletes on
record updates onrecord creates on
is controlled by
_aud-audit-policy
_Audit-policy-guid
_Audit-policy-name (AK1.1)_Audit-policy-description (IE1.1)_Audit-data-security-level_Audit-custom-detail-level_Audit-policy-active (IE2.1)
_aud-file-policy
_Audit-policy-guid (FK)_File-Name (IE1.1)_Owner (IE1.2)
_Audit-create-level_Audit-create-criteria_Audit-update-level_Audit-update-criteria_Audit-delete-level_Audit-delete-criteria_Audit-read-level_Audit-read-criteria_Create-event-id (FK) (IE2.1)_Update-event-id (FK) (IE3.1)_Delete-event-id (FK) (IE4.1)_Read-event-id (FK) (IE5.1)
_aud-field-policy
_Audit-policy-guid (FK)_File-Name (FK) (IE1.1)_Owner (FK) (IE1.2)_Field-Name (IE1.3)
_Audit-create-level_Audit-update-level_Audit-delete-level_Audit-read-level_Audit-identifying-field
_aud-event-policy
_Audit-policy-guid (FK)_Event-id (FK) (IE1.1)
_Event-level_Event-criteria
_aud-event
_Event-id
_Event-type (IE1.1)_Event-name (IE1.2)_Event-description (IE2.1)
Multiple active policies
Control by table / CUD operation
Override individual fieldsInternal & application defined audit events
Control by event id
21 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
is the group for
supplies context to
consists of
created
_aud-audit-data
_Audit-data-guid
_Database-connection-id (IE1.1)_Client-session-uuid (FK) (IE1.2)_User-id (IE2.1)_Audit-date-time (IE5.1)_Audit-event-group (FK) (IE3.1)_Db-guid (FK) (IE3.2)_Transaction-id (IE3.3)_Transaction-sequence (IE3.4)_Event-id (FK) (IE4.1)_Event-context (IE6.1)_Application-context-id (FK) (IE7.1)_Event-detail_Audit-custom-detail_Audit-data-security-level_Data-seal
_aud-audit-data-value
_Audit-data-guid (FK)_Field-name (IE1.1)_Continuation-sequence
_Data-type-code_Old-string-value_New-string-value_Old-blob-value_New-blob-value_Old-clob-value_New-clob-value_Audit-data-security-level_Data-seal
_client-session
_Client-session-uuid
_Client-name_User-id (IE1.1)_Authentication-date-time (IE2.1)_Server-uuid_Authentication-domain-type_Authentication-domain-name_Db-guid (FK) (IE3.1)_Session-custom-detail_Audit-data-security-level_Data-seal
Ap
pli
cati
on
D
ata
Policy Data
Audit Data MetaSchemaRecord client session
information
Configurable automated audit data with optional
context & grouping
Optional old/new value recording
ClientSession
AuditData
AuditData
Values
Audit Data
22 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
is the group for
supplies context to
consists of
created
_aud-audit-data
_Audit-data-guid
_Database-connection-id (IE1.1)_Client-session-uuid (FK) (IE1.2)_User-id (IE2.1)_Audit-date-time (IE5.1)_Audit-event-group (FK) (IE3.1)_Db-guid (FK) (IE3.2)_Transaction-id (IE3.3)_Transaction-sequence (IE3.4)_Event-id (FK) (IE4.1)_Event-context (IE6.1)_Application-context-id (FK) (IE7.1)_Event-detail_Audit-custom-detail_Audit-data-security-level_Data-seal
_aud-audit-data-value
_Audit-data-guid (FK)_Field-name (IE1.1)_Continuation-sequence
_Data-type-code_Old-string-value_New-string-value_Old-blob-value_New-blob-value_Old-clob-value_New-clob-value_Audit-data-security-level_Data-seal
_client-session
_Client-session-uuid
_Client-name_User-id (IE1.1)_Authentication-date-time (IE2.1)_Server-uuid_Authentication-domain-type_Authentication-domain-name_Db-guid (FK) (IE3.1)_Session-custom-detail_Audit-data-security-level_Data-seal
Audit Data MetaSchemaRecord client session
information
Configurable automated audit data with optional
context & grouping
Optional old/new value recording
23 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
What gets Audited?
Record level events– Create event
– Update event
– Delete event No need for schema triggers
– controlled through file / field policy
Audit EventSubsystem
Audit EventSubsystem
Dat
abas
eDatabase events
24 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
What gets Audited?
Authentication (login) Database connections Schema changes Audit policy administration Security administration Database utilities
– Start, stop, backup, restore, dump, load, copy, etc.
Audit archiving
Audit EventSubsystem
Audit EventSubsystem
Inte
rnalInternal events
25 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
What gets Audited?
Application context Audit event groups Application defined events (ID > 32000)
– For events with no corresponding database operation
– Fully control granularity and detail, e.g. 1 audit record for dispatch of an order
– Can be used for read auditing– Group into ranges to simplify reporting
Audit EventSubsystem
Audit EventSubsystem
Ap
pli
cati
on
Application events
26 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Securing Audit Data
Separation of duty– Audit administrator
– Audit archiver No updates to audit data Prevents deletion of events, e.g. archive Audit data is sealed to prevent tampering Secured administration tools and utilities
Sec
uri
ty S
ub
syst
emS
ecu
rity
Su
bsy
stem
27 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Recording the Real User
Trusted authentication systems / domains– Assert verified identity of real application
user
– Not dependent on _user records No blank user access to audit data!
Sec
uri
ty S
ub
syst
emS
ecu
rity
Su
bsy
stem
28 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Managing Audit Data
PROUTIL commands to enable auditing Unique database ID (GUID) and
description Audit Policy Maintenance Tool (APMT)
– Open API
– Securely deploy policies (via text dump or XML)
Secure dump/load options for audit data
DB Tools & Utilities
DB Tools & Utilities
Audit Policy Tools (APMT)
Audit Policy Tools (APMT)
29 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Archiving Audit Data
Auditing can generate lots of data – fast!– Consider 2 billion row limit
– Move audit data from short term to long term storage
– Delete audit data no longer required Fast audit archive (binary)
– Select by date range
– Output to bit bucket
– Output to binary file Ability to write custom archiver
Archiving Subsystem
Archiving Subsystem
30 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Querying Audit Data
Only users granted read permission– (Except SQL for Open Edge 10.1a)
Exposed as standard database tables Requires knowledge of schema
– Both application schema and metaschema– What are the identifying fields?– How is the context formatted? (Varies by event id)
Sample reports supplied Audit data searchable by
– User id, event id, date, context, transaction, audit group, DB connection, client session
Reporting Subsystem
Reporting Subsystem
31 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Auditing in OpenEdge® Key Value-Add
Common built-in auditing for both SQL/4GL clients Flexible audit policy management Secure audit data, policy and utilities
– Separation of duty– Purposed audit permissions– Verified user identity– Secure utilities and sealed data
Internal audit events (utilities, schema changes, etc.) Performance, performance, performance High performance archiving Multi-platform – not just Windows!
Why use it in place of own solution?
33 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Agenda Slide
Why auditing? Auditing in OpenEdge overview Using auditing in OpenEdge® Auditing best practices Future thinking
34 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Auditing in OpenEdge®Getting Started
Upgrade client / database to OpenEdge® 10.1a Enable auditing
Connect to database as the DBA Set up database security key via data admin Run APMT to load / enable shipped policies
Proutil dbname –C enableauditing area area1 [indexarea Area2] [deactivateidx]
Enabling auditing for internal events
35 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Auditing in OpenEdgeGetting Started
Run APMT– Add new policies for application schema
– Configure tables / fields to audit
– Enable policies Run application as normal Query audit data via sample or custom
reports
Enabling auditing for database events
36 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Auditing in OpenEdgeGetting Started
Modify database options to use application user id for auditing
Setup authentication system / domains Add application startup code to load
trusted domains
Asserting the real application user
SECURITY-POLICY:LOAD-DOMAINS(db)
37 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Auditing in OpenEdgeGetting Started
Modify 4GL application authentication code
Asserting the real application user
CREATE CLIENT-PRINCIPAL hCPhCp:USER-ID = “aswindel”hCp:DOMAIN-NAME = “progress”…result = hCP:SEAL(“pass phrase”)SECURITY-POLICY:SET-CLIENT(hCP)…hCP:LOGOUTDELETE hCP
Audited
Audited
38 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Auditing in OpenEdgeGetting Started
Set / clear context at appropriate places in application
Add code to record application events
ctx-id = AUDIT-CONTROL:SET-APPL-CONTEXT(audit-event-ctx[,event-detail[,user-data]])…id = AUDIT-CONTROL:LOG-AUDIT-EVENT (event-id, event-context[, event-detail[, user-data]])…AUDIT-CONTROL:CLEAR-APPL-CONTEXT
Supplying context / application events
39 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Auditing Best Practices
Consider application database as short term storage for audit data– Do not enable audit indexes– Use separate storage area for audit data– Archive often!
Use purposed database for audit archive / reporting Only audit what is absolutely necessary – tune with APMT Plan for reporting
– Group event ids into ranges– Structure context consistently– Leverage audit event groups
Coding style even more important (assigns, record scope, transaction scope)
40 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
OpenEdge Auditing Futures (beyond R10.1a)
Selective audit policy for specific data Read auditing More standard reporting Direct archiving to OpenEdge database SQL audit policy administration IDE integration
What else would you like to see???
Current thinking
41 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
In Summary Govt. regulations & security are a concern and an
opportunity– Start preparing for them NOW!
Upgrade to OpenEdge® 10.1 when it ships– OpenEdge auditing helps you survive the auditor
OpenEdge auditing adds value to your existing applications for minimal effort
Avoid Jail!
43 Auditing: Who Did What, When, Where, How? © 2005 Progress Software Corporation
Thank you for your time!