RDBMS Progress, Oracle and SQL Server in relationship with Big Brother/Big Data

RDBMS Progress, Oracle and SQL Server

in relationship with Big Brother/Big Data

Dmitri Levin

• Work with Progress since 1992• Work with Oracle since 2003• MS SQL Server – not much

• Microsoft Certified Professional (SQL-2012)• Oracle Certified Associate (11g)• Progress – not certified

PUG Challenge Americas 2014 2

Relational Databases Big Data / NoSQL

3PUG Challenge Americas 2014

( humongous )

NoSQL Types


Key-value store Amazon DynamoDB, Oracle NoSQL

Column Store Cassandra, Hbase, BigTable

Document databases MongoDB (JSON)

Graph stores Neo4j, HyperGraphDB

BigData ≈ Kind of “proutil db –C load –i”


A top speed load

No durability

RDBMS BigDataRelational Yes No Basic unit Row Document


Schema Yes No

PUG Challenge Americas 2014

RDBMS NoSQLRow Document

Document{ “name” : “Dmitri” , “last name” : “levin”, “date of birth” : 04/13/64 }

Document{ “name” : “Dmitri” , “name” : “John” }

7


RDBMS NoSQLRow DocumentTable Collection

Collection of Documents{ “name” : “Dmitri” , “lname” : “levin”, “dob” : 04/13/64 } { “name” : “John” , “lname” : “Doe”} { “foo” : 3 }

8


RDBMS NoSQLDatabase DatabaseTable CollectionRow DocumentColumn Field=key-value pairIndex IndexPrimary Key _id field

9


Database

Table

Row

Row

Database

Collection

Document

Document

10

Table

Row

Row

Collection

Document

Document

RDBMS NoSQL

Scaling Up Out $$

(NoSQL)


RDBMS BigDataRelational Yes No Basic unit Row DocumentSchema Yes No

Shard


Partitioning Sharding

Partition_1Partition_2

Shard_1 Shard_2

DR with Sharding


Shard_2Shard_1Shard_1

Shard_3

Shard_3Shard_2

A B C

RDBMS BigDataRelational Yes No Basic unit Row DocumentSchema Yes No

Scaling Up Out Normalization Yes NoReferential Integrity Yes NoTransactions/Durability Yes NoRow Locks Yes NoJoins Yes No

$$

(NoSQL)



ExamplesProgress mongoDB

Create Customer.Assign name = "Lift Line Skiing" country = “USA” address = “276 North Street” city = ” Boston” state = ” MA” postal-Code = 02114.

db.customer.insert({name: "Lift Line Skiing", country: “USA”, address: “276 North Street” City:” Boston”State:” MA”Postal-Code:02114} );


ExamplesProgress mongoDB

For each Customer no-lock:Display Cust-Num name country address city state Postal-Code.

db.customer.find();

1 Lift Line Skiing USA276 North StreetBostonMA02114

{ _id: ObjectID("509a8fb2f3f4948bd2f983a0"), name: "Lift Line Skiing", country: “USA”, address: “276 North Street” City: ”Boston”State: ”MA”Postal-Code: 02114}

Find first customer no-lock. db.customer.findOne();


Update db.customer.update({Cust-Num: 1}, {new-document});

Deletedb.customer.remove({Cust-Num: 1});

Delete everythingdb.customer.remove();

Examples


db.customer.find({Cust-Num: 1});{“cust-num”: 1, “name” : “Dmitri”}

Update db.customer.update({Cust-Num: 1}, {‘$Set’ {‘Age’: 50}});{“cust-num”: 1, “name” : “Dmitri”, “age”: 50}

Examples


CRUD

More Examples

http://try.mongodb.org

CreateReadUpdateDelete

Where it is used


HealthcareRetailSecurity (NSA)Emergency Management (FAA)Social Media



NewSQLRDBMS NoSQL???Oracle MongoDB


NewSQL

General Purpose

In-memory Databases

MongoDB


That is all about NO SQL

Progress, Oracle and SQL Server

Instance


Shared memoryAPW

BIW

AIW

WDOG

Background Processes

Sports.db

Sports.bi

Sports.lg


Progress v10 Oracle11g MS SQL Server

Instance Instance Instance

DatabaseDatabase Database

Database

Schema

Schema

SchemaSchema

Schema

Schema

Schema

27


System Databases

User Databases


Progress v11 Oracle12c MS SQL Server

Instance Container DB Instance

DatabaseDatabase

Database

Schema

Schema

Schema

Plugable DB

Plugable DB

Schema

Schema

Schema

Schema

Tenant

Tenant

Tenant

Backup

29

Physical Structure (data files)


Progress Oracle MS SQL Server

Database File sports.db Control File(s) control01.ctl N/A, master db

Structure file sports.st N/A N/A

System Area sports.d1 System Tablespace

system.dbf Primary datafile .mdf

D Files area.d Datafiles tablespace.dbf Secondary datafile .ndf

Before-Image File + After-Image File

sports.b1 + sports.a1

Undo tablespace +Redo Log

undotbs.dbf + redo01.log

Log files.ldf

Copy of After-Image File

sports.date_time_etc

Archive Log sports.date_time_etc

Log File sports.lg Alert Log alert_sports.log

Parameter File sports.pf pfile (or spfile) initsports.ora

30


SQL> ALTER DATABASE BACKUP CONTROLFILE TO TRACE;

Progress = _DbStatus-LastTran (Last Transaction Number)

or promon 7


SQL Server = LCN (Log Change Number)

Oracle = SCN (System Change Number)


Last Transaction Number


rfutil db-name -C roll forward [ endtime yyyy:mm:dd:hh:mm:ss | endtrans transaction-number ]

SQL> RECOVER DATABASE UNTIL TIME sysdate-1;SQL> RECOVER DATABASE UNTIL SCN 1234567;

34

Progress _DbStatus-LastTran

Oracle SCN

SQL Server LCN

RESTORE LOG db_name FROM DISK = 'C:\file.TRN' WITH RECOVERY, STOPATMARK = 'lsn:lsn_number'

RESTORE LOG db_name FROM DISK = 'C:\file.TRN' WITH RECOVERY, STOPAT = 'Mar 23, 2009 05:31:00 PM'













Log files.ldf




Log File sports.lg Alert Log alert_sports.log Error Log (ERRORLOG)

Parameter File sports.pf pfile (or spfile) initsports.ora Registry

35


.mdf, .ldf

.mdf, .ndf, .ldf













Log files.ldf






37



Instance Container DB Instance

Database

Database

Before-imageAfter-image

Redo LogUndo Tablespace Transaction Log

Transaction Log

38



With After-images

W/O After-images

ARCHIVELOG Mode

NOARCHIVELOG Mode

39

FULL Recovery Model

SIMPLE Recovery Model

Bulk-logged Recovery Model

PROD

DEV













Log files.ldf






40



OEM parameters

42




Area Tablespace Filegroup

System Area System Tablespace Primary Filegroup

Object Segment N/A

Cluster 8,64,512

Extentsystem managed

Extent8 pages

Block1,2,4 or 8K

Block2,4,8,16,32 or 64K

Page8K

Logical Structure

44

Background Processes/Threads


Progress Oracle description SQL Server threads

APW DBWn Database Writer Lazywriter

BIW, AIW LGWR Log Writer Log Writer

AIMGT ARCn Archive Process N/A

WDOG PMON Process Monitor SQLOS

SMON System Monitor Database CleanupDatabase Shrinking

CKPT Checkpoint process Database Checkpoint

Many other …. 80 at least

45


Progress: ps –ef | grep db_name| grep _mprshut

Oracle: ps –ef | grep ora


Oracle: select * from v$bgprocess;


SQL Server: select * from sys.dm_os_workers;

Meta-schema and VST


Progress Table Oracle View SQL Server

_Area DBA_TABLESPACES sys.filegroups

_AreaExtent DBA_DATA_FILES sys.master_files

_StorageObject DBA_OBJECTS sys.objects

_File DBA_TABLES sys.tables

_Field DBA_TAB_COLUMNS sys.columns

_Index DBA_INDEXES sys.indexes

_Index-field DBA_IND_COLUMNS sys.index_columns

_Sequence DBA_SEQUENCES sys.sequences

Progress Meta-schema tables -80 < _file-number < 0

Progress VST _file-number < -16384

50

VST/ Dynamic Performance/Management Views


Progress Oracle SQL Server

Virtual System Tables Dynamic Performance ViewsOr V$ ( Vee dollar) Views

Dynamic Management Views

_Connect v$session, v$process sys.dm_exec_sessions, sys.dm_exec_connections

_DbStatus v$instance, v$sysstat sys.dm_os_performance_counters

_Trans v$transaction sys.dm_tran_active_transactions

_Lock v$lock sys.dm_tran_locks

_Latch v$latch sys.dm_os_latch_stats

_AreaStatus v$tablespace sys.data_spaces

_Filelist v$datafile sys.dm_db_file_space_usage

_Startup (some) v$parameter sys.dm_server_registry

51



52

Supplied Test Databases

Sports2000 HR AdventureWorks2012


Partitioning TypesProgress Oracle MS SQL Server

List List List

Range Range Range

Hash

Sub-partitioning Composite

Composite


Data Protection

BackupReplicationClusteringFlashback

Backup


Progress and SQL Server – simple Oracle RMAN – More complicatedprobkup sports file_nameBACKUP DATABASE AdventureWorks TO DISK file_name WITH FORMAT;

All databases : Online and Offline (Hot and Cold)

All databases: Complete and Incremental/Differential

Oracle and SQL Server could be Full and Partial(some tablespaces or filegroups)

SQL ServerBACKUP LOG AdventureWorks TO DISK

Log backup

55

Compressed Backup


900 GB Progress DB makes about 750 GB probkup

1.4 TB Oracle DB makes either 1.2 TB regular backup or 250 GB RMAN compressed backup (BZIP2 is default).

Compressed backup could be done in SQL Server.

56

Replication


OE Replication SynchronousAsynchronous

Source Target

Oracle 12cFar Sync DB

Server 1 Memory

Apw, Biw, Aiw, Wdog

Server 2 Server 1 Memory

DBWn, LGWR ARCn, SMON

Server 2 Memory


Active - Passive Active – Active

DB58PUG Challenge Americas 2014

Progress Oracle RAC


Apw, Biw, Aiw, Wdog




Cluster db 2Progress Oracle RAC

Flashback

PUG Challenge Americas 065 Progress and Oracle 60

Flashback Database

Flashback Table

Flashback Drop

Flashback Query

Flashback Version Query

Flashback Transaction Query


Oracle Flashback DB:SQL> flashback database to timestamp sysdate-1;SQL> flashback database to SCN 1234567;

Oracle Flashback Query:SQL> select * from table_name as of timestamp systimestamp - interval ‘30' minute;

61

SQL Server -- Create database snapshot

CREATE DATABASE AdventureWorks_dbss1ON ( NAME = AdventureWorks_Data, FILENAME = 'C:\AdventureWorks_data_1.ss' ) AS SNAPSHOT OF AdventureWorks; GO

Flashback Database


Connecting to a database

62

Progress Client Server -db db_name -H host -S serviceShared memory -db /full_path/db-name

Oracle Listener

MS SQL Server Login, Password

Windows credentials




-H host –S service1

-H host –S service2 S

Host:port:SIDSID

ListenerPort 1521

SID2

Connecting to a database

Host:port:SID2

65

Shutdown

• Unconditional Shutdown• Emergency Shutdown

• Shutdown Normal• Shutdown Transactional• Shutdown Immediate• Shutdown Abort


Startup


Closed ClosedNomount modeMount mode

Open Open

ProrestRestoreinit*.ora

.ctl

Shared MemoryBackground Proc

67


Questions

Web page www.proora.comEmail [email protected]

68