Choosing Database Technology

Jens ColdeweyColdewey Consulting

Uhdestraße 12D-81477 München

Germanyjens_coldewey@acm.orghttp://www.coldewey.com

Choosing a DatabaseTechnology

Tutorial at Object World 98Frankfurt, October, 1st1998

Roadmap through the next hours

4 Introduction

4 The Problem: StoringObjects

4 Requirements of theStakeholders

4 The important forces indepth

4 Turning forces into adecision

What are we talking about?

4 You...

– develop medium tolarge size systems

– use object technology

– need to store yourobjects

– have to decide for adatabase technology

4 It doesn't matter...

– what domain you arein

– whether you work for aproduct or a project

– what language youuse

! I'm not talking about specific products !

Deciding for a database you have totake care of the different stakeholders

EndUsers

Opera-tions

ProjectManage-

ClientManage-

TeamUsers Project

You have to find the best balance

4Every group hasdifferent objectives

4These "forces" oftencontradict each other

4A good decision is apragmatic balance ofthese forces

This tutorial analyzesthe forces. It's yourterm to decide.

Before we talk about solutions we haveto identify the problem: Storing objects

4 Introduction

è The Problem:Storing Objects

4 Turning forces intodecisions Database

Storing objects you want OO features tobe preserved...

4Complex Objects

4Object Identity

4Encapsulation

4Classes

4Class Hierarchy

4Polymorphism

4Extensibility

...while new requirements emerge

4Persistence

4Query Facility

4Concurrency

4Recovery

4Security

4Secondary Storage Management

(Modified from [Atk+89])

What are the options?

Database

ObjectDatabase

Object/RelationalCoupling

StreamPersistence

ObjectServer

PageServer

ORDBMS AccessLayer

Object databases directly store objectsin the database

4 Tight languageintegration

– Access

– Typing

– References

4 ODMG defines"standard" interface

4 Products: ObjectStore,Versant, Objectivity/DB,Poet, GemStone,...

4 Literature: [Cat94]

Relational databases need a complexmapping mechanism

4 Relational Paradigm

– Tables and table-operations

– References via foreignkeys

– No extended typing

– No inheritance

4 SQL - Standardized

4 Products: TOPLink,Persistence

4 Literature: [Kel98]

O/R Access Layer

Object / Relational databases try tocombine both worlds

4 Extends RelationalParadigm

– Complex Objects

– Typing

– Nested Tables

4 E-SQL (vendor specific)

4 No standards by now

4 Products: DB/2,Informix, Oracle 8,Unidata

4 Literature: [StM96]

To start with the analysis of the forceswe turn back to our stakeholders

4 Introduction

è Requirements ofthe Stakeholders

4 Turning forces intodecisions

EndUsers

Opera-tions

ProjectManage-

ClientManage-

TeamUsers Project

Client's management usually has fivethings in mind

Opera-ting Cost

Invest-ment

Protec-tion

Develop-mentCost

Time toMarket

Func-tionality

Client'sManage-

The end user's requirements are usually"obvious"

Func-tionality

Relia-bility

Perfor-mance

EndUser

Operations' requirements are often lessobvious - and they will haunt you!

Main-tain-

ability

Re-covery

AccessControl

Relia-bility

AdminEffort

Re-sources

Oper-ations

Some forces derive from your ownproject management

VendorSupport

Invest-ment

Protec-tion

Devel-opmentEffort

TrainingLicenseCost

ProjectManage-

And finally the development team needsto get optimal support for their work

VendorSupport

Stability

DBFeatures

ToolSupport

Archi-tecture

Functionality

Performance

Reliability

AccessControl

Resources

Maintain-ability

ArchitecturalConfor-mance

DB Features

Stability

ToolSupport

Training

OperatingCost

Time toMarket

Admin Effort

License Cost

Develop-ment Effort

DevelopmentInvestmentProtection

Client'sInvestmentProtection

VendorSupport

In the following discussion I concentrateon discriminating issues

4Some of these issues depend more on theproduct than on the technology

4The following concentrates on whattechnology heavily influences:

– Architectural Conformance

– Functionality and DB Features

– Performance

– Maintainability

4The rest is just sketched

4 Introduction

è The importantforces in depth (I)

ArchitecturalConformance

Functionalityand Database

Features

Performance

Maintainabilityand

DevelopmentSupport

The Rest

The Architectural Conformance splitsinto three areas of interest

Architec-tural Confor-

Data Architecture

methodology D

Many large systems consist of severalapplications on the same database

Contracts Liability CommisionRepor-

tingStatistics

Enterprise Database

This is the architecture most (relational)database experts have in mind

+ Well-known architecture

+ Easy integration withlegacy applications

+ Easy Client/Serverdesign

+ Many useful tools

− Database designchanges propagatemerciless ⇒Application specificviews needed

− Elephantiasis of thedatabase

The object view is different

Functionality

Func-tionalty

DataFunc-

tionalty

Func-tionalty

The object approach avoids someproblems but generates new ones

+ Good encapsulation ofdata

+ Defined responsibilitieskeep databasesmanageable

+ Missing centraldatabase model speedsup development

− Additionalcommunication is newand complex

− Analyzing information ofseveral applications ishard

− Tool market is still in itsinfancy

− Paradigm shift is hard

Both DB technologies support botharchitectures but have different focus

+ Feasible- Schema evolution

may become anightmare

+ ProposedArchitecture

+ Schema evolutionsupported well

+ ProposedArchitecture

+ Good tool support

+ Feasible- "Brushing against

the tools"

Object DB (O)RDB

DataArchitecture

methodology D

Storing objects you want to preserveyour methodology

4Language integration– A single language (and type system) should apply, no

matter whether information is transient or persistent

4 Information Hiding– Only the object itself knows what data it is storing

4Separation of concerns– Every part of the system should have its well-defined,

exclusive responsibility

Many successful system have been built withoutsticking to these rules dogmatically

The paradigm gap is quite obvious here

OODB ORDB RDB

+ Very well- Vendor

specificLan

gr. - Extended

SQL- Explizit

transfer

+ FullLanguageFeatures

- No intrinsicencapsu-lationIn

g o Depends onUsage

+ Full ObjectPower

o Depends onUsage

- Data centricapproach

- SQL- Explizit

transfer

DataArchitecture

methodology D

All solutions support single serverarchitectures with specific strengths

DatabaseEngine

QueryEngine

ModelModelModel

DatabaseProxy

DatabaseClient DB Client

Access Layer

PageServer

ObjectServer

ORDB RDB

Requests P

Requests O

For multi-server architectures you needextra services

LockServer

Server1

DB 1Server 2

Server 3

DB3Trans-action

Service

Server1

DB 1Server 2

Server 3

Page Server TM / CORBA

The product is more important than thetechnology

OODB ORDB and RDB

+ Good supportS

+ Page Server:homogenous ser-vers supported well

- Others: Coming up

+ Established trans-action processorsand servers

+ Standards for distri-buted transactionsM

+ Good support

Architectural considerations don'tsuppose a single solution

Database Orientation

Language Intergration

Information Hiding

Separation ofConcerns

Object Orientation

Single Server Support

Multiple HomogenousServers

MultipleHeterogenous

Servers

Database Orientation

Language Intergration

Information Hiding

Separation ofConcerns

Object Orientation

Single Server Support

Multiple HomogenousServers

MultipleHeterogenous

Servers

ORDBMS

4 Introduction

è The importantforces in depth (II)

4 Summary

Features

Performance

Maintainabilityand

DevelopmentSupport

The Rest

It is important not to miss any of theaspects that comprise functionality

Functio-nality & DBFeatures

Datastorage

There are a lot of details to beconsidered

Structure

4 Objectgranularity

4 Inheritance

4 Number ofClasses

4 Collections

4 Versioningand History

Data Storage

4 TransactionModel

4 Concurrencyand Locking

4 Scalability

Dynamics

4 AccessCharacteristics

4 QueryComplexity andFlexibility

4 Mass Updates

Object granularity is an important factor

4 Analysis and designtechniques influenceobject size

– Large objects resultfrom a data-focussedanalysis

– Small fine-grainedobjects result from ause-case drivenapproach

4 High reusability oftenresults in fine-grainedobjects

ODBMS (Page) ODBMS (Object)

O/RDBMS RDBMS

Inheritance support is one of the crucialdifferences

4 Inheritance supportimplies:

– Subtypes

– Polymorphic queries

– Complete tree support

– Polymorphicaggregations

– Fast addition of newsubclasses

– (Multiple Inheritance)

Subtypes

PolymorphicQueries

Full Tree

PolymorphicAggregations

Addition ofSubclasses

MultipleInheritance

ODBMS ORDBMSRDBMS

The number of classes prevent a purelyrelational solution

4 A fine-grained designusually results in manyclasses

4 The more classes youhave the more you relyon polymorphism

4 A direct O/R approachmaps classes totables...

ODBMS O/RDBMS RDBMS

Support for templates and collections isless natural than you may guess

4Compliance to class library

4Different database representations

– Indexed Collections → Direct Storage (Bi-directional cursors!)

– Dictionaries →Indices

– Sets →Special (ODB) or UNIQUE (RDB)

4ODBs offer additional bi-directionalAssociations

Important differences between products!

Support for versions and history aresometimes helpful

4 Implementingversioning with arelational database iswell-understood butneeds resources

4 Many object databasessupport one-dimensional multi-thread versions

4 More of a goodie than acrucial feature

Concerning structure ODBMS fit best

Fine-grainedObjects

Inheritance

Many ClassesCollections

Versioning

ODBMS ORDBMS RDBMS

Datastorage

e Dynam

Database applications show typicalaccess characteristics

OpenTask

OpenProject

Work withObjects

AnotherContract

FindContract

Contractswhere...

Navigation Queries

ODBMS support navigation,(O)RDBMS querying

4 ODBMS

– store objectsregardless of class

– are optimized to followreferences

– provide clustering foradditional navigationsupport

– have weak queryengines

4 (O)RDBMS

– store objectsaccording to class

– use (slow) joins tonavigate

– have excellent queryengines

4 ORDBMS

– can navigateaggregations fast

This is one of the most important forces

Consequently (O)RDBMS offer muchbetter support for queries

4 ODBMS

– Medium support forcomplex objectselections

– Attribute queriesviolate encapsulation

– No ad-hoc queries(Caution: DataWarehouses...)

4 (O)RDBMS

– Excellent support forcomplex queries

– Paradigm supportsfree queries on alldata (everything isglobal!)

– Good support for ad-hoc queries

ORDBMS usually don't support massupdates or stored procedures

Processing

Batch Control

ObjectDatabase

(Object)RelationalDatabase

Requests O

Summary: Navigational applications callfor ODBMS, querying apps for RDBMS

Navigation

Mass-Updates

QueryingAd-hoc Queries

QueryComplexity

ORDBMS RDBMS ODBMS

Datastorage

Today's applications feature twodifferent transaction styles

4 Check-in/Check-outsemantics

4 Long Transactions

– May last for hours oreven weeks

– High probability ofconflicts

– Only few transactionsper second

– Call for nestedtransactions

4 Classic semantics

4 Short transactions

– Last for a fraction ofseconds

– Low probability ofconflicts

– More than 1000transactions persecond

The differences become relevant inextreme areas only

+ Up to 100 Tx/sec+ Up to 1000 users

+ Support for nestedtransactions

+ Timestamps andversioning toresolve conflicts

++ Up to 1000+ Tx/sec++ Several 1000 users

o Feasible but noexplizit support

Object DB (O)RDB

There are two different lockingtechniques in the ODBMS world

4 Object Locking

+ Low risk of conflictsand deadlocks

– High overhead forlocking

4 Page Locking

– Higher risk of conflictsand deadlocks

+ Better performance

4 Unless you haveextreme performancerequirements, the besttradeoff is hard to find

4 Some products offerboth strategies

4 You have to experimentwith your project'sprofile

Locking can cause disasters with naive O/R Mapping

Class A

Class B Class C

Class D Class E Class F

Table A

Table B Table C

Table D Table ETable E

All technologies provide good scalability

4 Both technologies workwith extremly largedatabases

4 Access time for simpleretrieval:

– ODBMS: const

– RDBMS: logm n

Some hard data from the MMIS project

ODBMS tuned

RDBMS0

[Million Records]

ODBMS tuned

ODBMS RDBMS

Source: A. Chaudhri: Object Databases in Practice - Tutorial; OOPSLA'97 Workshop on Experiences using Object Data Management in the Real Worldhttp://www.soi.city.ac.uk/~akmal/oopsla97.dir/workshop.html

Beware of the locking characteristics ofan O/R access layer!

ORDBMS

Long Tx

Short Tx

ScalabilityLocking low load

Locking high load

Long Tx

Short Tx

ScalabilityLocking low load

Locking high load

4 Introduction

è The importantforces in depth (III)

Features

Performance

Maintainabilityand

DevelopmentSupport

The Rest

In general ODBMS have significantlybetter performance

4 Ratio between ODBMSand RDBMS

– Navigational Lookup:10-100 times

– Queries: 0,1-10 times

– Insert: 1-10 times

4 Access layers make iteven worse

4 Advantage decreaseswith transaction load

4 Collection of bench-marks:www.soi.city.ac.uk/~akmal/

4 Don't take thesenumbers for granted:Make your own tests!

Performance depends on several factors

NetworkLoad

Tuning

Locking

AccessCharac-teristics

Tx Over-head

Caching

Perfor-mance

Key to performance: Direct access andclustering instead of foreign keys

x:=Lookupx->y->z.do()

hysicalP

osition

4 The more the appli-cation uses navigationthe higher your gain

4 Significant differencesbetween products

– Page servers showfaster access butworse locking behavior

– Moving an object maychange OID

4 ORDBMS also use OIDs(called REF)

Tuning techniques and support differ

4 ODBMS

– Clustering

– Cache adjustment

– Indices and queryoptimization

4 Mediocre tools forprofiling

4 (O)RDBMS

– Denormalization (!)

– Query optimization

– Indices

– Technical parameters

4 Excellent profilingsupport

A real world comparison...(Object Store for NT 5.0 against JDBC 1.1.3/Oracle 7.3)

Komplex ClassGIF Image

JPEG Image

RDBMS / JDBC0,1

ODBMSRDBMS / JDBC

Source: Strategic Technology Resources: Java Data Management - Comparing ODBMS and RDBMS Implementations of Quantum Objects , White Paper, Chicago, Illinois; 1997; http://www.odi.com/content/white_papers/str-odb-rdb.pdf

...and a more elaborate benchmark

Traversalcold

Traversalwarm

RandomLookup

RandomLookupwarm

Insertcold Insert

OODBMS Small

ODMS large

RDBMS Small

RDBMS large

140OODBMS SmallODMS largeRDBMS SmallRDBMS large

Source: R. Catell: An Engineering Database Benchmark in: J. Gray (ed.): The Benchmark Handbook for Database and Transaction Systems, Morgan-

Kaufman, San Mateo, California. 1991

A good benchmark takes all this intoaccount

4 A benchmark shouldcontain

– Emulation of accesscharacteristics

– Database size

– Simulation of heavymulti-user operation

– Checks onfragmentation

4 Please keep in mind:

– Performance onlyneeds to be "goodenough"

– Benchmarks shouldbe time-boxed

4 Introduction

è The importantforces in depth (IV)

Features

Performance

Maintainabilityand

DevelopmentSupport

The Rest

Maintainability is more thanjust using OO

Skills

SchemaEvolution

MarketPosition

SourceCom-plexity

Maintain-ability

Over the time the object model changes("schema evolution")

4 ODBMS

– Provide support foradding attributes

– Provide APIs toprogram morecomplex changes(lazy update)

4 RDBMS

– Views can handlemost changes viaadministration

The vendor's market position isimportant to estimate long-term support

4ODBMS

– Large vendors exist for a decade now

– Total revenue less than Oracle's expenses onadvertising

– Long-term survival of some vendors unknown

4 (O)RDBMS

– Vendors have established market positions

– Long-term survival near to guaranteed

– O/R Databases have unknown future

Complex sources only annoy developersbut may cause real problems later

4 More sources mean

– More complex confi-guration management

– More points to change

– More opportunities forerrors

4 ODBMS

– Language integrationprovides single-source(with exceptions)

4 (O)RDBMS

– Database definitionand mapping definitionin extra files

– Behavior in OOlanguage and (E)SQL(and storedprocedures)

Which solution fits your maintenanceneeds depends on your requirements

ORDBMS

Schema Evolution

SourceManagement

Market Position

Skills

Schema Evolution

SourceManagement

Market Position

Skills

Some final remarks on developmentsupport

3rd PartyTools

Profiling

TestingTools

DebuggerSupport

VendorSupport

IDESupport

ToolSupport

4 Introduction

è Turning forcesinto decisions

Now that we have identified the forces,it's up to you to find a balance...

4 Identify relevant forces

4 Find your project'scorresponding needs

4 Identify the options thatstill remain

4 Evaluate products

– attend classes

– write prototypes

4 Come up withsuggestions

A minimal set of forces to decidetechnology may look like this

4Basic architecture (Database oriented versusobject oriented)

4Distribution architecture

4Object granularity

4Access characteristics

4Number of classes

4Transaction load

4Market Position

This list of forces clearly showsstrengths and weaknesses of both

ORDBMS

Distribution

DatabaseOrientation

Tx Load

Query support

Market PositionObject Size

Object Server

Number ofClasses

NavigationalAccess

Distribution

DatabaseOrientation

Tx Load

Query support

Market PositionObject Size

Object Server

Number ofClasses

NavigationalAccess

... and it's up to you tofight the political stands

4 Try to see theevaluation with the eyesof the stakeholders

4 Try to foresee theirobjections

4 Try to understandpersonal fears

4 Don't be dogmatic

"People hate changes"T. DeMarco

Thanks for their support

For additional input

4 Akmal Chaudhri,Logica UK Ltd.

4 Uwe Beßle,Versant

4 Rüdiger Eberlein,Oracle

For fruitful discussions

4 Wolfgang Keller,EA Generali

References

[Atk+89] M. Atkinson, F. Bancilhon, D. DeWitt, K. Dittrich, D. Maier, S Zdonik: The Object-Oriented Database System Manifesto in: Deductive and Object-Oriented Databases,Proceedings of the First International Conference on Deductive and Object-OrientedDatabases (DOOD'89), pp. 223-240

[Cat91] R. G.G. Catell: An Engineering Database Benchmark in: J. Gray (ed.): The BenchmarkHandbook for Database and Transaction Systems, Morgan-Kaufman, San Mateo, California.1991

[Cat94] R. G. G. Catell: Object Data Management - Object-Oriented and Extended RelationalDatabase Systems - Revised Edition; Addison-Wesley, Reading, Massachusetts, 1994

[Cha97] A. Chaudhri: Object Databases in Practice - Tutorial; OOPSLA'97 Workshop onExperiences using Object Data Management in the Real Worldhttp://www.soi.city.ac.uk/~akmal/oopsla97.dir/workshop.html

[Kel98] W. Keller: Object/Relational Access Layers - A Roadmap, Missing Links, and More Patternsin: J. Coldewey, P. Dyson (eds.): Proceedings of the 3rd European Conference on PatternLanguages of Programming and Computing, Universitäts Verlag Konstanz, to be publishedhttp://www.coldewey.com/europlop98/

[StM96] M. Stonebraker, D. Moore: Object-Relational DBMSs - The Next Great Wave; Morgan-Kaufman, San Mateo, California, 1996

[STS97] Strategic Technology Resources: Java Data Management - Comparing ODBMS andRDBMS Implementations of Quantum Objects, White Paper, Chicago, Illinois; 1997;http://www.odi.com/content/white_papers/str-odb-rdb.pdf

Choosing Database Technology

Documents

Enhancing Database Querying Skills by Choosing a More ... · Enhancing Database Querying Skills by Choosing a More Appropriate Interface Jozef Hvorecký ... semi-graphical version

Choosing the Appropriate Database Development Tool.doc

AsadaS Choosing Technology Report

Database Technology- ITM

Choosing the right database

Database Technology

PARALLEL DATABASE TECHNOLOGY

Choosing The Right Database For Your Cloud Application

Evolution of Database Technology - Leiden Universityliacs.leidenuniv.nl/~bakkerem2/dbdm/02_dbdm2014_databases.pdf · Database Technology By EM Bakker ... Data collection, database

Database Technology Database Applications

Seven Steps for Choosing the Right Database Cleansing Company - SalesInside Inc

Choosing the Right NoSQL Database Type: Tips and Solutions

Seven Steps for Choosing the Right Database Cleansing Company - Sales Inside

Integrating intelligent database technology and trusted database

Relational Database Technology

Choosing a Talent Management Technology Partner

Сергей Сверчков и Виталий Руденя. Choosing a NoSQL database

Choosing technology

NCL Technology Database

Choosing a new database