December 3, 2014 Sam Siewert

CS317 - File and Database Systems

Lecture – Final Review of Concepts

http://www.thedailybeast.com/articles/2013/03/22/less-is-moo-the-genius-of-gary-larson.html

FIRST HALFWeek 1-6, Week 7 Mid-Term

Sam Siewert 2

Week 1-6 – As Describe in SyllabusNotes on File Systems Presented Week-1

– Meta-data, Namespace, I-nodes

Connolly-Begg Chapters 1 to 13– Part-I - DBMS Background

Chapters 1, 2, 3 – Background on DBMS, History P. 26Chapter 3 – Architecture (ANSI SPARC, P. 36-37)

– Part-II - Relational Model, E.F. Codd, SQL DML/DDLChapters 4, 5 – Relational Model, Algebra and Calculus

– Relational Algebra (P. 132)– Tuple Relational Calculus (P. 133 to 136)– NO DOMAIN RELATIONAL CALCULUS

Chapters 6, 7, 8 – Core SQL, SQL/PSMChapter 9 – SQL:2011 ORDBMS Extensions

– OOA, OOD, OOP and OODBMS Alternative (Persistence of Instantiated Objects from Classes)

– OODBMS Discussed in concept, but deferred until Chapter 27

– Part-III - Database Analysis and DesignChapter 10 – DBMS Development LifecycleChapter 11 – DreamHome Case Study (Assignment #3), P. 112Chapter 12 – Basic ER Models (Chen’s Diagrams)Chapter 13 – Enhanced ER Models

Sam Siewert 3

Study SuggestionsReview Mid-Term

Scan Book Chapters

Read Through Notes and Study any Unfamiliar Concepts

Sam Siewert 4

NORMALIZATIONWeek 8

Sam Siewert 5

Normalization – Week 8Most Starting Points for RDBMS Design include UNF Data (e.g. Spreadsheets) – Duplication in Columns!

We studied Process to go from UNF to 3NF

We studied Residual Issues with 3NF and BCNF as Potential Solution

Goal in UNF->3NF->BCNF Normalization is to Eliminate Insert, Delete and Modify Hazards while Preserving Loss-less Join and Dependencies

Tends to Produce More Tables, Simpler, with FKs!

Sam Siewert 6

7

The Process of Normalization

Ignore Higher Forms

8

The Process of Normalization

Preview of UNF -> 3NFMinimizes Update Anomalies [Insert, Update, Delete], Page 420 to 426 – One Client Renting Multiple Properties – Typical of Spreadsheet

Sam Siewert 9

cNo cName pNo pAddr rentStart rentFinish Rent oNo oName

CR76 John Kay PG4,

PG16

6 Lawrence Street,5 Novar Drive

7/12,

9/13

8/31,

9/14

350,

50

CO40,

CO93

Tina Murphy,

Tony Shaw

…

Client

PropertyOwner

PropertyForRent OwnerRental

CONCEPTUAL / LOGICAL DBMS DESIGN

Week 9

Sam Siewert 10

Conceptual DBMS Design - Week 9MySQL Workbench CASE Tool

ER/EER Use to Specify an RDBMS Schema

Forward and Reverse Engineering of DB Schemas

Goals and Concepts for Conceptual and Logical Design from Chapter 16 & 17

General Familiarity with MySQL Workbench Major Features and Methods of Use

Sam Siewert 11

PHYSICAL DBMS DESIGNWeek 10-11

Sam Siewert 12

DBMS Logical and Physical Design –Week 10/11

Chapter 16 & 17 – Conceptual DBMS Design Methods Mapping onto Physical

Physical Design– Indexing – B-Tree, B+ Tree, Hash Tables– Storage Engines – MyISAM, InnoDB (File or Partition),

MEMORY– RAID – RAID 1, RAID 0, RAID 5; RAID 50, RAID 10 (Ignore

other RAID levels)

Differences between B-Tree and B+ Tree

Value of R-Tree Sam Siewert 13

DBMS SECURITYWeek 11-12

Sam Siewert 14

DBMS Security - Week 11/12Authorization AuthenticationOne-Way HashSymmetric EncryptionPublic Key EncryptionSigning (E.g. Driver Signing)Networked DBMS Client to DBMS Server

Threats – What are they, how Significant to DBMS

Basic Encryption Concepts and Methods

RAID and RTO/RPO Disaster Recovery Concepts Sam Siewert 15

DBMS TRANSACTIONSWeek 12

Sam Siewert 16

DBMS Transactions – Week 12Recall That Many DBMS Clients can Issue Many Transaction Requests– Thread Per Connection– Thread Created for Each Transaction Request– ACID Goals (Atomicity, Consistency, Isolation, Durability)

Two Solutions in Use Today – Strict 2PL (Locks with Automatic Scoping), Time-stamp Request Queue– Similar Issues in OS, but Not Exactly the Same [DBMS - Lost

Update, Uncommitted Dependency, Inconsistent Analysis]– Similar Solutions in OS, but Not Exactly the Same (E.g.

Semaphores and Locks)

Alternatives for Scheduling Reads/Writes encapsulated in Transactions are Challenging (NP-Hard) and/or Insufficient– Conflict Serialization – Insufficient to Solve all 3 Problems– View Serialization – NP Hard Algorithm, Not Practical

Sam Siewert 17

Comparison of Methods

Pearson Education © 2014 18

Orders conflicting operationsAs “some” serial schedule

Including those with data corruptionAnd inconsistent DB states

Less Restrictive Operation OrderingAs “some” serial schedule[Impractical – NP-Complete]

DBMS Transaction SummaryConcurrency due to Multiple Clients and Transactions Mapped to OS Threads [Readers, Writers]

Causes Problems with Split Transactions– Lost Update– Dirty Read or Uncommitted Dependency– Fuzzy Read, Phantom Read

Serialization Using Locks – Not Sufficient for Consistency if Lock Scope Too Small and Can Cause Deadlock!Recoverability – Atomicity Requirement2PL – Growing and Shrinking Phase – Helps with Lock Scope, but Still have Deadlock/LivelockCascading Roll-back and Rigorous/Strict 2PLDeadlock Problem – Prevention, Detect & BreakAlternative Timestamp Methods

Rigorous/Strict 2PL with Deadlock Prevention or Detect & Break or Timestamp is Best

Sam Siewert 19

DISTRIBUTED DBMSWeek 13

Sam Siewert 20

Distributed DBMS – Week 13Why? – Scaling and Disaster Recovery

DR– Datacenters in 2 Geo Locations – Natural Disaster, Terrorist

Attack, WAN outages, Power grid failure, etc.– RTO / RPO – Recovery Time and Point Objectives– Asynchonrous Mirrors, Active / Passive Fail-Over and Fail-Back

Cluster Scaling – Shared Nothing– Load Balance Large Number of Clients and Transactions– Scaling Beyond Single Server– Synchronous Low-latency / High-bandwidth Network (e.g. 10GE,

40G Infiniband, 100G Converged Enhanced Ethernet)

Sam Siewert 21

MySQL ReplicationUsed for DR [Disaster Recovery] and for Geo Content and Services Distribution– East Coast / West Coast – ERAU Daytona, ERAU Prescott– ACTIVE-ACTIVE for Reads, Writes to MASTER

http://dev.mysql.com/doc/refman/5.0/en/replication.htmlAsynchronous Compared to Clustering [Synchronous]One Server is Master - MySQL Replication Solutions

Sam Siewert 22

MySQL Replication - DRFail-Over Master in Case of DisasterBEFORE MASTER FAILURE

Sam Siewert 23

MySQL Replication – DR Fail-OverFail-Over Master in Case of DisasterAFTER MASTER FAILUREChallenges – Fail-Back [Manual] and Split-Brain

Sam Siewert 24

To Restore MASTER We Must:

1) Repair Issue [E.g. HW fix]2) Re-Sync Writes3) Fail-Back to Original MASTER

MySQL ClusterMySQL Shared Nothing Cluster – NDBD, Network DB DaemonSynchronous – So Ideally Gigabit, 10GE, 40G IB or Better ClusterSimilar to PNFS Concept – Parallel Network File System (NDB MgtBottleneck?)

Sam Siewert 25

DBMS CONNECTORSWeek 13-14

Sam Siewert 26

RDBMS Client Connector - Week 13/14

Data Processing and User Interface Client Applications Need to Connect to DBMS Server

Improved Features Relative to Direct SQL Command Line for Users

Language Specific Connectors– ODBC – Open Database Connection, Middleware API Standard,

Typically Implemented in C Language– JDBC – Java Database Connector (ODBC Bridge)– MySQL C++ API Connector– MySQL C API and Driver for C Connector

Study C Examples – conntest.c

Sam Siewert 27

OODBMS AND UMLWeek 14

Sam Siewert 28

OODBMS and UML – Week 14OODBMS, Object Stores, and NoSQL Alternatives to RDBMS

Basic Definitions of Class Hierarchy, Inheritance, Encapsulation, Abstract Class, Concrete or Refined Class, Instantiation of Object, Dynamic Binding, Parametric and Ad-hoc Polymorphism

OOP – Programming Languages for Most Direction Implementation of OOA/OOD

PPL – Persistent Programming Language – One OODBMS Approach to Integrate OOP with DBMS

Type Gap in RDBMS and Type Mapping Between Store and Memory in OODBMS and Object Stores

Sam Siewert 29

Recall Object Definition – From Week 4Instance of a Class [Hierarchy from Abstract that Can’t Be instantiated to Concrete Sub-classes]Classes Define Public and Private Data and Methods to Operate on that DataSub-classes Inherit from Parent Classes and Refine Data Abstraction and MethodsOOPS – Java, C++, …, back to Smalltalk and Lisp CLOSEncapsulation and Abstraction is the GoalImplementation Hiding and Interface DefintionEach OOP Has Variations on Support [E.g. Multiple Inheritance, Abstract Classes and Methods, Polymorphism [Parametric, Ad-hoc, Operator Overloading]E.g. Oracle’s Java Object Tutorial

Sam Siewert 30

Origins of the Object-Oriented Data Model

Pearson Education © 2014 31

Two-Level Storage Model for RDBMS

Pearson Education © 2014 32

Single-Level Storage Model for OODBMS

Pearson Education © 2014 33

OODBMS Type MappingPersistent OOPL Approach– Need to Store Instantiated Objects with Persistence Attributes– Store Attribute Data and Methods (Code)

Pointer Swizzling – OIDs (Index into Look-Up Table) Provides Mapping between Methods (Function Addresses) and Data (Data Addresses)– Similar to General Descriptor Concepts – E.g. a File pointer and a File

descriptor (index that is simple integer)– Involves OID Look-Up Indirection

No Swizzle – VM Paging of Objects from Storage to Memory– VM Basic Concept – Logical Memory Space Larger than Physical, Disk

Backs Up Pages in Virtual Memory Space not Loaded in RAM– VM Addressing is Logical Memory Address translated to Physical Address

with Page Fault to Handle Unloaded Pages– OODBMS can Page Objects from Disk to Memory– VM Pages are Not Normally Durable, but Objects Paged Must be!

Sam Siewert 34

Accessing an Object with an OODBMS

Pearson Education © 2014 35

Client-Server Architecture

Pearson Education © 2014 36

SummaryIgnore FDM [Functional Data Model] and Paging Details in Chapter 27

OODBMS is a Work in Progress, but Promising

NoSQL, Object Stores, Innovation in File Systems

RDBMS Remains Ideal for Transaction Workloads

OODBMS, Object Stores, NoSQL – Growing Alternative for Newer Workloads – Object Oriented Data (BLOBs) and Big Data Analytics

Sam Siewert 37

UML – Week 14Unified Set of OO Diagraming Methods for OOA/OODBasic Concepts on Indicated Diagram Types

Sam Siewert 38

UML – Design GoalsProvide ready-to-use, expressive visual modelinglanguage …

Provide extensibility and specialization …

Be independent of particular programming languagesand development processes.Provide a formal basis …

Encourage growth of object-oriented tools market.Support higher-level development concepts such ascollaborations, frameworks, patterns, and components.

Integrate best practices.

Pearson Education © 2014 39

UML - DiagramsStructural:– Class diagrams (similar to EER with Hierarchy,

Attributes, and Class Name, Methods Added)– object diagrams (similar to ER)– component diagrams– deployment diagrams.

Behavioral:– use case diagrams– sequence diagrams– collaboration diagrams– statechart diagrams– activity diagrams.

Pearson Education © 2014 40

UML Class DiagramVery Similar to EER (Almost Identical, Just Added Detail and Extensions) – Lecture-Week-6-2– Encapsulation of Attributes and Methods to Operate on Them– Recall RDBMS has SQL/PSM and ORDBMS Has Extensions for

Methods– EER However Has Class Hierarch and Attributes, but No

Methods Specified

Sam Siewert 41

http://www.agilemodeling.com/artifacts/classDiagram.htmLecture-Week-6-2, Slide 11 to 16

UML – Object DiagramsModel instances of classes and used to describe system at a particular point in time.

Can be used to validate class diagram with “real world” data and record test cases.

Similar to ER (Lecture Week-6-1)

Pearson Education © 2014 42

UML – Deployment DiagramsDepict configuration of run-time system, showinghardware nodes, components that run on thesenodes, and connections between nodes.

Pearson Education © 2014 43

UML – Use Case Diagrams

Pearson Education © 2014 44

UML – Use Case Diagrams

Pearson Education © 2014 45

UML – Statechart DiagramsShow how objects can change in response toexternal events.Usually model transitions of a specific object.

Pearson Education © 2014 46