Rtdb-Academic Course by S.H.son

7/28/2019 Rtdb-Academic Course by S.H.son

1/189

1

Real-Time Database Systems and

Data Services: Issues and Challenges

Sang H. Son

Department of Computer Science

University of Virginia

Charlottesville, Virginia [email protected]


2/189

2

Outline

Introduction: real-time database systems and real-time data services

Why real-time databases?

Misconceptions about real-time DBS

Paradigm comparison Characteristics of data and transactions in real-time DBS

Origins of time constraints

Temporal consistency and data freshness

Time constraints of transactions

Real-time transaction processing

Priority assignment

Scheduling and concurrency control

Overload management and recovery


3/189

3

Outline (contd)

Advanced real-time applications

Active, object-oriented, main-memory databases

Flexible security paradigm for real-time databases

Embedded databases

Real-world applications and examples

Real-time database projects and research prototypes

BeeHive system

Research issues, trends, and challenges

Exercises


4/189

4

I. Introduction

Outline

Motivation: Why real-time databases and data services?

A brief review: real-time systems Misconceptions about real-time DBS

Comparison of different paradigms:

Real-time systems vs real-time database system

Conventional DBS vs real-time DBS


5/189

5

Some Facts about Real-Time Databases

Fact 1: As the complexity of real-time systems and application isgoing up, the amount of information to be handled by real-timesystems increases, motivating the need for database and dataservice functionality (as opposed to ad hoc techniques and internal

data structures)

Fact 2: Conventional databases do not support timing and temporalrequirements, and their design objectives are not appropriate forreal-time applications

Fact 3: Tasks and transactions have both similarities and distinctdifferences, i.e., traditional task centric view is not plausible to real-time databases.


6/189

7

Something to Remember ...

Real-time FAST

Real-time nonosecsor secs

Real-time means explicit or implicit time constraints

A high-performance database which is simply fast without the

capability of specifying and enforcing time constraints are notappropriate for real-time applications


7/189

8

A Brief Review: Real-Time Systems

A system whose basic specification and design correctnessarguments must include its ability to meet its timeconstraints.

Its correctness depends not only on the logical

correctness, but also on the timeliness of its actions.


8/189

9

Review: Real-Time Systems

Characteristics of real-time systems

timeliness and predictability

typically embedded in a large complex system

dependability (reliability) is crucial

explicit timing constraints (soft, firm, hard)

A large number of applications

aerospace and defense systems, nuclear systems, robotics,process control, agile manufacturing, stock exchange, network

and traffic management, multimedia computing, and medicalsystems

Rapid growth in research and development

workshops, conferences, journals, commercial products

standards (POSIX, RT-Java, RT-COBRA, etc)


9/189

10

Time Constraints

d t

v(t)

v0

d2 t

v(t)

v0

d1

Hard and firm deadline

Soft deadline


10/189

11

Databases for Real-Time Systems

Critical in real-time systems (any computing needs correct data)

real-time computing needs to access data:

real-world applications involve time constrained access to

data that may have temporal property traditional real-time systems manage data in application-

dependent structures

as systems evolve, more complex applications requireefficient access to more data

Function of real-time databases

gathering data from the environment, processing it in thecontext of information acquired in the past, for providingtimely and temporally correct response


11/189

12

What is a Real-Time Database?

A real-time database (RTDB) is a data store whose operationsexecute with predictable response, and with application-acceptable levels oflogical and temporal consistency of data,in addition to timely execution of transactions with the ACIDproperties.

C. D. LockeChief Scientist, TimeSys Co.


12/189

14

What is the gain of using RTDBS?

More efficient way of handling large amounts of data

Specification and enforcement of time constraints

Improved overall timeliness and predictability

Application semantic-based consistency and concurrencycontrol

Specialized overload management and recovery

Exploitation of real-time support from underlying real-timeOS

Reduced development costs


13/189

15

Gain of Using RTDBS (More Specifically)

Presence of a schema - avoid redundant data and its description

Built-in support for efficient data management - indexing, etc

Transaction support - e.g. ACID properties

Data integrity maintenance

But

Not all data in RTDB is durable: need to handle different types of datadifferently (will be discussed further later)

Correctness can be traded for timeliness

- Which is more important? Depends on applications, but timeliness ismore important in many cases

Atomicity can be relaxed: monotonicqueries and transactions

Isolation of transactions may not always be needed

Temporally-correctserializable schedules serializable schedules


14/189

16

Objectives of Real-Time Databases

Correctness requirements:

consistency constraints

time constraints on data and transactions

Objectives timeliness and predictability:

dealing with time constraints and violations

Performance goals:

minimize the penalty resulting from actions either delayed ornot executed in time

maximize the value accruing to the system from actionscompleted in time

support multiple guarantee levels of quality for mixed workloads


15/189

17

Why Not Using Conventional Databases?

Inadequacies of conventional databases:

poor responsiveness and lack of predictability

no facility to support for applications to specify andenforce time constraints

designed to provide good average response time,while possibly yielding unacceptable worst case

execution time resource management and concurrency control in

conventional database systems do not support thetimeliness and predictability


16/189

18

Differences from Traditional Databases

Traditional database systems

persistent data and consistency constraints

efficient access to data

transaction support: ACID properties

correct execution of transactions in the context ofconcurrent execution and failure

designed to provide good average performance

Databases for real-time systems

temporal data, modeling a changing environment response time requirements from external world

applications need temporally coherent view

actively pursue timeliness and predictability


17/189

19

Misconceptions on Real-Time Databases....


18/189

20

Misconceptions about RTDBS (1)

Advances in hardware till take care of RTDBS requirements.

fast (higher throughput) does not guarantee timing constraints

increase in size and complexity of databases and hardware willmake it more difficult to meet timing constraints or to show such

constraints will be met hardware alone cannot ensure that transactions will be scheduled

properly to meet timing constraints or data is temporally valid

transaction that uses obsolete data more quickly is still incorrect

Real-time computing is equivalent to fast computing. minimizing average response time vs satisfying individual timing

constraints

predictability, not speed, is the foremost goal


19/189

21


Advances in standard DBS technology will take care of RTDBrequirements.

while novel techniques for query processing, buffering, andcommit protocols would help, they cannot guarantee timelinessand temporal validity

time-cognizant protocols for concurrency control, commitprocessing and transaction processing are mandatory

There is no need for RTDBS because we can solve all the problemswith current database systems

adding features such as validity intervals and transactiondeadlines to current database systems is in fact moving towardsto developing a real-time database system

such approach (adding features in ad hoc manner) will be lessefficient than developing one from the ground up with such

capabilities


20/189

22


Using a conventional DBS and placing the DB in main memory issufficient.

although main-memory resident database eliminate disk delays,conventional databases have many sources of unpredictability,such as delays due to blocking on locks and transactionscheduling

increases in performance cannot completely make up for the lackof time-cognizant protocols in conventional database systems

A temporal database is a RTDB.

while both of temporal DB and RTDB support time-specific dataoperations, they support different aspects of time

in RTDB, timely execution is of primary concern, while intemporal DB, fairness, resource utilization, and ACID properties

of transactions are more important


21/189

23


Problems in RTDBS will be solved in other areas.

some techniques developed in other areas (e.g., RTS and DBS)cannot be applied directly, due to the differences between tasksand transactions, and differences in correctness requirements

there are unique problems in RTDBS (e.g., maintaining temporalconsistency of data)

RTDBS guarantee is meaningless unless H/W and S/W never fails

true, in part, due to the complexity involved in predictable andtimely execution

it does not justify the designer not to reduce the odds of failure inmeeting critical timing constraints

Reference: Stankovic, Son, and Hansson, Misconceptions About Real-Time Databases, IEEE Computer, June 1999.


22/189

27

Conventional vs. Real-Time Databases:

Correctness Criteria

Conventional Databases: Logical consistency

ACID properties oftransactions:

Atomicity

Isolation

Consistency

Durability

Data integrity constraints

Real-Time Database Systems:

Logical consistency

ACID properties (may berelaxed)

Data integrity constraints

Enforce time constraints

Deadlines of transaction

External consistency

absolute validity interval (AVI)

Temporal consistency

relative validity interval (RVI)


23/189

28

Real-time Systems vs. RTDBS

Real-time systems Task centric

Deadlines attached to tasks

Real-time databases

Data centric

Data has temporal validity, i.e., deadlines also

attached to dataTransactions must be executed by deadline to keep

the data valid, in addition to produce results in atimely manner


24/189

29


25/189

30

II. Characteristics of Data and Transactions

Outline

The origin of time constraints

Types of time constraints

Real-time data and temporal consistency

Real-time transactions


26/189

31

The Origin of Time Constraints

Meeting time constraints is of paramount importance in real-time database systems. Unfortunately, many of these timeconstraints are artifacts.

If a real-time database system attempts to satisfy them all, itmay lead to an over-constrained or over-designed system.

Issues to be discussed:

1. What are the origins of (the semantics of) time constraintsof the data, events, and actions?

2. Can we do better by knowing the origins of timeconstraints?

3. What is the connection between time-constrained events,data, and real-time transactions?


27/189

32

Example #1: Objects on Conveyor Belts

on a Factory Floor

Recognizing and directing objects moving along a set ofconveyer belts on a factory floor.

Objects features captured by a camera to determineits characteristics.

Depending on the observed features, the object isdirected to the appropriate workcell.

System updates its database with information aboutthe object.


28/189

33

Example #1 (contd)

Features of an object must be collected while the objectis still in front of the camera.

Current object and features apply just to the object in

front of the camera

Lose validity once a different object enters the system.

Objects features matched against models in database.

Based on match, object directed to selected workcell.Alternative: discard object and later bring it back again

in front of the camera.


29/189

34

Example #2: Air Traffic Control

System makes decisions concerning

incoming aircrafts flight path

the order in which they should land

separation between landings

Parameters: position, speed, remaining fuel, altitude, typeof aircrafts and current wind velocity.

Aircraft allowed to land => subsequent actions of thisaircraft become critical: cannot violate time constraints

Alternative: Ask aircraft to assume a holding pattern.


30/189

35

Factors that Determine Time Constraints

Focus: externally-imposed temporal properties The characteristics of the physical systems being monitored and

controlled: speed of the aircraft, speed of conveyer belt, temperature and

pressure The stability characteristics as governed by its control laws:

servo control loops of robot hands, fly-by-wire, avionics, fuelinjection rate

Quality of service requirements: sampling rates for audio and video, accuracy requirement for

results

Human (re)action times, human sensory perception: time between warning and reaction to warning

Events, data and actions inherit time constraints from these factors

They determine the semantics (importance, strictness) of timeconstraints.


31/189

36

All Time Constraints are Artifacts?

May be not all of them, but even many externally-imposedconstraints are artifacts:

Length of a runway or speed of an aircraft - determined

by cost and technology considerations;

Quality of service requirements - decided by regulatoryauthorities;

Response times guaranteed by service providers -determined by cost and competitiveness factors


32/189

37

Designer Artifacts

Subsequent decisions of the database system designer introduceadditional constraints:

The type of computing platform used (e.g. centralized vs.distributed)

The type of software design methodology used (e.g., data-centric vs. action-centric)

The (pre-existing) subsystems used in composing the system

The nature of the actions (e.g., monolithic action vs. graph-structured or triggered action)

Time constraints reflect the specific design strategy and thesubsystems chosen as much as the externally imposed timingrequirements


33/189

38

Decisions on Time Constraints

Difficulty of optimal time constraints

Determining all related time constraints in an optimalfashion for non-trivial systems is intractable => divideand conquer (and live with acceptable decisions)

Multi-layer decision process

The decisions made at one level affect those at theother level(s)

While no decision at any level is likely to beunchangeable, cost and time considerations will oftenprevent overhaul of prior decisions


34/189

39

Decisions on Time Constraints (2)

Decisions to be made

Whether an action is periodic, sporadic, or aperiodic

The right values for the periods, deadlines, and offsets

within periods

Importance or criticality values

Flexibility (dynamic adaptability) of time constraints


35/189

40

Time Constraints of Events

Three basic types of time constraints

1. Maximum: delay between two events

Example: Once an object enters the view of the camera, objectrecognition must be completed within t1 seconds

2. Minimum: delay between two events

Example: No two flight landings must occur within t2 seconds

3. Durational: length of an event

Example: The aircraft must experience no turbulence for atleast t3seconds before the seat-belt sign can be switched offonce again

Constraints can specify between stimulus and response events(max, min, and duration between them can be stated)


36/189

41

Time Constraints of Events (2)

The maximum and minimum type of time constraints ofrecurring (stimulus) events: rate-based constraints

Time constraints determine the constraints ontransactions:

Rate-based constraints -> periodicity requirementsfor the corresponding actions

Time constraints relating a stimulus and its response -> deadline constraints

Specifications of minimal separation between responseto a stimulus and the next stimulus -> property of thesporadic activity that deals with that stimulus


37/189

42

Data in Real-Time Database Systems

Data items reflect the state of the environment

Data from sensors - e.g., temperature and pressure

Derived data - e.g., rate of reaction

Input to actuators - e.g., amount of chemicals, coolant

Archival data - e.g., history of (interactions with)environment

Static data as in conventional database systems


38/189

43

Time Constraints on Data

Where do they come from? state of the world as perceived by the controlling

system must be consistent with the actual state

Requirements

timely monitoring of the environment timely processing of sensed information

timely derivation of needed data

Temporal consistency of dataabsolute consistency: freshnessof data between

actual state and its representation

relative consistency: correlationamong dataaccessed by a transaction


39/189

45

Static Data and Real-Time Data

Static data data in a typical database

values not becoming obsolete as time passes

Real-time (Temporal) data

arrive from continuously changing environment represent the state at the time of sensing

has observed timeand validity interval

users of temporal data need to see temporally coherent

views of the data (state of the world) When must the data be temporally consistent?

ideally, at all times

in practice, only when they are used by transactions


40/189

46

An Example

Data object is specified by(value, absolute validity interval, time-stamp)

Interested in {temperatureand pressure}

with relative validity intervalof 5

Let current time = 100

temperature= (347, 10, 95) and pressure= (50, 20, 98)

-- temporally consistent

temperature= (347, 10, 98) and pressure= (50, 20, 91)

-- temporally inconsistent


41/189

47

What Makes the Difference?

We have a set of predicates to be satisfied by data

Why not use standard integrity maintenance techniques?

Not executing a transaction will maintain logical

consistency, but temporal consistency will be violated Satisfy logical consistency by CC techniques, such as 2PL

Satisfy temporal consistency by time-cognizant transactionprocessing

AVI and RVI may change with system dynamics, e.g. modechanges


42/189

48

Time Constraints Associated with Actions

Time constraints dictate the behavior of the environment constrain the rates and times at which inputs arrive at the

system

Example: seek permission to land only when aircraft is 10mins from airport

Time constraints prescribe performance of the system

dictate the responsiveness of the system to these inputs

Example: respond to a landing request within 30 seconds

Time constraints are imposed to maintain data temporalconsistency

Example: actions that update an aircrafts dynamicparameters in 1 second


43/189

49

Distinct Types of Transactions

Write-only transactions (sensor updates): obtain state of theenvironment and write into the database

store sensor data in database (e.g., temperature)

monitoring of environment

ensure absolute temporal consistency

Update transactions (application updates)

derive new data and store in database

based on sensor and other derived data

Read-only transactions

read data, compute, and report (or send to actuators)


44/189

50

Time Constraints on Transactions

Time constraints on transactions

some come from the need to maintain temporal consistencyof data

some come from the requirements on reaction time, dictating

the responsiveness of the system some come from the designers choice, specifying the rates

and times at which inputs arrive at the system

transactions value depends on completion time


45/189

51

Types of Time Constraints

Based on type of time constraints:

Periodic

- Every 10 secs Sample wind velocity

- Every 20 secs Update robot position

Aperiodic

- If temperature > 1000

within 10 secs add coolant to reactor

Based on Value:

Hard: must execute before deadline Firm: abort if not completed by deadline

Soft: diminished value if completed after deadline


46/189

52

Dealing with Time Constraint Violations

Large negative penalty => a safety-critical or hard time constraint

typically arise from external considerations

important to minimize the number of such constraints

No value after the deadline and no penalty accrues => a firm deadline

typically, alternatives exist

Result useful even after deadline => a soft deadline

system must reassign successors parameters - so that the overallend-to-end time constraints are satisfied

Firm and soft time constraints offer the system flexibility - not presentwith hard or safety-critical time constraints

E l f Ti C t i t S ifi d


47/189

53

Examples of Time Constraints Specified

using ECA (Event-Condition-Action) Rules

The time constraints can be specified using ECA rules

ON (10 seconds after initiating landing preparations)

IF (steps not completed)

DO (within 5 seconds abort landing)

ON (deadline of object recognition)

IF (action not completed)

DO (increase importance, adjust deadlines)

ON (n-th time violation within 10 secs)

IF (crisis-mode)

DO (drop all non-essential transactions)


48/189

54

Time Constraints: Discussion

Understand the issues underlying the origin and semantics of timeconstraints

not all deadlines are given.

need ways to deriving time constraints (and semantics) in theleast stringent manner

flexibility afforded by derived deadlines must be exploited

deadline violation must also be handled adaptively

Control strategies can be specified by ECA rules


49/189

55


50/189

56

III. Real-Time Transaction Processing

Outline

Priority assignment

Scheduling paradigms

Priority inversion problem

Concurrency control protocols

Predictability issues

Overload management and recovery


51/189

57

Priority Assignment

Different approaches

EDF: earliest deadline first

highest value (benefit) first

highest (value/computation time) first complex function of deadline, value, slack time

Priority assignment has significant impact on database systemperformance

Assignment based on deadline and value has shown goodperformance


52/189

59

Goals of Real-Time Transaction Scheduling

Maximize the number of transactions (both sensor and user) thatmeet deadlines

Keep data temporally valid

on overload, allow invalid intervals on data (note that data withinvalid interval may not be used during that invalid time)

overload management by trading off quality for timeliness andschedule contingency (or alternative) versions of transactions

more on overload management later ...


53/189

60

Execution Time of Transactions

texec = tdb + tI/O + tint + tappl + tcomm

tdb = processing of DB operations (variable)

tI/O = I/O processing (variable)

tint = transaction interference (variable)

tappl = non-DB application processing (variable & optional)

tcomm = communication time (variable & optional)


54/189

61

Scheduling Paradigms

Scheduling analysis or feasibility checking of real-time computationscan predict whether timing constraints will be met

Several scheduling paradigms emerge, depending on

whether a system performs schedulability analysis

if it does, whether it is done statically or dynamically, and

whether the result of the analysis itself produces a schedule orplan according to which computations are dispatched at run-time


55/189

62

Different Paradigms

1. Static Table-Driven approaches: Perform static schedulability analysis

The resulting schedule is used at run-time to decide when acomputation must begin execution

2. Static Priority Driven Preemptive Approaches:

Perform static schedulability analysis but unlike in the previousapproach, no explicit schedule is constructed

At run-time, computations are executed (typically) highest-priority- first

Example: rate-monotonic priority assignment - priority isassigned proportional to frequency


56/189

63

Different Paradigms (2)

3. Dynamic Planning Based Approaches: Feasibility is checked at run-time, i.e. a dynamically arriving

computation is accepted for execution only if it found feasible (that is,guaranteed to meet its time constraints)

One of the results of the feasibility analysis is a schedule or plan that is

used to decide when a computation can begin execution.4. Dynamic Best-effort Approaches:

No feasibility checking is done

The system tries to do its best to meet deadlines, but since noguarantees are provides, a computation may be aborted during its

execution


57/189

64

Dealing with Hard Deadlines

All transactions have to meet the timing constraints

best-effort is not enough

a kind of guarantee is required

Requires

periodictransactions only

resource requirements known a priori

worst-case execution time of transactions are known

Use static table-driven or priority-driven approach

schedulability analysis is necessary

run-time support also necessary


58/189

65

Dealing with Soft/Firm Deadlines

Two critical functions: assign transaction priorities

resolve inter-transaction conflicts using transaction parameters:deadline, criticality, slack time, etc.

For firm deadlines, abort expired transactions

For soft deadlines, the transaction is continued to finish in general,even if the deadline is missed

Various time-cognizant concurrency controls developed, many ofwhich are extensions of two-phase locking (2PL), timestamp, andoptimistic concurrency control protocols


59/189

66

Time-cognizant Transaction Scheduling

Earliest deadline first (EDF) Highest value first

Highest value density first (value per unit computation time)

Weighted formula: complex function of deadline, value, and remainingwork, etc.

Earliest Data Deadline First:considering the validity interval Example: DD(Y) is used as the virtual deadline of transaction T

Activate

TR T

Begin

TR T

Read X Read Y

Deadline

of TR T

DD(X)DD(Y)


60/189

67

Example 1 : Commit Case

Activate

TR T

Begin

TR T

Read X Read Y

Deadline

of TR T

Commit

X and Y are valid

TR T makes deadline

DD(X)DD(Y)

DD = Data deadline


61/189

68

Example 2 : Abort Case

Activate

TR T

Begin

TR T

Read X Read Y

Deadline

of TR T

DD(Y) DD(X)

ABORT


62/189

69

Example 3 : Forced Wait

Activate

TR T

Begin

TR T

Read X

Read Y

Deadline

of TR T

DD(X)DD(Y)

Force TR T to Wait for Update to Y

since it will occur soon!


63/189

70

Example 4 : With Data Similarity

Activate

TR T

Begin

TR T

Read X

Read Y

15.70

Deadline

of TR T

DD(X)

Commit

Deadline of TR T is met

Data X is OK

Data Y is similar (defined in DB)

DD(Y) - Y updated to 15.78


64/189

71


65/189

72

Transactions: Concurrency Control

Pessimistic

Optimistic (OCC)

Hybrid (e.g., integrated real-time locking)

Speculative

Semantic-based Priority ceiling


66/189

E l f 2PL T t ti


67/189

74

Example of 2PL: Two transactions

T1:write_lock (X);

read_object (X);

X = X + 1;

write_object (X);

unlock (X);

Priority T1 > Priority of T2

T2:read_lock (X);

read_object (X);

write_lock (Y);

unlock (X);

read_object (Y);Y = X + Y;

write_object (Y);

unlock (Y);

E l f 2PL D dl k


68/189

75

Example of 2PL: Deadlock

T1:

read_lock (X);

read_object (X);

write_lock (Y); [blocked]

:

:

=> DEADLOCK !

T2:read_lock (Y);

read_object (Y);

write_lock (X); [blocked]

:

:

C fli t R l ti i 2PL


69/189

76

Conflict Resolution in 2PL

2PL (or any other locking schemes) relies on blocking requestingtransaction if the data is already locked in an incompatible mode. Whatif a high priority transaction needs a lock held by a low prioritytransaction? Possibilities are ...

let the high priority transaction wait

abort the low priority transaction

let low priority transaction inherit the high priority and continueexecution

The first approach will result in a situation called priority inversion Several conflict resolution techniques are available, but the one that use

both deadline and value show better performance

Priority Inversion Problem in Locking


70/189

77

Priority Inversion Problem in Locking

Protocols

What is priority inversion?A low priority transaction forces a higher priority transaction to

wait

highly undesirable in real-time applications

unbounded delay may result due to chained blocking and

intermediate blocking:

Example: T0 is blocked by T3 for accessing data object, then T3is blocked by T2 (priority T0 > T2 > T3)

E l f 2PL P i it I i


71/189

78

Example of 2PL: Priority Inversion

T1:

write_lock (X); [blocked]

read_object (X);X = X + 1;write_object (X);unlock (X);

T2:read_lock (X);

read_object (X);

write_lock (Y);unlock (X);

read_object (Y);

Y = X + Y;write_object (Y);unlock (Y);

time

Priority

inversion

S l ti t P i it I i P bl


72/189

79

Solutions to Priority Inversion Problem

Priority abort

abort the low priority transaction - no blocking at all

quick resolution, but wasted resources

Priority inheritance

execute the blocking transaction (low priority) with the priorityof the blocked transaction (high priority)

intermediate blocking is eliminated

Conditional priority inheritance

based on the estimated length of transaction inherit the priority only if blocking one is close to completion;

abort it, otherwise

C diti l P i it I h it P t l


73/189

80

Conditional Priority Inheritance Protocol

Ti requests data object locked by Tj

if Priority (Ti) < Priority (Tj)

then block Ti

else

if (remaining portion of Tj > threshold)

abort Tj

else

Ti waits while Tj inherit the priority of Ti to execute

Wh C diti l P i it I h it ?


74/189

81

Why Conditional Priority Inheritance?

Potential problems of (blind) priority inheritance:

life-long blocking - a transaction may hold a lock during itsentire execution (e.g., strict 2PL case)

a transaction with low priority may inherit the high priorityearly in its execution and block all the other transactions withpriority higher that its original priority

especially severe if low priority transactions are long

Conditional priority inheritance is a trade-off between priority

inheritance and priority abort Not sensitive to the accuracy of the estimation of the transaction

length


75/189

82

Performance Results

Priority inheritance does reduce blocking times. However, it isinappropriate under strict 2PL due to life-time blocking of the highpriority transaction. It performs even worse than simple waitingwhen data contention is high

Priority abort is sensitive to the level of data contention

Conditional priority inheritance is better than priority abort whendata contention becomes high

Blocking is a more serious problem than resource waste, especiallywhen deadlines are not tight

In general priority abort and conditional priority inheritance arebetter than simple waiting and priority inheritance

Deadlock detection and restart policies appear to have little impact

Optimistic Concurrency Control


76/189

83

Optimistic Concurrency Control

No checking of data conflicts during transaction execution read phase: read values from DB; updates made to local copies

validation phase

backward validation or forward validation

conflict resolution

write phase: if validation ok then local copies are written to the DB

otherwise discard updates and (re)start transaction

Non-blocking

Deadlock free

Several conflict resolution policies

OCC: Validation phase


77/189

84

OCC: Validation phase

If a transaction Ti should be serialized before a transaction Tj, thentwo conditions must be satisfied:

Read/Write rule

Data items to be written by Ti should not have already beenread by Tj

Write/Write rule Tis should not overwrite Tjs writes

OCC Example


78/189

85

OCC Example

T1:

read_object (X);X = X + 1;write_object (X); validation

T3:

read_object (Y);Y = Y + 1;write_object (Y);

...

T2:read_object (X);read_object (Y);

Y = X + Y;write_object (Y); validation

OCC: Conflict Resolution


79/189

86

OCC: Conflict Resolution

When a transaction T is ready to commit, any higher-priorityconflicting transaction is included in the set H

Broadcasting commit (no priority consideration)

T always commits and all conflicting transactions are aborted

With priority consideration: if H is non-empty, 3 choices

sacrifice policy: T is always aborted

wait policy: T waits until transactions in H commits; if they docommit, T is aborted

wait-X policy: T commits unless more than X% of conflictingtransactions belong to H

OCC: Comparison


80/189

87

OCC: Comparison

Broadcasting commit (no priority consideration)

not effective in real-time databases

Sacrifice policy: wasteful

theres no guarantee the a transaction in H will actually

commit; if all in H abort, T is aborted for nothing Wait policy: address the above problem

if commit after waiting, it aborts lower priority transactions afterwaiting, which may have not enough time to restart and commit

the longer T stays, the higher the probability of conflicts

Wait-X policy: compromise between sacrifice and wait

X=O: sacrifice policy; X=100: wait policy

performance study shows X=50 gives the best results

Priority Ceiling Protocol


81/189

88


Why?

to provide blocking at most once property

the system can compute (pre-analyze) the worst case blockingtime of a transaction, and thus schedulability analysis for a setof transaction is feasible

A complete knowledge of data and real-rime transactions necessary:for each data object, all the transactions that might access it needto be known

true in certain applications (hard real-time applications)

not applicable to other general applications



82/189

89


For each data object O:

write-priority ceiling: the priority of the highest prioritytransaction that may write O

absolute priority ceiling: the priority of the highest priority

transaction that may read or write O r/w priority ceiling: dynamically determined priority

which equals absolute priority ceiling if O is write-locked; equalswrite priority ceiling if O is read locked

Ceiling rule: transaction cannot lock a data object unless its priority

is higher that the current highest r/w priority ceiling locked by othertransactions

Inheritance rule: low priority transaction inherits the higher priorityfrom the ones it blocks

Good predictability but high overhead


83/189

90


84/189

91

Overload Management and Recovery ...

M i O l d


85/189

92

Managing Overloads

The result of overload is a slow response for the duration of theoverload

In real-time databases, catastrophic consequences may arise:

hard real-time transactions must be guaranteed to meet deadlines

under overloads

transaction values must be considered when deciding whichtransactions to shed

missing too many low-valued transactions with soft deadlines mayeventually degrade system performance

Dealing with overloads is complex and practical solutions are needed

Quality-Timeliness Tradeoffs during


86/189

93

Overload

Achieve timeliness by trading off

completeness: approximate processing by sampling data andextrapolation

consistency: relax correctness requirements in controlled manner

(e.g., epsilon-serializability, similarity)

currency: process transactions using older versions of data(within the tolerance range of the application)

precision: use algorithms that produce lower precision resultswithin the deadline

Exploit concepts from imprecise computing, monotonic computing,mandatory/optional structures, multi-precision algorithms,primary/contingency transactions, etc.

Scheduling for Overload Management


87/189

94

Scheduling for Overload Management

Background

Dynamic real-time systems are prone to transient overloads

requests for service exceed available resources for a limited timecausing missed deadlines

may occur when faults in the computational environment reducecomputational resource available to the system

may occur when emergencies arise which require computationalresources beyond the capacity of the system

Overloads cause performance degradation

Schedulers are generally not overload-tolerant

Scheduling for Overload Management (2)


88/189

95


Resource management has two components: scheduling andadmission control

Scheduling determines the execution order of admitted transactions,which might not be enough to handle the current overload situation

Admission control determines which transactions should be grantedsystem resources

To resolve transient overloads, the system needs both

when to execute transactions and selecting which transactions tobe executed (original, alternative, or contingency transaction, if

the transaction is accepted)



89/189

96


Goal: dynamic overload management with graceful performancedegradation (meeting all critical deadlines)

Problem: need to handle complex workload

critical and non-critical transactions -- some are sporadic and

others aperiodic (no minimum inter-arrival time informationavailable)

non-critical transactions can be discarded in a controlled mannerwhile critical transactions are replaced by alternative orcontingency transactions (with shorter execution time)

resources are reallocated among transactions that are admitted tothe system using value-functions

Scheduling for Overload: Assumptions


90/189

97

Scheduling for Overload: Assumptions

Transaction and Workload:

Critical transactions are sporadicand have a correspondingcontingency transaction

Non-critical transactions areaperiodic

Each transaction is pre-declaredand pre-analyzed with knownworst case execution time

Critical deadlines must beguaranteed even under overload

conditions

System Characteristics:

Dedicated CPU for schedulingactivities is desirable; otherwise,only simple policies can beimplemented

Scheduling Module


91/189

98

Scheduling Module

Scheduler consists of several components

Pre-analysis of schedulability: critical transactions are pre-analyzed tocheck whether they can be executed properly and how muchreduction in resource requirement can be achieved by using

contingency transactionsAdmission controller determines which transactions will be eligible for

scheduling

Scheduler can schedule according to different metric

deadline-driven

value-driven

Overload Resolver decides the overload resolution actions

Dispatcher patches from the top of the ready queue (highest priority)

S h d li C t


92/189

99

Scheduling Components

Admission

Controller

Transaction

Scheduler

Transactions

Ov erload

Resolver

Dispatcher

Ready queue

........

Rejection queue

........

Overload Resolution Strategies


93/189

100

Overload Resolution Strategies

Admission Controller:

Reject transaction

Admit contingency action

Scheduler: Drop transaction (firm/soft)

Replace transaction withcontingency action (hard)

Postpone transaction execution

(soft)

Recovery Issues


94/189

101

Recovery Issues

Recovery of temporal as well as static data necessary

Not always necessary to recover original state because of temporalvalidity intervals and application semantics:

if recovery takes longer than the absolute validity interval, it

would be a waste to recover that value example: recovery from a telephone connection switch failure

if connection already established: recover billing informationand resources, but no need to recover connection information

if connection was being established: recover assigned

resources

Recovery Issues (2)


95/189

102

Recovery Issues (2)

Real-time database recovery must consider

time and resource availability: recovery must not jeopardizeongoing critical transactions

available transaction semantics (or state): contingency or

compensating transactions can be used state of the environment: extrapolation of state may be possible,

or more up-to-date data may be available from the sensor soon

Its appropriate to use partitioned and parallel logging so that criticaldata with short validity interval can be recovered first, without going

through the entire log classify data according to its update frequency and importance,

and utilize non-volatile high-speed memory for logging


96/189

103


97/189

104

IV. Database Techniques for Real-Time

Applications

Outline

Active, main-memory, and object-oriented databases

Flexible security paradigm for real-time applications

Embedded databases

Active Database Systems


98/189

105

y

Database manager reacts to events

Transactions can trigger other transactions

Triggers and alerters

Actions specified as rules ECA-rules (event - condition - action)

Upon Event occurrence, evaluate Condition, and if condition issatisfied, trigger Action

Coupling Modes: immediate(triggered action will be executed right

away), deferred(it will be executed at the end of the currenttransaction), detached(scheduled as a separate transaction)

Cascaded triggering is possible

Active Real-Time Database Systems


99/189

106

y

Real-time systems are inherently reactive

ECA-rules provide uniform specification of reactive behavior

Problems with active database systems techniques:

Additional sources of unpredictability

event detection and rule triggering

all coupling modes are not feasible

No specification of time constraints

Techniques are not time-cognizant

Temporal scope...


100/189

107

Total response time

Event detection time Rule execution time Action exec. time

timeevent

occurrence

event

detection

composite event detection

event delivery

rule retrieval

action execution start

action spawning

condition evaluation

action execution complete

p p

Main Memory RTDBS


101/189

108

y

Characteristics of Main Memory DBS

the primary copy of data resides in main memory, not in disksas in conventional database systems

memory resident data need a backup copy on disk

Why being pursued?

it becomes feasible to store larger databases in memory asmemory becomes cheaper and chip densities increase

direct access in memory provides much better response timeand transaction throughput, and improved predictability due tothe lack of I/O operations

Main Memory RTDBS (2)


102/189

109

y ( )

Difference from disk-resident databases with large cache

it can use index structures specially designed for memoryresident data (e.g., T-tree instead of B-tree)

it can recognize and exploit the fact that some data residepermanently in memory -> data partitioning

Data partitioning can be effectively used different classes of data: hot, warm, cool, and cold, based on

the frequency of access and/or timing constraints of the access(deadline of the transactions)

in telephone switching systems, for example, routing

information is hot, while customer address data is cold

Main Memory RTDBS (3)


103/189

110

y ( )

Consequences of memory board failures in MMDBS

typically, entire machine need to be down, losing the entire DB,while in disk-resident DB, only the affected portion will beunavailable during recovery

recovery is time-consuming, and having a very recent backup

available is highly preferred -> more backups need to be takenfrequently, resulting in high cost

--- performance of backup mechanism is critical

Impacts of Memory-Residency in RTDBS


104/189

111

y y

Concurrency control

since lock duration is short, using small locking granules toreduce contention is not effective

--- large lock granules are appropriate in MM-RTDBS

even serial execution can be a possibility, eliminating the cost ofconcurrency control

potential problems of serial execution:

long transactions to run concurrently with short ones

need synchronization for multiprocessor systems

lock information can be included in the data object, reducing thenumber of instructions for lock handling

--- performance improvement

Impacts of Memory-Residency in RTDBS (2)


105/189

112

Impacts of Memory Residency in RTDBS (2)

Commit processing

to protect against failures, logging/backup necessary

--- log/backup must reside in stable storage (e.g., disks)

before a transaction commits, its activity must be written in the

log: write-ahead logging (WAL)

logging threatens to undermine performance advantage:

response time: transaction must wait until logging is done on disk ->logging can be a performance bottleneck

possible solutions:

small in-memory log, using non-volatile memory (e.g., flashmemory)

pre-commit and group commit strategy



106/189

113


Query processing

sequential access is not significantly faster than random accessfor memory-resident data -> techniques taking advantage offaster sequential access lose merit

query processor must focus on actual processing costs, instead ofminimizing the number of disk access

costly operations such as index creation or data copying shouldbe identified first, and then processing strategy can be designedto reduce their occurrences

because of memory residency of data, it is possible to constructcompact data structures for speed up

--- e.g., join operation using pointers

Trends in Memory Resident RTDBS


107/189

114

Trends in Memory-Resident RTDBS

Extended use of pointers

relations are stored as linked list of tuples, which cane be arrays ofpointers to attribute values

Combined hashing and indexing:

linear hashing for unordered data and T-tree for ordered data Large lock granules or multi-granule locks

Deferred updates, instead of in-place update

Fuzzy checkpointing for reduced transaction locking time

Special-purpose hardware support for logging and recovery

Object-oriented design and implementation

Object-Oriented RTDBS


108/189

115

OO data models support

support for modeling, storing and manipulation of complex objectsand semantic information into databases

encapsulated objects

OO data models need (for RT applications) time constraints on objects, i.e, attributes and methods

Objects more complex -> unit of locking is the object -> lessconcurrency

memory-resident RTDB may fit well with this restriction

inter-object consistency management could be difficult

Need better solutions to provide higher concurrency and predictableexecution for RT applications


109/189

116


110/189

117

Real-Time Security Paradigm ...

Real-Time Secure Data Management


111/189

118

Characteristics

transactions with timing constraints

data with temporal properties

mixture of sensitive and unclassified data

Requirements timeliness and predictability

temporal consistency

adaptive security enforcement

high performance

Real-Time Secure Data Management


112/189

119

Issues integrate support of different types of

requirements

predictability yet flexible execution

conflicts between real-time and security

real-time management of resources

high performance yet fault-tolerant

trade-offs

scalability of solutions

Database Security


113/189

120

Security services

to safeguard sensitive information

encryption, authentication, intruder detection ...

Multilevel security (MLS)

objects are assigned with security classification

subjects access objects with security clearance

no flow of information from higher level to lower one

Applications

almost everywhere (becoming a buzzword)

more flexibility necessary (from static, known environment todynamic unknown environment)

Trends


114/189

121

Increasing number of systems operate in unpredictable (evenhostile) environments

task set, resource requirements (e.g., worst-case executiontime) ...

High assurance required for performance-critical applications

System properties for high assurance

real-time (timeliness, temporal consistency ..)

security (confidentiality, authentication ..)

fault-tolerance (availability, reliability ..)

Each property has been studied in isolation

Security and Real-Time


115/189

122

Security and Real Time For timeliness, no priority inversion in real-time applications

tasks with earlier deadline or higher criticality has higher

priority for better service

In secure systems, no security violation is allowed

Incompatible under the binary notion of absolute security

priority inversion vs security violation

Higher security services require more resources

Example of the Problem


116/189

123

Example of the Problem

Both require lock on the resource

How to resolve this conflict? if lock is given to T1, security violation

if lock is given to T2, priority inversion

T1- high priority

- high security

T2- low priority

- low securityAccess Access

Resource

Requirement for Real-Time Secure DBS


117/189

124

Requirement for Real-Time Secure DBS

Supporting both requirements of real-time

and security for real-time databases:

How to provide acceptably high security

while remains available and provides timely

services?

Research Issues


118/189

125

Research Issues

Flexible security vs absolute security

paradigm for flexible security services

identifying correct metrics for security level

Adaptive security policies

Mechanisms to enforce required level of security and trading-offwith other requirements:

access control, authentication, encryption, ..

time-cognizant protocols, data deadlines, ...

replication, primary-backup, ...

Specification to express desired system behavior

verification of consistency/completeness of specification

Flexible Security Services


119/189

126

Flexiblevs absolute (binary) security

traditional notion of security is binary: secure or not

problem of binary notion of security: difficult to provideacceptable level of security to satisfy other conflicting

requirements research issue: quantitative flexible security levels

One approach

represent in terms of % of potential security violations

problem: not precise --- percentage alone reveals nothing

about implications on system security

e.g., 1%violation may leak most sensitive data out

Flexible Security for Access Control


120/189

127

Possible approaches to provide flexible security

control potential violations between certain security levels

even if it allows potential security violations, it does not completelycompromise the security of the system

use different algorithms in an adaptive manner

A possible configuration

Top secret

Secret

Confidential

Unclassified

A

Top secret

Secret

Confidential

B

Top secret Top secret

Secret Secret

Confidential ConfidentialUnclassified Unclassified Unclassified

C D

Flexible Security Policies (5 levels)


121/189

128

Completely secure: no violations allowed

Secure levels 2, 3 & 4: high 3 levels kept completely secure

Secure levels 3 & 4: high 2 levels kept completely secure

Split security: violations allowed between top 2 levels, andamong low 3 levels

Secure level 4: highest level kept completely secure

No security: violations can occur between any levels

Gradual security: control the number of violationbetween eachlevel

Performance of Flexible Access Control


122/189

129

Performance of Flexible Access Control

Significant improvement in real-time performance as more potentialcovert channels allowed:

completely secure (6.5%) vs no security (3.3%) for 500 dataitems

complete secure (5%) vs no security (1%) for 1000 data items

Trade-off capacities of security policies are strictly ordered: from completely secure through multiple secure levels to no

security

Improved Functionality


123/189

130

Exploiting real-time properties for improved/new features

Example: Intrusion detection

sensitive data objects are tagged with time semantics thatcapture normalbehavior about read/update

time semantics should be unknown to intruder violation of security policy can be detected:

suspicious update request can be detected using aperiodic update rate

tolerance in the deviation from normalbehavior can be

parameterized

Adaptable Security Manager


124/189

131

Need for resource tradeoffs in database services

Adaptable security manager fits well with the concept of multipleservice levels in real-time secure database systems

Short term relaxation of security could be preferable to missed

critical deadlines aircraft attack warning during burst of battlefield updates

loss of production time for missed agile manufacturingcommand

Features of Adaptable Security Manager


125/189

132

Multiple security levels on users/objects or communications

computation costs increase with level of security

Client negotiated range of security levels for transaction

communicationsDynamic level changes as a function of real-time load

Security Manager Environment


126/189

133

session &

transaction

requests

Security Manager

Client

Table

Session

Table

Beehive

TransDatatransactionresults

thread n

thread n-1

DB

Scheduler

Mapper/

Admission

Control

data flow

execution

control

transaction object &

session data

client

security

level & key

session

keys

& status transaction

handoff

object read

& write

Security Level Synchronization


127/189

134

Sec Mgr

events

Client Xevents

Client X

level

Sn

Sec Mgr

level

3

2

1

0

Rn+1Sn

Sn+1 Rn

prepare for

2 step switch

Sn+2

Rn+1

prepare to switchlast message

accounted for

Rn+2

Sn+2

switch

Sn+3

Rn+3

received

acknowledgment

time

LEGEND

Sn

Rn

transaction request

request with switch

acknowledgment

transaction response

message

response with

switch command

send the nth

message

receive the nth

message

t1

t2 t3 t4

t5

Rn

3

21

0

Performance: Adaptive vs. Non-Adaptive


128/189

135

0

20

40

60

80

100

120

2 1.5 1 0.5 0.2

deadline/mean execution time ratio

%deadlinesmade

100% adaptive

50% adaptive

0% adaptive

In adaptive control,the system lowers the security dynamically

Level Switching


129/189

136

(100% adaptive client)

0

20

40

60

80

100

120

2 1.5 1 0.5 0.2

deadline/mean execution time ratio

%d

eadlinesmade

3

2

1

0

L

E

V

E

L

% MADE

LEVEL

It shows the security level change and the miss ratio change

Performance Results


130/189

137

Good performance gains achievable in soft real-time system duringoverload conditions

When the overload is not severe, switching the security level canbring the desired performance back (as shown in the graph)

If the system is too much overloaded or some component failed,then even reducing the security level to 0 cannot keep the systemworking properly (meeting critical deadlines)

Performance gain depends also on other factors such as messagesize and I/O cost: significant performance improvement with large

message sizes with large I/O overhead


131/189

138


132/189

139

Embedded Real-Time Databases....

Whats an Embedded Database?


133/189

140

Same principal functionality as a desktop database (exluding the

most complex operations)

Two types:

Application-embedded databases

Generally not much real-time requirements Device-embedded databases

Embedded systems

Strict timing constraints involved

Requirements of Device-Embedded DBS


134/189

141

Small footprint due to limitied storage and memory resources

~150 Kb

High dependability

continous uptime with little or no maintenance (i.e., the database

should be able to perform recovery by itself). Mobile capabilities

Interoperability and portability

Communication and synchronization with external databases

Real-time constraints

Maintainability

Security

Existing Embedded Databases


135/189

144

Progress

Ardent Software

InterSystems

Centura SQLBase Embedded Database

IBM DB2 Everywhere and Universal Database Satellite

Microsoft SQL Server Oracle8i Lite

Sybase SQL Anywhere Studio (and UltraLite Deployment Option)

Pervasive.SQL 2000 SDK for Mobile and Embedded Devices


136/189

Embedded Database Systems Applications


137/189

146

Embedded Database Systems Applications

Mobile databases

Portable computing devices

Smart ceullular phones with Internet access

PDAs

Laptops

Embedded systems Car engine control, brake system control, ...

Tiny embedded databases

Smart cards

Intelligent appliances

Network routers and hubs Set-top Internet-access boxes

Embedded DBS: Research Challenges


138/189

148

Portability and interoperability Availability

Recovery protocols that recover the database while the databaseis still guaranteeing some level of service

Continuous up-time

Query language What are the necessary operators (application dependent)

Concurrency control schemes

Architecture

Building a database from a portfolio of modules (components)

Application-dependent tuning of functionality and configuration

Minimizing functionality -> minimizing memory usage


139/189

150

Applications


140/189

152

Air Traffic control

Aircraft Mission Control

Command Control Systems

Distributed Name Servers

Manufacturing Plant Navigation and Positioning Systems

automobiles, airplanes, ships, space station

Network Management Systems

Applications (2)


141/189

153

Real-time Process Control

Spacecraft Control

Telecommunication

Cellular phones

Normal PBX Training Simulation System

Pilot training

Battlefield training

Air Traffic Control


142/189

154

Multiple control centersControls airspace

Terminal areas

Enroute airspace

Databaseaircraft: aircraft identification, transponder, code,

altitude, position, speed, etc.

flight plans: origin, route, destination, clearances,

etc.environment data: airport configuration, navigation

facilities, airways, restricted areas, notifications,weather data, etc.

Air Traffic Control (2)


143/189

155

Contents and Size of DB350 airports, 250 navigation facilities, and 1500 air-

crafts

weather data, etc.

DB size: ~20,000 entities

Time requirements

mean latency: 0.5 ms

max latency: 5 ms

external consistency: 1 sec

temporal consistency: 6 secs

permanence requirements: 12 hours

Military Aircraft Mission Control (contd)


144/189

156

Database:

Tracking information

2000 air vehicles

250 ground entities, e.g., vehicles, airports,

radars, etc. flight plan, maps, intelligence etc.

DB size: ~3000 - 4000 entities

Military Aircraft Mission Control


145/189

157

Time requirements

mean latency: 0.05 ms

max latency: 1 - 25 ms

external consistency: 25 ms

temporal consistency: 25 mspermanence req.: 4 hours

Integrated Automobile Control


146/189

160

TCM - Transmission Control Module

TCS - Traction Control System

CBC - Corner Braking Control

DCS - Dynamic Safety Control ESP - Electronic Stabilization Program

Car Diagnosis Systems

Hard and soft TCs Significant interaction with external environment

Distributed


147/189

161

VI. Real-Time Database Rsearch Prototype:


148/189

162

VI. Real Time Database Rsearch Prototype:

BeeHive System

Commercial RTDBs


149/189

163

Polyhedra

http://www.polyhedra.com/

Tachys, (Probita)

http://www.probita.com/tachys.html

ClustRa

http://www.clustra.com

DBx

Eaglespeed (Lockheed-Marthin)

RTDBMS (NEC)

(Mitsubishi)

RTDBS Research Projects


150/189

164

BeeHive University of Virginia, USA

DeeDS

University of Skvde, Sweden

Rodain

University of Helsinki, Finland

RT-SORAC

University of Rhode Island, USA

MDARTS

University of Michigan, USA STRIP

Stanford University, USA

BeeHive: Global Multimedia Database

Support for Dependable, Real-Time

A li ti


151/189

165

Applications

Real-Time Systems Group

Dept of Computer Science

University of Virginia

Applications of BeeHive


152/189

166

Applications of BeeHive

Real-Time Process Control

hard deadlines, main memory, need atomicity andpersistence

limited or no (i) schema, (ii) query capability

Agile Manufacturing

Business Decision Support Systems

information dominance

Intelligence Community

Global Virtual Multimedia Real-Time DBs

SatelliteR l Ti A ti


153/189

167

Imagery

News

Services

Need SummaryReport by 4 PM

Troop

Positions

Network

World Wide

Real-Time, Active

Temporal DBs

Archival

DBs

The Problem

Scenario

Multimedia

DB

Transaction Deadlines in BeeHive


154/189

168

Hard - deadline must be met else catastrophic result

suitable for some RTDB, in which timing constraintsmust be guaranteed, and the system supports

predictability for certain guarantees Firm - deadline must be met else no value to executing

transaction (just abort)

Soft - deadline should be met, but if not, continue to

process until complete

Absolute Validity Interval (AVI) and

Relative Validity Interval (RVI)


155/189

169

10 10 10 10 10 10

20 20 20

X

Y

Absolute Validity Interval (X) = 10

Absolute Validity Interval (Y) = 20

Relative Validity Interval X-Y < 15

Data in BeeHive


156/189

170

Data from sensors (including audio/video)

Derived data

Time-stamped data

Absolute consistency - environment and DB

Relative consistency - multiple pieces of data

Schema and meta data

User objects (with semantics)

Pre-declared transactions (with semantics)

Global Virtual Databases - BeeHive


157/189

171

Dynamically reconfigure and collect DBs (Tailored forSome Enterprise)

Interact with External DBs

Utilize Distributed Execution Platforms

PropertiesReal-Time

QoS

Fault Tolerance

Security

BeeHiveSystem


158/189

172

Search

RDBMS

OODB

RAW

B

W

B

W

B

W

B

W

BeeHive System

Native BeeHive Sites

BeeHive

Wrapper

BeeHive Object Model


159/189

173

BeeHive Object is specified by

N, the object ID

A, set of attributes (name, domain, values)

value -> value and validity interval

semantic information

M, set of methods

name and location of code, parameters, execution

time, resource needs, other semantic informationCF, compatibility function

T, timestamp

BeeHive Transactions


160/189

174

BeeHive transaction is specified by < TN, XT, I, RQ, P >

TN, unique ID

XT, execution code

I, importance

RQ, set of requirements (for each ofRT, QoS, FT, andsecurity) and optional pre and post conditions

P, policy for tradeoffs

Example: if all resources cannot be allocated reduceFT requirement from 3 to 2 copies.

Dealing With Soft/Firm RT Transactions


161/189

175

Resolve inter-transaction conflicts in time cognizantmanner (concurrency control)

Assign transaction priorities (cpu scheduling)

Goals


162/189

176

Maximize the number of TRs (sensor and user) thatmeet deadlines

Keep data temporally valid

on overload allow invalid intervals on data (note thatdata with invalid interval may not be used during thatinvalid time)

The External Interface


163/189

177

Raw Data

Structured Data

Data

manipulation

BeeHive

integration

BeeHive

Cogency

Monitor

Returned data

ObjectData

maintained

by cogency

monitor

(external

to

BeeHive)

Internet Open

Databases

Unstructured

Data

Taxonomy of external data


164/189

178

Structured (databases)

Unstructured (search engines)

Raw (video, audio, sensors)

Cogency Monitor


165/189

179

Support value added services

RT, FT, QoS, and Security

Execute client supplied functionality

Map incoming data into BeeHive objects

Monitor the incoming data for correctnessand possiblymake decisions based on the returned data

not just a firewall

Cogency Monitor GUI


166/189

180

Pick and choose from template (value added services)

Upon one choice - compatibility function executes thatlimits other choices

Identify added functionality

correctness

mapping to internal BeeHive objects

Automatic generation of the Cogency Monitor

generated automatically using library functions (notimplemented yet)


167/189

Unstructured cogency monitor


168/189

182

Current Research Activities in BeeHive


169/189

183

Basic BeeHive

Storage

Manager

SecurityRTDB

Internals

Admission

Control

RT

Threads

Database

BeeHive Front EndJava

Cogency Monitor

Expand

DB

Simulation

Summary: BeeHive Project


170/189

184

Global real-time database system

object-based with added object semantics

support in RT, FT, QoS, and Security

different types of data: video, audio, images and text

sensors and actuators Novel component technology

data deadline, forced delay, conditional priority inheritance

real-time logging and recovery

security-performance tradeoff

resource models for admission control

Cogency Monitor


171/189

185


172/189

Future Research Areas that Affect RTDBResearch


173/189

191

Interface and component libraries open interfaces

dealing with semantic mismatches

micro and macro components

Real-time data/information centric view

as opposed to task centric view currently used Adaptive scheduling and decision making based on changing situation

and incomplete workload and component profiles

Component-based tool sets

Configuration tools

Tools to specify and integrate requirements of real-time and fault-tolerance

Future Research Areas that Affect RTDBResearch (2)


174/189

192

New Requirements

Complex software must evolve

Software must be portable to other platforms

develop once

verify oncecertification and verification is very expensive

port and integration should be automatic

Flexible real-time data support

one-size-fits-all does not work: small with minimal functionalityfor embedded systems while complete and full functionality forback-end server applications

Future Research Areas in RTDBS


175/189

193

Resource management and scheduling temporal consistency guarantee (especially relative validity

intervals)

interactions between hard and soft/firm RT transactions

transient overload handling

I/O and network scheduling

models which maximizes both concurrency and resourceutilization

support of different transaction types for flexible scheduling:

Alternative, Compensating, Imprecise Recovery

availability (partial) of critical data during recovery

semantic-based logging and recovery

Future RTDBS Research Areas (2)


176/189

194

Concurrency Control

alternative correctness models (relaxing ACID properties)

integrated and flexible schemes for concurrency control

Fault tolerance and security models to interact with RTDBS

Query languages for explicit specification of real-time constraints ->RT-SQL

Distributed real-time databases

commit processing

distribution/replication of data

recovery after site failure

predictable (bounded) communication delays

Future RTDBS Research Areas (3)


177/189

195

Data models to support complex multimedia objects

Schemes to process a mixture of hard, soft, and firm timingconstraints and complex transaction structures

Support for integrating requirements of security and fault-tolerance

with real-time constraints Performance models and benchmarks

Support for more active features in real-time context

techniques for bounding time in event detection, rule evaluation,rule processing mode, etc.

associate timing constraints with triggering mechanisms Interaction with legacy systems (conventional databases)


178/189

196

VIII Exercises


179/189

197

VIII. Exercises

Exercise (1)


180/189

198

Suppose we have periodic processes P1 and P2, which measure pressureand temperature, respectively. The absolute validity interval of both ofthese parameters is 100 ms. The relative validity interval of atemperature-pressure pair is 50 ms. What is the maximum period of P1and P2 that ensures that the database system always has a valid

temperature-pressure pair reading?

Exercise (2)


181/189

199

Sometimes a transaction that would have been aborted under the two-phase locking protocol can commit successfully under the optimisticprotocol. Why is that? Develop a scenario in which a case of suchtransaction execution occurs.

Exercise (3)


182/189

200

Explain why EDF does not work well in a heavily loaded real-timedatabase systems, and propose how you can improve the success rateby adapting EDF. Will your new scheme work as well as EDF in lightlyloaded database systems? Will it work well in real-time applicationsother than database systems?

Exercise (4)


183/189

201

Generate examples of an application where it is permissible to relax oneor more ACID properties of transactions in real-time database systems.

Exercise (5)


184/189

202

Suppose a transaction T has a timestamp of 100. Its read-set has X1and X2, and its write-set has X3, X4, and X5. The read timestamps ofthese data objects are (prior to adjustment for the commitment of T) 5,10, 15, 16, and 18; their write timestamps are 90, 200, 250, 300, and 5,respectively. What should be the read and write timestamps after the

successful commitment of T? Will the value of X3, X4, and X5 will bechanged when T commits?

Exercise (6)


185/189

203

Why the concurrency control protocols used in conventional databasesystems are not very useful for real-time database systems? What arethe information that can be used by real-time database schedulers?

Exercise (7)


186/189

204

Compare pessimistic and optimistic approaches in concurrency controlwhen applied to real-time database systems. Discuss different policies inoptimistic concurrency control and their relative merit

Documents

Rtdb-Academic Course by S.H.son