Selecting and Implementing An Embedded Database
System
Presented by Jeff Webb
March 2005
Article written by Michael Olson
IEEE Software, 2000
OverviewKey Strategy – Focus on application
requirementsEmbedded DB products vary
Some do less than what you need Some will do more
Choose the tool that most closely matches your needs
OverviewAfter choosing OS, HW platform and DB
you must design for reliabilityDesign for performance up frontEvaluate performance once the
application is built
Evaluate Database Services Available High end RDM systems provide:
Concurrency Transaction processing Disaster recovery
Although these features may be needed, enterprise systems are seldom a good choice due to: platform differences packaging differences
Evaluate Database Services AvailableSeveral embedded DB systems use the
same techniques as enterprise systems but in smaller packages.
Often, full blown disaster recovery is not needed
Many embedded databases are configurable allowing inclusion or exclusion of services
Consider Services RequiredNow… consider which services you
need. For example:Locking?
Will run faster without lockingRecovery from failures matter?
Lack of, or disabling will increase performance
Operating Systems For Embedded systems Hundreds of OS for variety of processor
hardware DB developers have an enormous job porting
their software to the variety of platforms Few OS dominate the market
MS Win CE Neutrino VxWorks Wind Rivers
Databases For Embedded systemsClassification
Client Server relational systems Client Server OO systems Embedded libraries
ClassificationClient Server relational systems
+ Programmers already know SQL - Extra run-time cost of client server
communications
Classification Client Server OO systems
Appear to be a good choice however Most are designed for Unix systems and
their deeply engrained assumptions about memory management and interprocess communications are difficult to port to embedded OS.
ClassificationEmbedded libraries
Created specifically for embedded systems Provide simple language API that does not
require SQL + Faster execution, increased reliability - Require developers to master
nonstandard programming interfaces
Platform Support Consider combination of:
OS Processor Storage system Exotic processor boards
Rare combinations can be difficult to fit Often embedded OS vendors maintain lists of
partner companies whose products run on their systems
Performance ConsiderationsHigh concurrency?Size of databaseMultiple control threads?Can not rely on standard benchmark
measuring systemsEvaluation of actual performance is a
must
Designing the ApplicationDesign for performanceConsider:
Speed Predictability & Reliability
Designing the Application Speed Predictability & Reliability
Data representationAccess patternsConfiguration
Designing for SpeedData representation/Access patterns/Configuration
Most embedded DB tools operate on a fixed set of data types
Every fetch and store may require translation
A few systems, mostly library types allow storage in program-native format
Designing for Speed Data representation/Access patterns/Configuration
Consider the queries that the application will need to make
Data should be laid out accordinglyCan keys be used that will allow related
records to be physically stored together?
B+tree storage typically performs faster than hash table algorithms
Designing for Speed Data representation/Access patterns/Configuration
Must understand the chosen systems configuration options Memory use for secondary cache Write data to disk or store in memory? Locking system granularity Entire locking system on/off Vendors often choose the wrong defaults
Designing the ApplicationDesign for performanceConsider:
Speed
Predictability & Reliability
Predictability & ReliabilityMay need to run with no humans
presentNot easyFanatically check return values and
error indicatorsResource exhaustion
More common in embedded systems Track and release resources yourself
Predictability & ReliabilityAre transactions required so that
changes won’t be lost after a crash?
The recovery system must be callable from the application program on start up?
Performance tuningCommon causes for poor performance
Contention – 2 or more threads contending for same data
Disk-to-memory transfers Deadlocks
Contention/Disk-to-memory transfers/ Deadlocks
When several threads are waiting for the same resources Use record level locking if possible Use smaller pages if possible to make
page level locks more like record level locks
Touch the Hot data last and hold it for short periods of time
Contention/Disk-to-memory transfers/ Deadlocks
Disk latency – mechanicalFlash RAMMore memory for buffer cache Indexes
Contention/Disk-to-memory transfers/ Deadlocks
Obj O1 locked
T1
Obj O2 locked
T2
T1 waits for a lock on O2
T2 waits for a lock on O1
Deadlocks Turn off locking if it is not needed and if the
application permits Break large transactions into several smaller
transactions Write transactions so that they all acquire the
same resources in the same order Consider Optimistic Concurrency Control
Price Helps make final decision Some are available at no cost Licensing methods vary
Per developer Per application using it Per deployment platform Per user is less common in embedded systems
During development / deployment
Questions