Upload
daniel-manning
View
222
Download
0
Tags:
Embed Size (px)
Citation preview
Who Needs to Know about Who Needs to Know about Database Design ConceptsDatabase Design Concepts
System Architects Applications Designers System Managers Operators End users and of course DBA’s
System Architects Applications Designers System Managers Operators End users and of course DBA’s
In short just about everyone can benefit
Why?Why?
DataDatabasebase
The database is usually the center of the system universe
The database is usually the center of the system universe
End Users
Reporting Development
OLTP
The supports the business Business drives the data
The supports the business Business drives the data
Everyone effects the Everyone effects the DatabaseDatabase
?
?
?
One users can bring the system to its kneesOne users can bring the system to its knees
Locking IssuesResource Hogging
The Database is a Common The Database is a Common ResourceResource
Poor database performance means poor business performance
Poor database performance means poor business performance
Inaccurate Reporting
Slower Development
Wasted Resources
Customer Complaints
DatabaseDatabaseDatabaseDatabase InflexibleArchitectures
How is a Database built?How is a Database built?
Business Needs Analysis Conceptual Design Logical Design Physical Design Testing Deployment Maintenance
Business Needs Analysis Conceptual Design Logical Design Physical Design Testing Deployment Maintenance
Design StepsDesign Steps
Same logic as Application Same logic as Application Development life cycleDevelopment life cycle
BNA Conceptual Design Logical Design Physical Design Tuning Deployment Maintenance
BNA Conceptual Design Logical Design Physical Design Tuning Deployment Maintenance
RFP Requirements Gathering Functional Design Detail Design & Build Acceptance Testing Roll-out Enhancements
RFP Requirements Gathering Functional Design Detail Design & Build Acceptance Testing Roll-out Enhancements
Blueprint for DisasterBlueprint for Disaster
A poor foundation means.... a shaky future...
A poor foundation means.... a shaky future...
Building BlocksBuilding Blocks
BNA
Conceptual Design
Logical Design
Physical Design
Test
Deploy
Maint
Each block builds on the previous ones
Each block builds on the previous ones
Like the PyramidsLike the Pyramids
Business Needs AnalysisBusiness Needs Analysis
The initial proposal...The initial proposal...
Scopes the project Mission Statement Mission Critical
Success Factors
Scopes the project Mission Statement Mission Critical
Success Factors
Project ScopeProject Scope
High level view of the project Establishment of project goals Establishment of project boundaries Rough view of data requirements
High level view of the project Establishment of project goals Establishment of project boundaries Rough view of data requirements
Mission StatementMission Statement
Defines project scope as clearly and concisely as possible
Defines project scope as clearly and concisely as possible
The Phoenix project will convert the existing paper order entry system to anelectronic one within one year and providethe basis for a customer tracking system at completion.
Helps retain project focus through delivery
Mission Critical Success Mission Critical Success FactorsFactors
Define the basic project metrics , to measure its success while controlling project scope
Define the basic project metrics , to measure its success while controlling project scope
System must be completed within one year System must process 1,500 orders per day Planned growth is 45,000 orders per year Response time must be 3 seconds for on-line Standard inventory reports will be produced System will use existing hardware
MCSF and Red FlagsMCSF and Red Flags
MCSF establish a framework with the customerand eliminate unrealistic “wish list” expectations
MCSF establish a framework with the customerand eliminate unrealistic “wish list” expectations
“45,000 orders per year are only 22.5 per day, are some days heavier for order entry?”
“If you want to use your existing PC-XT’s, you’re not getting three second response time...”
BNA DeliverablesBNA Deliverables
Basic system requirements Project Scope Mission Statement Mission Critical Success Factors
Conceptual DesignConceptual Design
Identify information sources– Existing forms, reports, documentation– Existing training materials– Key People
Identify information sources– Existing forms, reports, documentation– Existing training materials– Key People
Requirements gatheringRequirements gathering
Understanding the customer’s requirements meansunderstanding their business as well as they do...
Understanding the customer’s requirements meansunderstanding their business as well as they do...
Uncovering Business RulesUncovering Business Rules
A business rule is... Something you know about the data, upon which design decisions will be based.
A business rule is... Something you know about the data, upon which design decisions will be based.
The new system is great but how do I enter thecustomer’s billing address
?!??!?
Business Rules and MetadataBusiness Rules and Metadata
Metadata dictionary is data about dataMetadata dictionary is data about data
Used as a reference for all usersUsed as a reference for all users
Cash Batch Header
Customer addresses Billing
Shipping
Corporate
Conceptual Design Conceptual Design DeliverablesDeliverables
Methodology, Standards and Guidelines System Hardware and Software requirements Rough data requirements from Functional Design
Specification Beginning of Metadata Dictionary List of known Business Rules
Logical DesignLogical Design
Defining the data structures to fit the business needsDefining the data structures to fit the business needs
Entity Analysis and Normalization Producing the ERD CRUD Diagram Transaction Analysis Table Analysis
Entity Analysis and Normalization Producing the ERD CRUD Diagram Transaction Analysis Table Analysis
Where do you start?Where do you start?
School assignment, 5-10 tables, no problem...
Small subsystem, 10- 20 tables, not to tricky...
Real system design, 100+ tables, hmmmm...hmmmm...
Full blown mission critical, 800+ tables, Yikes!!!Yikes!!!
Which comes first, the Chicken or the Which comes first, the Chicken or the Attribute?Attribute?
Do you create the data sets from the applications? Or do you define the applications once you have the data?
Do you create the data sets from the applications? Or do you define the applications once you have the data?
It is an evolving process, functional design to detail design, data and applications needs will grow and develop together
It is an evolving process, functional design to detail design, data and applications needs will grow and develop together
???
Time to Cook, Time to Cook, Entity SoupEntity Soup
Order Form Order number Customer number Item 1...
Entity SoupEntity SoupCus
tom
er
Prof
ile
Throw all entity attributesinto the “entity soup”
Throw all entity attributesinto the “entity soup”
Use Normalization to sortthem out into coherenttables
Use Normalization to sortthem out into coherenttables
Use the followingUse the followingrules...rules...
Use the followingUse the followingrules...rules...
Pick your Starting EntitiesPick your Starting Entities
ORDER CUSTOMER
Order items
Order infoParts
info SalespersonEmployee
Customer info
Populate the EntitiesPopulate the Entities
ORDER CUSTOMER
Order items
Order infoParts
info SalespersonEmployee
Customer_nameAddress
Order item
Item desc
Populate the EntitiesPopulate the Entities
ORDER CUSTOMER
SalespersonEmployee
Customer_nameAddress
Order itemItem desc
Some are not... (an Employee fills an order, and also takesan order as the customer’s sales person... where does it go?)
Populate the EntitiesPopulate the Entities
ORDER CUSTOMER
Parts info Sales
personEmployee
Customer_nameAddress
Order itemItem desc
Don’t worry, pick one or the other, but not both!Don’t worry, pick one or the other, but not both!
You don’t wantto duplicate data
Populate the EntitiesPopulate the Entities
CUSTOMER
Parts info Sales
personEmployee
Customer_nameAddress
Eliminate similar entity attributesEliminate similar entity attributes
Sales person and Employee are the same
Employee#Name
A Word about attribute naming A Word about attribute naming conventionsconventions
Homonyms - Same name, different meaning Customer address and Employee Address
Synonyms - Same meaning, different nameBadge_ID, Manager_id, Emp_no
Homonyms - Same name, different meaning Customer address and Employee Address
Synonyms - Same meaning, different nameBadge_ID, Manager_id, Emp_no
Build in Intuitiveness where possible
NormalizationNormalization
Breaking entities down into the most molecular level needed to be viewed by the business
Tinkertoys
Lego’s Lincoln Logs
WOOD BLOCKS?
Breaking entities down into the most molecular level needed to be viewed by the business
Tinkertoys
Lego’s Lincoln Logs
WOOD BLOCKS?
The process of NormalizationThe process of Normalization
Orders Customer
Order info Cust info
Items infoEmp info
InventoryDependents
Cust Credit
Warehs loc Part Descr
Normalization will take the large “super structures” and break them down...
The process of NormalizationThe process of Normalization
Orders Customer
Order info Cust info
Items info
Emp info
Inventory
Dependents
Cust Credit
Warehs loc
Part Descr
into the smallest components needed by the business view
Normal FormsNormal Forms
1NF, Select “Candidate Key”, remove repeating groups.
2NF, Functional dependencies dependent on the whole key.
3NF, Removal of Transitive Dependencies. BCNF, 4NF,5NF Domain-key NF
1NF, Select “Candidate Key”, remove repeating groups.
2NF, Functional dependencies dependent on the whole key.
3NF, Removal of Transitive Dependencies. BCNF, 4NF,5NF Domain-key NF
Normal FormsNormal Forms
1NF - “The Key” 2NF - “The whole Key” 3NF - “Nothing but the Key”
1NF - “The Key” 2NF - “The whole Key” 3NF - “Nothing but the Key”
If you remember this, and it makes sense, you understand the basics of normalization
Determinants and Repeating Determinants and Repeating GroupsGroups
Determinant - A field with a known value that can uniquely identify the value in another field.
Based on the Business Rules governing those fields
Repeating Group - Field that has multiple values
Determinant - A field with a known value that can uniquely identify the value in another field.
Based on the Business Rules governing those fields
Repeating Group - Field that has multiple values
When in doubt, ask the entity...Hey Employee with Emp_ID = 100,Do you know your name?... (yes)Do you know your shoe size?... (yes)What car do you own?... (which one?)
1NF, “1NF, “the key”the key”
Select a “primary key” from “candidate keys”Select a “primary key” from “candidate keys”
Remove repeating groups into their own table
Remove repeating groups into their own table
EMP_ID SSN Name & Start_date Shoe size Dependents
Whether something is a key, or repeating group is determined by the Business Rules for that attribute
2NF, “2NF, “the whole key”the whole key”
In a “concatenated key”, the entire key is a determinant for every attribute, (again based on Business Rules)
In a “concatenated key”, the entire key is a determinant for every attribute, (again based on Business Rules)
WAREHOUSE & BIN_LOC Whs_address Bin_Size Qty
To get warehouse address using the key, you must first finda part that is stored there
3NF, “3NF, “nothing but the key”nothing but the key”
Removal of non-key determinants to their own tables
Removal of non-key determinants to their own tables
CUST_NO Name Address ZIP
ZIP City State
Normalization ResultsNormalization Results
Orders table
Order_no N6Cust_no N10Ord_date DateShip_date DateEmp_id N8Invt_flg BooleanC_rec C12
MetadataDictionary
The results of the normalization process are clearly defined data structures that fits the business needs and a Metadata Dictionary which clearly describes the data
The results of the normalization process are clearly defined data structures that fits the business needs and a Metadata Dictionary which clearly describes the data
Entity Relation DiagramEntity Relation Diagram
An ERD it the roadmap to navigate through the database. Establish relationships with foreign keys.
An ERD it the roadmap to navigate through the database. Establish relationships with foreign keys.
Orders
O_Items
CustomerParts
C_Credit
Inventory
WarehseRelationships are based on Business Rules Make the ERD as intuitive as possible
Converting Function to TransactionConverting Function to Transaction
Order Entry
EnterOrder
Transaction #102 Add_an_Order Start Transaction Validate Customer Validate Credit Add Order Commit
Business Function
Business Function
ApplicationRequirements
ApplicationRequirements
Database Transaction Analysis
Database Transaction Analysis
Transaction Analysis SheetsTransaction Analysis Sheets
Trans #102 Description
Tables Used
Retrieval Typ Time of Day Priority Frequency Security Business Rules
Add an Order
Order, Item Customer, Customer_credit Index and index only M-F 8:00am-5:30pm High, 3 seconds up to 1,500/hour Update by supervisor Customer credit validatedbefore order is entered
Lays groundwork for physical design.
Allows transactions to becataloged and quantifiedfor performance tuning.
Lays groundwork for physical design.
Allows transactions to becataloged and quantifiedfor performance tuning.
CRUD DiagramCRUD Diagram
Table
Tra
nsa
ctio
nAdd Order
Fill Order
Update Inv
Mailing Rep
Update Whs
Credit Rep
OR
DE
R
CU
STO
ME
RC
_CR
EDIT
INV
EN
TO
RY
C RC RC U
D
U
R R
CU
R R
CRUD stands for Create, Read, Update, Delete CRUD stands for Create, Read, Update, Delete
CRUD tables show the interaction betweentables and transactions
An excellent tool forperformance andlocking predictions
CRUD tables show the interaction betweentables and transactions
An excellent tool forperformance andlocking predictions
Table Analysis SheetsTable Analysis Sheets
Table: OrdersDescription
CardinalityGrowthPrimary KeyForeign KeysAddDeleteUpdateReadConstraintsSecurityViews
Stores normal orderprocessing records100,000 from old sys45,000 new per yearOrder_noCust_noTrans #102, 106, 114Trans #116, 96Trans # 69Rept # 4, 6 13, 61Must have valid Cust_noUpdate by Supervisor used by Cust_Ord view
Table analysis is looking at transactions from a tablepoint of view.
Often useful for uncovering forgotten maintenance routines
Table analysis is looking at transactions from a tablepoint of view.
Often useful for uncovering forgotten maintenance routines
Logical Design Sanity CheckLogical Design Sanity Check
Order_no N6Cust_no N10Ord_date DateShip_date DateEmp_id N8Invt_flg BooleanC_rec C12
Add_Order
Fill_OrderUpd_Order
Orders
O_Items
The last step in Logical Design is to make sure that the datastructures do indeed support the transactions the end usersmust perform.
The last step in Logical Design is to make sure that the datastructures do indeed support the transactions the end usersmust perform.
Logical Design DeliverablesLogical Design Deliverables
Logical Tables ERD Transaction Analysis CRUD Diagram Tables Analysis
Physical DesignPhysical Design
Physical Layout Locking Strategies Indexing Options
Physical database construction Physical database construction
Built on the foundation provided by the Logical Design deliverables
Physical LayoutPhysical Layout
How does the database layer on the Operating System?
What are its memory requirements? How is it backed up and restored? How is it horizontally and vertically partitioned? How are the transaction logs stored? How many users can it support?
The answers to these questions can usually be found on the software documentation for the product
Physical LayoutPhysical Layout
File Placement Physical Size Log file placement Rollback segment
location Memory usage CPU usage
Build for performance, flexibility, recoverability and cost savings
Build for performance, flexibility, recoverability and cost savings
Physical Layout Physical Layout
Table: OrdersDescription
CardinalityGrowthPrimary KeyForeign KeysAddDeleteUpdateReadConstraintsSecurityViews
Stores normal orderprocessing records100,000 from old sys45,000 new per yearOrder_noCust_noTrans #102, 106, 114Trans #116, 96Trans # 69Rept # 4, 6 13, 61Must have valid Cust_noUpdate by Supervisor used by Cust_Ord view
Use the Table Analysis work sheets to help planthe size and growth potential of the tables and storage areas.
Use the Table Analysis work sheets to help planthe size and growth potential of the tables and storage areas.
Locking issuesLocking issues
Databases have concurrent users Locking protects record integrity
Read operations are usuallycompatible with other reads
Updates are exclusive and lockthe record
Read operations are usuallycompatible with other reads
Updates are exclusive and lockthe record
Locking ScenarioLocking Scenario
UPDATE
READREADREAD
READ
?
?
?? Make sure to prototype transactions
Transaction InteractionTransaction Interaction
Tra
nsa
ctio
nAdd Order
Fill Order
Update Inv
Mailing Rep
Update Whs
Credit Rep
OR
DE
R
CU
STO
ME
RC
_CR
EDIT
INV
EN
TO
RY
C RC RC U
D
U
R R
CU
R R
Use the CRUD Diagram and Transaction Analysis to uncover potential problem areas
Use the CRUD Diagram and Transaction Analysis to uncover potential problem areas
Trans #102 Description
Tables Used
Retrieval Typ Time of Day Priority Frequency Security Business Rules
Add an Order
Order, Item Customer, Customer_credit Index and index onlyM-F8:00am5:30pmHigh200/day - 1500 max
Issues of IndexesIssues of Indexes
Indexes are like a “phone book” They speed record retrieval They speed join operations between tables The more indexes, the faster things can be
found But... they slow updates
YellowPages
Index ExampleIndex Example
Name SSN DOB
Indexes
TableSelect * from employees wherename=‘Jones’and dob=‘05/14/59’;
Jones 55-555-5555 05/14/59
Index ExampleIndex Example
Name SSN DOB
Indexes
TableDelete from employeeswherename=‘jones’;
Jones 55-555-5555 05/14/59UPDATE
UPDATEUPDATE
UPDATE
One deleteforces 4 updates
Trans #102 Description
Tables Used
Retrieval Type Time of Day Priority Frequency
Add an Order
Order, Item Customer, Customer_credit Range and index only exact match M-F 8:00am-5:30pm High, 3 seconds up to 1,500/hour
Selecting the correct fields to Selecting the correct fields to IndexIndex
Select indexes based on the Transaction Analysis accessstrategies. Optimize the indices to enhance performance.
Select indexes based on the Transaction Analysis accessstrategies. Optimize the indices to enhance performance.
Don’t index haphazardly Don’t over-index tables Select the correct
– structure
– fill factor
– placement
Don’t index haphazardly Don’t over-index tables Select the correct
– structure
– fill factor
– placement
Logical Design DeliverablesLogical Design Deliverables
Physical Database Schema optimized to support mission critical transactions– Table and Index structures– File and storage area sizes– File and storage area locations– Backup and recovery architecture
TestingTesting
Create a testing environment that mimics the production environment as closely as possible
Create a testing environment that mimics the production environment as closely as possible
Build in benchmark gathering from the start
DeploymentDeployment
Deploy as unobtrusively as possible Deploy as unobtrusively as possible
The New
System!
1/1/1997
Complete and update documentation Load production data Beta Test deployment Train end users
Always have “Plan B” readyin case of unforeseen problems
MaintenanceMaintenance
As part of the design process, build in performanceestablish a maintenance programtrain the support staff
As part of the design process, build in performanceestablish a maintenance programtrain the support staff
Reorganize storage areas Reoptimize statistics Monitor transaction performance Rebuild indexes