Upload
lisbet
View
133
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Data Warehouse Fundamentals. Rabie A. Ramadan, PhD 2. What did you do in Your Assignment ?. For an airlines company, how can strategic information increase the number of frequent flyers? Discuss giving specific details. - PowerPoint PPT Presentation
Citation preview
Data Warehouse Fundamentals
Rabie A. Ramadan, PhD
2
2
What did you do in Your Assignment ?
For an airlines company, how can strategic information increase the number of frequent flyers? Discuss giving specific details.
You are a Senior Analyst in the IT department of a company manufacturing automobile parts. The marketing heads are complaining about the poor response by IT in providing strategic information. Draft a proposal to them explaining the reasons for the problems and why a data warehouse would be the only viable solution.
3
What did you do in the Project ? Egypt Election System
• Governorates’ database system • Multiple databases on Multiple Servers
• Summarization System • Meta data
• Data Warehouse Server
• Web page with query based system
4http://www.inf.unibz.it/dis/teaching/DWDM/index.html
5
Definitions & Motivations
Why Data Mining? Explosive Growth of Data: from terabytes to petabytes Data Collections and Data Availability
• Crawlers, database systems, Web, etc.
Sources• Business: Web, e-commerce, transactions, etc.
• Science: Remote sensing, bioinformatics, etc.
• Society and everyone: news, YouTube, etc.
6
Why Data Mining?
Problem: We are drowning in data, but starving for knowledge!
Solution: Use Data Mining tools for Automated Analysis of massive data sets
7
What is Data Mining?
Data mining (knowledge discovery from data)• Extraction of interesting (non-trivial, implicit,
previously unknown and potentially useful) patterns or knowledge from huge amount of data
8
What is Data Mining?
Alternative names• Knowledge discovery (mining) in databases (KDD),
• knowledge extraction,
• data/pattern analysis,
• data archeology,
• Data dredging,
• information harvesting,
• business intelligence,
• etc.
9
Knowledge Discovery (KDD) Process
10
Knowledge Discovery (KDD) Process
11
Typical Architecture of a Data Mining System
12
Confluence of Multiple Disciplines
13
Why Confluence of Multiple Disciplines?
Tremendous amount of data• Scalable algorithms to handle terabytes of data (e.g., Flickr
had 5 billion images in September, 2010 [http://blog.flickr.net/en/2010/09/19/5000000000/])
High dimensionality of data• Data can have tens of thousands of features (e,g., DNA
microarray)
14
Why Confluence of Multiple Disciplines?
15
Different Views of Data Mining Data View
• Kinds of data to be mined Knowledge view
• Kinds of knowledge to be discovered Method view
• Kinds of techniques utilized Application view
• Kinds of applications
16
Data to Mined
In principle, data mining should be applicable to any data repository
We will have examples about:• Relational databases• Data warehouses• Transactional databases• Advanced database systems
17
Relational Databases
18
Data Warehouses
19
Transactional Databases
20
Advanced Database Systems(1)
21
Advanced Database Systems(2)
22
Knowledge to be Discovered
23
Characterization and Discrimination
24
Characterization and Discrimination (1)
25
Class Activity
• Differentiate between Data Mining and Data warehousing?
Data warehousing is merely extracting data from different sources, cleaning the data and storing it in the warehouse. Where as data mining aims to examine or explore the data using queries
What are the Different problems that “Data mining” can solve? Data mining can be used in a variety of fields/industries like marketing,
advertising of goods, products, services, AI, government intelligence. How does the data mining and data warehousing work
together? Data warehousing can be used for analyzing the business needs by storing
data in a meaningful form. Using Data mining, one can forecast the business needs. Data warehouse can act as a source of this forecasting.
26
Frequent Patterns, Associations, Correlations
27
Classification and Prediction
28
Cluster Analysis
29
Outlier Analysis
30
Evolution Analysis
31
Techniques Utilized
32
Applications Adapted
33
Major Challenges in Data Mining
34
Summary