Upload
marybabu10
View
69
Download
6
Tags:
Embed Size (px)
Citation preview
INTELLIGENT DATA OUTSOURCINGINTELLIGENT DATA OUTSOURCINGAn Improved Storage Availibility in Large Scale Data CentersAn Improved Storage Availibility in Large Scale Data Centers
2
OUTLINEOUTLINE
● Introduction
● Existing System
● Proposed System
● Analysis
● Conclusion
● References
3
INTRODUCTIONINTRODUCTION
Big data
● broad term for large datasets
● Business technology for modern enterprises
● Accuracy leads to redused risks
● Storage is a critical component
● Stored in disks
4
STORAGE SYSTEMS TASKSSTORAGE SYSTEMS TASKS
● High priority foreground tasks
● Low priority background tasks
Storage system Tasks
Foreground tasks Background tasks
5
LOW PRIORITY BACKGROUND TASKSLOW PRIORITY BACKGROUND TASKS
● RAID Reconstruction
● RAID Resynchronisation
● Disk Scrubbing
6
INEFFICIENCIES OF EXISTING SYSTEMINEFFICIENCIES OF EXISTING SYSTEM
● Time consuming
● Data loss
● Inefficient Storage Availibility
● Failure Induced optimization
● Does not exploit the predictable nature
● Passive
7
INTELLIGENT DATA OUTSOURCINGINTELLIGENT DATA OUTSOURCING
● Dynamically captures data popularity
● Exploits temporal and spatial access locality
● Balance between background tasks workflow and user I/O requests
● Portable
8
OPTIMIZATION SCHEMEOPTIMIZATION SCHEME
EXISTING SYSTEM
● Reactive Optimization
● Request Based
● Exploits Temporal Locality
PROPOSED SYSTEM
● Proactive Optimization
● Zone Based
● Exploits both temporal and spatial locality
9
OPTIMIZATIONOPTIMIZATION
Reactive Optimization
● Starts after the crash
● Passive
Proactive Optimization
● Starts before the crash
● Active
10
ACCESS LOCALITYACCESS LOCALITY
Temporal locality
● Repeated data access within small time
● Request Based Optimization
Spatial locality
● Clustered data access within small storage areas
● Zone Based Optimization
Access Locality
Temporal locality Spatial locality
11
CHARACTERISTICS OF IDOCHARACTERISTICS OF IDO
● Proactive Zone Based Optimization
● Temporal Locality
● Spatial Locality
● User I/O
● Background I/O
12
DESIGN OF IDODESIGN OF IDO
MAIN THREE OBJECTIVES
● Improving the storage availibility
● Improving the I/O performance
● Providing high portality
13
IDO ARCHITECTUREIDO ARCHITECTURE
14
IDO FUNCTIONAL MODULESIDO FUNCTIONAL MODULES
Hot Zone Identifier
Data Migrator
Request Distributor
Task Predictor
DataReclaimer
● Hot Zone Identification
● Task Prediction
● Request Distribution
● Data Migration
● Data Reclamation
15
KEY DATA STRUCTURESKEY DATA STRUCTURES
ZONE_TABLE
● Num
● Popularity
● Flag
D_MAP
● D_offset
● S_offset
● Len
16
HOT DATA IDENTIFICATIONHOT DATA IDENTIFICATION
THREE DESIGN ISSUES
● By exploiting the spatial locality of workloads
● By exploiting the temporal locality of requests
● By implementing intelligent modules datastructures
17
PROACTIVE DATA MIGRATIONPROACTIVE DATA MIGRATION
● Hot zone identified
● Task Predictor detects task
● Data Migrated
● Flag set to 01
● RAID Reconstructed
● Flag set to 10
● RAID Reclaimed
● Corresponding D_map deleted
18
IMPROVED STORAGE AVAILIBILITY FOR I/OIMPROVED STORAGE AVAILIBILITY FOR I/O
I/O read request
● IDO determines target data zone
● Read request issued to degraded/surrogate RAID set
● Popularity updated
● IDO checks D_map
I/O write request
● Checks D_map for write request hits
● D_map updated
● Sequentially written to surrogate RAID set
19
DATA CONSISTENCYDATA CONSISTENCY
TWO ASPECTS CONSIDERED
● Key data structures
● Redirected write data on surrogate RAID
20
ANALYSISANALYSIS
Overhead Analysis
● Performance Overhead
● Memory Overhead
21
CONCLUSIONCONCLUSION
● Proactive optimisation accelerates low priority background tasks
● Zone Based approach boosts the performance of low priority background tasks
● Designed and implemented a proactive zone based optimisation to outsource data
22
REFERENCESREFERENCES
● S. Wu, H. Jiang, D. Feng, L. Tian, and B. Mao. Proactive Data Migration for Improved Storage Availability in Large-Scale Data Centers. IEEE Transactions on Computers, 2015.
● S. Wu, H. Jiang, D. Feng, L. Tian, and B. Mao. Improving Availability of RAID-Structured Storage Systems by Workload Outsoucing. IEEE Transactions on Computers, 2011.
● S. Wu, B. Mao, D. Feng, and J. Chen. Availability-Aware Cache Management with Improved RAID Reconstruction Performance. In CSE’10, Dec. 2010.
● L. Xiang, Y. Xu, John C. S. Lui, and Q. Chang. Optimal Recovery of Single Disk Failure in RDP Code Storage Systems. In SIGMETRICS’10, Jun. 2010.
23
THANK YOU!!!THANK YOU!!!