Scheduling and Guided Search for Cloud Applications Cristiana
Amza
Slide 2
Big Data is Here Data growth (by 2015) = 100x in ten years [IDC
2012] Population growth = 10% in ten years Monetizing data for
commerce, health, science, services, . [source: Economist] courtesy
of Babak Falsafi
Slide 3
Data Growing Faster than Technology WinterCorp Survey,
www.wintercorp.com Growing technology gap courtesy of Babak
Falsafi
Slide 4
Challenge 1: Costs of a datacenter 4 Estimated costs of
datacenter: 46,000 servers $3,500,000 per month to run Data
courtesy of James Hamilton [SIGMOD11 Keynote] 3yr server and 10yr
infrastructure amortization Server & Power are 88% of total
cost Server & Power are 88% of total cost
Slide 5
Datacenter Energy Not Sustainable Modern datacenters 20 MW! In
modern world, 6% of all electricity, growing at >20%! Billion
Kilowatt hour/year 2001 2005 2009 2013 2017 A Modern Datacenter 17x
football stadium, $3 billion 50 million homes courtesy of Babak
Falsafi
Slide 6
Amazon Cant Recover All Its Cloud Data From Outage Max Eddy, 27
April 2011, www.geekosystem.com When the Cloud Fails: T-Mobile,
Microsoft Lose Sidekick Customer Data Om Malik, 10 October 2009,
gigaom.com Whoops Facebook loses 1 billion photos Chris Keall, 10
March 2009, The National Business Review Cloud Storage Often
Results in Data Loss Chad Brooks, 10 October 2011,
www.businessnewsdaily.com 6 Cloudy with a chance of failure
courtesy of Haryadi S. Gunawi Challenge 2: Data Management
(Anomalies)
Slide 7
Problems are entrenched I have been working in this area since
2001 Problems have only grown more complex/intractable Same old
Distributed Systems problems New: Levels of indirection (remote
processing, deep software stacks, VMs, etc) Eg: Cloud monitoring
and logging data (terrabytes per day) But, no notable success
stories with analyzing such data 7
Slide 8
8 Reduce Map MapReduce parallelism : Embarrassing/simplistic
Works for aggregate op Simple scheduling Challenge 3: Paradigm
Limitations
Slide 9
9 Hadoop/Enterprise: Separate Storage Silos Hardware $$$
Periodic data ingest Cross-silo data management $$$ Hadoop
Slide 10
What can we do ? Find Meaningful Apps We can produce/find tons
of data Need to analyze something of vital importance to justify
draining vital resources Otherwise the simplest solution is to stop
creating the problem(s) 10
Slide 11
What can we do ? Consolidate Research Agendas Find overarching,
mission critical paradigms State of the art: MapReduce too
simplistic Develop standards, common tools and benchmarks Integrate
solutions, think holistically Enforce accountability for data
center/Cloud provider 11
Slide 12
Opportunity 1: The Brain Challenge Started to explore
Neuroscience workloads in 2010 A Brain Summit/Workshop held at IBM
TJ Watson Started a collaboration with Stephen Strother at Baycrest
a year later An application that is both data and compute intensive
Boils down to an optimization problem in a highly parametrized
search space 12
Slide 13
Opportunity 2: Guided Modeling Performance modeling, energy
modeling, anomaly modeling, biophysical modeling All tend to be
interpolations/searches/optimizations in highly parametrized spaces
Key idea: Develop a common framework that works for all Extend the
way MapReduce standardized aggregation ops Guidance: Operator
Reduction, Linear Interpolation, etc 13
Slide 14
Building models takes time High Latency Low Latency DB Memory
Storage Memory Avg. Latency 14 32 data points 32x32= 102 4 sampling
points Actuate a live system and take experimental samples. Sample
in 512MB chunks; 15 minutes for each point 16GB Exhaustive sampling
takes 11 days!
Slide 15
Goal: Reduce Time by Model Reuse High Latency Low Latency DB
Resources Storage Resources Avg. Latency Dynamic Resource
Allocation [FAST09] Capacity Planning [SIGMOD13] What-if Queries
[SIGMETRICS10] Anomaly Detection [SIGMETRICS10] Towards A Virtual
Brain Model [ HPCS14 ] 15 Provide resource-to-performance mapping
More Less
Slide 16
Use less resources: Customer wants 1000 TPS. What is the most
efficient (e.g., CPU/Memory) to deliver it? Share resources: Can I
place customer As DB along side customer Bs DB? Will their
service-levels be met? Service Provider Management Interactions 16
Use the right amount of resources: What will be the performance
(e.g., query latency) if I use 8GB of RAM instead of 16GB? Solve
performance problems: Im only getting 500 TPS. Whats wrong? Is the
cloud to blame? Customer DBA Need to build performance models to
understand
Slide 17
Libraries/archive of models 17 Black-box Models Minimal
assumptions Needs lots of samples Could over-fit Analytical Models
No samples required Difficult to derive Fragile to maintain
Gray-box Models Few samples needed Can be adapted Still need to
derive Data driven Knowledge driven Use an Ensemble of models
Slide 18
Model Ensemble approach 18 1. Guidance as trends and patterns y
x y x y x Use data 2. Automatically tune the models Test & Rank
3. Rank the models use a blend y x y x y x y x Repeat (if needed) 1
2
Slide 19
How to specify guidance 19 SelfTalk Language to describe
relationships Provide a catalog of common functions Details in
SIGMETRICS10 paper y x Specifies model inputs and parameter s
Curve- fitting and validation algorithms
Slide 20
HINT myHint RELATION LINEAR(x,y) METRIC (x,y) {
x.name=MySQL.CPU y.name=MySQL.QPS } CONTEXT (a) {
a.name=MySQL.BufPoolAlloc a.value >= 512MB } Refine models using
data 20 Use hints to link relations to metrics y x Hints that CPU
linearly correlated to QPS Learns parameters using data (or
requests more data) But working-set should be in RAM
Slide 21
Rank models and blend High Latency DB Memory Storage Memory
Avg. Latency 21 16GB 1. Divide search space into regions y y y 3.
Associate best-model to region 2. n-fold cross- validation to
rank
Slide 22
Prototype 22 SelfTalk, Catalog of models, data MySQL &
Storage Server How should I partition resources between two
applications A and B?
Slide 23
Runtime Engine 23 Model Matching Model Repository model new
workload reuse similar data/models selective, iterative and
ensemble learning process Expand Samples Model Validation and
Refinement refine if necessary
Slide 24
Ex 1: Predicting buffer pool latencies 24
Slide 25
Ex 2: Model Transformation CacheL1L2L3 Core i7 256 kB1024
kB8192 kB
Slide 26
Guidance: Step function (3D to 2D reduction) L1L2L3 Xeon 256
kB2048 kB20480 kB Core i7 256 kB1024 kB8192 kB
Slide 27
With minimum new samples CacheL1L2L3 Xeon 256 kB2048 kB20480
kB
Slide 28
Ex 3: Modeling and Job Scheduling for Brain Data centers
usually have heterogeneous structure variety of multicores, GPUs,
etc. Different stages of application have different resource
demands (CPU versus data intensive) Job scheduling to available
resources becomes non-trivial Guided modeling helps 28
Slide 29
Functional MRI Goal Studying brain functionality Procedure
Asking patients (subjects) to do a task and capturing brain slices
measuring blood oxygen level. Correlating images to identify brain
activity 29
Slide 30
Functional MRI Overall Pipeline Data Acquisition 30
Slide 31
Functional MRI 31
Slide 32
NPAIRS As Our Application NPAIRS Goal: Processing images to
find images correlations Feature Extraction: A common technique in
image processing applications (e.g. Face Recognition) Using
Principal Component Analysis to extract Eigen Vectors Finding a set
of Eigen Vectors which is a good representative for the whole set
of subjects Machine Learning Methods, Heuristic Search, etc.
32
Slide 33
NPAIRS SPLIT-HALF 2 FULL DATA Scans Design Data Scans
Statistical Parametric Map SJ2 SPM SJ1 Design Data SPLIT-HALF 1
REPRODUCIBILITY ESTIMATE (r) v SJ2 v SJ1 Split J 33
Slide 34
Output of NPAIRS 34
Slide 35
NPAIRS Flowchart 35
Slide 36
NPAIRS Profiling Results 36
Slide 37
GPU Execution Profile 37
Slide 38
NPAIRS Execution on Different Nodes 38
Slide 39
Job Modeling: Exhaustive Sampling 39 Sample Set: 1 to 99
Fitness Score R 2 : 0.995 Total Run Time: 64933
Slide 40
Uniform Sampling 40 Sample Set: 2 12 22 32 42 52 62 72 82 92
Using 5-fold cross validation Fitness Score R 2 : 0.990 Total Run
Time: 6368
Slide 41
Guidance: Step function + Fast Sampling 41 Sample Set: 2 4 8 12
16 20 24 32 48 96 Fitness Score R 2 : 0.993 Total Run Time: 5313
16.6% Time Saving!
Slide 42
Heterogeneous CPU Only (3Fat + 5Light Nodes) 42
Slide 43
3 Fat, 3 Light, 3 GPU nodes 43
Slide 44
Resource Utilization 44
Slide 45
Overall Execution Time Comparison 45
Slide 46
Conclusions Big Data processing is driving a quantum leap in IT
Hampered by slow progress in data center management We propose to
investigate guided modeling Promising preliminary results with
Neuroscience workloads 7x speedup of NPAIRS on small CPU+GPU
cluster 46
Slide 47
Backup Slides 47
Slide 48
Modeling Procedure Get sample set and split it using 5-fold
cross validation Fit the model using 4 folds sample data Test the
model using 1 fold sample data Try 5 type of splits, and sum the
model error. If the error is less than the threshold, stop. We
found the model. 48
Slide 49
Notes Total run time is the summary of total sampling time.
Modeling time is negligible. Use the exhaustive data set as the
true value, and the fitted model to predict the value. Then compute
the coefficient of determination R 2. 49