Guaranteed Performance While Querying Ever-Growing Data

Guaranteed Performance While Querying Ever-Growing Data

Michael ArmbrustBEARS Conference – February 2012

P QL

Web Applications Grow Rapidly

Success Disaster?

• Force developers to use simple operations (get/put)

• Makes complexity obvious

• Force developers to use simple operations (get/put)

• No optimization or data independence

PIQL Solution

P QL

Performance Insightful Query Language

• SQL-like• Builds on existing

scalable storage• Guaranteed

performance independent of scale

Problem with Cost Based OptimizationPlan Choices

Sequential Scan

Random Lookups

BIG DATA: The State of the Art

Algorithms

Machines

People

search

Watson/IBM

8

AMP: A Holistic Approach

search

Watson/IBM

Machines

People

Algorithms

9

BDAS: Berkeley Data Analysis System A Top-to-Bottom Rethinking of the big data analytics stack integratingAlgorithms, Machines, and People

Infra. Builder

Algo/Tools

Data Collector

Data Analyst

Higher Query Languages / Processing Frameworks

Resource Management

StorageData

CollectorCrowd

Interface

Analytics Libraries, Data Integration

Data Source Selector

Result Control Center

Visualization

Qua

lity

Co

ntr

ol

Mo

nito

ring

/Deb

ug

gin

g Poster Session@amplab465 SODA Hall