StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama

StatusQuo: Making Familiar Abstractions Perform Using Program Analysis

Alvin Cheung Samuel Madden

Armando Solar-Lezama MIT

Owen Arden Andrew C. Myers

Cornell

Developing Database Applications

1/8/2013 CIDR '13 2

Application Server SQL Database

Java Application Logic

SQL Query


1/8/2013 CIDR '13 3

SQL Database


PL/SQL Stored Procedures

SQL Query

Application Server


1/8/2013 CIDR '13 4

SQL Database



SQL Query

Application Server

Language Choice for Application Logic

Application Distribution

Program analysis to the rescue!

StatusQuo

•  Express application logic in ways that programmers are comfortable with

•  Job of compiler & runtime to determine the most efficient implementation

1/8/2013 CIDR '13 5

Two Key Technologies

•  Infer queries from imperative code

•  Migrate computation between servers for optimal performance

1/8/2013 CIDR '13 6

Relational Operations in Imperative Code

1/8/2013 CIDR '13 7

List getUsersWithRoles () { List users = getUsersFromDB(); List roles = getRolesFromDB();

List results = new ArrayList();

for (User u : users) {

for (Role r : roles) {

if (u.roleId == r.id)

results.add(u); }}

return results; }

SELECT * FROM user

SELECT * FROM role

List getUsersWithRoles () {

return executeQuery(

“SELECT u FROM users u, roles r

WHERE u.roleId == r.id

ORDER BY u.roleId, r.id”; }

convert to

Relational Operations in Imperative Code

1/8/2013 CIDR '13 8

List getUsersWithRoles () { List users = getUsersFromDB(); List roles = getRolesFromDB();

List results = new ArrayList();




results.add(u); }}

return results; }

List getUsersWithRoles () {

return executeQuery(

“SELECT u FROM users u, roles r

WHERE u.roleId == r.id

ORDER BY u.roleId, r.id”; }

convert to

Goal

Find a variable that we can rewrite into a

SQL expression

post-condition variable results

Query By Synthesis (QBS)

•  Identify potential code fragments –  i.e., regions of code that fetches persistent

data and return values

•  Find SQL expressions for post-condition variables

•  Try to prove that those expressions preserve program semantics –  if so, convert the code!

1/8/2013 CIDR '13 9

Initial Code Fragments Identification

•  Find program points that retrieve

persistent data

•  Run an inter-procedural analysis that: – determine where persistent data are used – delimit code fragment to analyze

1/8/2013 CIDR '13 10

Search for Post-Condition Expressions

1/8/2013 CIDR '13 11

List getUsersWithRoles () { List users = query(select * from users); List roles = query(select * from roles);

List results = [];




results = results : [] }}

return results; }

Relations involved: users, roles Possible expressions to consider for results: σf(users) topf(users) πf(users ⨝g roles) πf(σg(users) ⨝h roles) other expressions involving users, roles

Infinite search space size!

users roles

results

Constraints for Post-Condition Expressions

1/8/2013 CIDR '13 12

List getUsersWithRoles () { List users = query(select * from users); List roles = query(select * from roles);

List results = [];




results = results : [] }}

return results; }

users roles

results

Hoare-style program verification

If

post-condition expression

outer loop invariant

outer loop invariant outer loop terminates

then post-condition expression

is true and

is true

Still need a smarter way to search

results = πuser( users ⨝roleId = id roles )

results = πuser( users[0 .. i] ⨝roleId = id roles )

Search for Post-Condition Expressions and Invariants

•  Use program synthesis as search engine

1/8/2013 CIDR '13 13

Program synthesizer

Symbolic desc. of search space

Solution constraints

Expression that satisfies all the constraints

Symbolic manipulation

Counter-example driven search

Experiments

1/8/2013 CIDR '13 14

Real-world Evaluation

1/8/2013 CIDR '13 15

Wilos (project management application) – 62k LOC

Operation type # Fragments found

# Fragments converted

Projection 1 1

Selection 13 10

Join 7 7

Aggregation 11 10

Total 33 28

100

1K

10K

100K

1000K

0 20K 40K 60K 80K 100K

Exec

uti

on

tim

e (m

s)

Number of roles / users in DB

original (lazy)

inferred (lazy)

Performance Evaluation: Join Query

1/8/2013 CIDR '13 16

Nested-loop join Hash join! O(n2) O(n)


1/8/2013 CIDR '13 17

SQL Database



SQL Query

Application Server

Application Distribution

Running Example

discount = executeQuery("select discount from customers where id = " + cid);

totalAmount = orderTotal * (1 – discount);

credit = executeQuery("select credit from customers where id = " + cid);

if (credit < totalAmount)

printToConsole("Only " + credit + " in account!");

else

executeUpdate("update customer set credit = " + (credit – totalAmount) + " where id = " + cid);

1/8/2013 18 CIDR '13

Actual Execution






else


1/8/2013 19 CIDR '13

DB

APP

APP

DB

DB

Actual Execution






else


1/8/2013 20 CIDR '13

network communication




DB

APP

APP

DB

DB

Speeding up Execution






else


1/8/2013 21 CIDR '13

DB

APP

DB







else


1/8/2013 22 CIDR '13

data dependency DB

APP

DB

control dependency







else


1/8/2013 23 CIDR '13

DB Server

DB

APP

DB

control dependency

data dependency

Introducing Pyxis

•  “Store-procedurizes” DB apps and pushes computation to the DB

•  Adaptively controls the amount of computation pushed to DB for optimal performance

•  No programmer intervention required

1/8/2013 24 CIDR '13

Using Pyxis

1/8/2013 CIDR '13 25

How Pyxis Works

1/8/2013 CIDR '13 26

Instrument

Partition

Monitor

App Server DB Server

Deploy

Java SQL Java SQL Java

Java

Java

SQL Java SQL

SQL Java

Java SQL Java SQL Java SQL Java

SQL

Java

control transfer

How Pyxis Works

1/8/2013 CIDR '13 27

Monitor

App Server DB Server

Deploy Java

Java

SQL Java

SQL

SQL

Java

control transfer

Instrument

Partition



Generating Program Partitions

•  Deploy and profile application as-is •  Construct a dependence graph of

program statements – captures both control and data flow

•  Formulate linear program from profile data and dependence graph – solution gives a partitioning of the source

code

1/8/2013 CIDR '13 28

Executing Partitioned Programs

•  Pyxis compiler translates partitioned code into standard Java code

•  Pyxis runtime executes compiled Java code – runtime is just another Java program

running on a standard JVM –  includes monitoring component to

determine partition switching

1/8/2013 CIDR '13 29

Experiments

1/8/2013 CIDR '13 30

Experiment Setup

•  TPC-C Java implementation – 20 terminals issuing new order transactions – DB server has 16 cores total

– Compared against two implementations:

•  JDBC: everything on app server except for JDBC stmts •  Manual: custom “store procedurized”

implementation where everything is on the DB server

1/8/2013 31 CIDR '13

5

10

15

20

25

100 300 500 700 900 1100 1300

Ave

rage

Lat

ency

(ms)

Average Thruput (xact / s)

JDBC

Manual

Pyxis

All Cores Available

1/8/2013 32 CIDR '13

Pyxis generated implementation: 3x latency reduction 1.7x thruput increase

StatusQuo

Ease DB application development

Convert imperative program statements into declarative SQL

Fully automatic code partitioning using

application and server characteristics

db.csail.mit.edu/statusquo

1/8/2013 CIDR '13 33

Documents

StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama