Randomized Accuracy Aware Program Transformations for Efficient Approximate Computations

Randomized Accuracy Aware Program Transformations for Efficient Approximate Computations

Sasa MisailovicJoint work with

Zeyuan Allen Zhu Jonathan Kelner Martin Rinard

MIT CSAIL

…

… … … …

• Nodes represent computation• Edges represent flow of data

…

• Functions – process individual data• Reduction nodes – aggregate data

… … … …

…

min

avgavg avgavg

• Functions – process individual data• Reduction nodes – aggregate data

… … … …

…

min

avgavg avgavg

Function substitution• Multiple implementations• Each has expected error/time

f2 f3f1

… … … …

…

min

avgavg avgavg

Function substitution• Multiple implementations• Each has expected error/time

… … … …

Sampling inputs of reduction nodes• Reductions consume fewer inputs

…

min

avgavg avgavg

… … … …

…

min

avgavg

Sampling inputs of reduction nodes• Reductions consume fewer inputs

… … … …

Tradeoff Space

Error

Time

Tradeoff Space

Error

Time

Optimal Tradeoff Curve

Error

Time Using the tradeoff curve:• Minimize time subject to error bound• Minimize error subject to time bound

Our Result

Analysis Optimization

Original program

Error bound

Optimizedprogram

Transformations

• Randomized computation• Guaranteed expected error/time tradeoff• -approximation of optimal tradeoff

Outline

Model of Computation

Tradeoff Curve Construction

Optimized Program Selection

Related Work

Model of Computation

…

f f

t

u v

w

t

u v

w

ff ggg g… … … …

min

avg avg avg avg

Model of Computation

…

f f

t

u v

w

t

u v

w

ff ggg g… … … …

min

avg avg avg avg

min

1

n

n n

f g

t

u v

w

avg avgm m

Structure of Computation

• Computation nodes DAGs of functions Functions: arbitrary code Process individual inputs

• Reduction nodes Aggregation functions Average, min, max, sum…

• Computation Tree Computation nodes and

reduction nodesmin

1

n

n n

f g

t

u v

w

avg avgm m

min

Accuracy-Aware Transformations

Function substitution• Multiple versions • Execute with probability • Each has error/time spec

Reduction sampling• Consume inputs

Probability of selecting eachinput:

• Derived error/time specifications

Average:

Min/max: 1

n

n n

avg avgm m

min

Accuracy-Aware Transformations

Function substitution• Multiple versions • Execute with probability • Each has error/time spec

Reduction sampling• Consume inputs

Probability of selecting eachinput:

• Derived error/time specifications

Average:

Min/max: 1

n

n n

avg avgm m…

avg

min

Accuracy-Aware Transformations

Function substitution• Multiple versions • Execute with probability • Each has error/time spec

Reduction sampling• Consume inputs

Probability of selecting eachinput:

• Derived error/time specifications

Average:

Min/max: 1

n

n n

avg avgm m…

avg

min

Accuracy-Aware Transformations

Function substitution• Multiple versions • Execute with probability • Each has error/time spec

Reduction sampling• Consume inputs

Probability of selecting eachinput:

• Derived error/time specifications

Average:

Min/max: 1

n

n n

avg avgm m…

avg

min

Program Configuration Vector

Defines transformed program • Functions: probability of

executing each version• Reductions: number of

elements to sample

1

n

n n

avg avgm m

…0.3 0.6 0.1 …

min

Configuration Vector

• Specifies program version• Functions: probability of

executing each version• Reductions: number of

elements to sample

1

n

n n

avg avgm m

…0.3 0.6 0.1 …

Find optimal program=

Find configuration vector that achieves optimal accuracy vs. performance tradeoff

min

1

n

n n

avg avgm m

Divide and conquer• For each subcomputation

construct tradeoff curve• Dynamic programming

Properties• Polynomial time• -approximation of

true tradeoff curve

Tradeoff Curve Construction: Algorithm

min

Tradeoff Curve Construction: Algorithm

1

n

n n

avg avgm m

Tradeoff Curve Construction: Algorithm

1

m m

n

n n

min

avg avg

Tradeoff Curve Construction: Algorithm

1

m m

n

n n

min

avg avg

Tradeoff Curve Construction: Algorithm

1

n

n n

min

avgm

Tradeoff Curve Construction: Algorithm

1

n

n n

min

Tradeoff Curve Construction: Algorithm

1

n

n n

min

Tradeoff Curve Construction: Algorithm

1

n

n n

min

Tradeoff Curve Construction: Algorithm

1

n

min

Tradeoff Curve Construction: Algorithm

Computation Node Optimization

Variables: • Probability to execute each version of • Range: • Sum:

(E2,T2)(E0,T0) (E1,T1)

Linear program

Computation Node Optimization

Variables: • Probability to execute each version of • Range: • Sum:

Objective:

Constraint:(E2,T2)(E0,T0) (E1,T1)

Linear program

Computation Node Optimization

Variables: • Probability to execute each version of • Range: • Sum:

Objective:

Constraint:(E2,T2)(E0,T0) (E1,T1)

Linear program

Computation Node Optimization

Variables: • Probability to execute each version of • Range: • Sum:

Objective:

Constraint:(E2,T2)(E0,T0) (E1,T1)

Linear program

The Algorithm: Reduction Nodes

Given error bound , find number of elements to sample

m

avg

The Algorithm: Reduction Nodes

Given error bound , find number of elements to sample

s.t.

m

avg

(𝐸𝑠𝑢𝑏 ,𝑇 𝑠𝑢𝑏)

The Algorithm: Reduction Nodes

Given error bound , find number of elements to sample

s.t.

From approximate tradeoff curve:

for

m

avg

(𝐸𝑠𝑢𝑏 ,𝑇 𝑠𝑢𝑏)

The Algorithm: Reduction Nodes

Given error bound , find number of elements to sample

m

avg

(𝐸𝑠𝑢𝑏 ,𝑇 𝑠𝑢𝑏)

s.t.

• Univariate optimization problem• Analogously, minimize error subject to

Approximate Tradeoff Curve

Error

Time

Bidimensional Discretization

Take elements at regular intervals

Approximating Tradeoff Curve

Error

Time

Bidimensional Discretization

Approximating Tradeoff Curve

Error

Time

Approximating Tradeoff Curve

Randomized configuration:• Execute with probability • Execute with probability

Error

Time

𝜅 1−𝜅

𝐴

𝐵

𝐶

Approximating Tradeoff Curve

Error

Time

�̂�𝐸𝜀𝐸

𝑇

Approximating Tradeoff Curve

Error

Time

𝑇

�̂�

𝑇

T

Properties of the Algorithm

Performance• Number of tradeoff curve points: • Most expensive operation: bidimensional discretization

Calling LP solver times Each call can have variables

Precision• Precision decreases linearly with the number of nodes • To obtain -approximation set intermediate

Space• Storing tradeoff curves:

Obtaining Optimized Programs

Tradeoff curves for all subcomputations:• Each curve contains partial configuration

Probability of executing local function nodes Number of inputs to sample from reduction node Error tolerated by subcomputation

• Distribution over optimal program configurations

Incrementally construct configuration vector:• For every execution• Traverse the tree, starting from root• Time to get full vector:

Related WorkAccuracy-aware transformations• Empirical justification: training/test input set

[Rinard ICS ‘06, Rinard OOPSLA ’07, Ansel et al. PLDI ‘09, Misailovic et al. ICSE ’10, Baek & Chilimbi PLDI ‘10 Hoffmann et al. ASPLOS ‘11, Sidiroglou et al. FSE ‘11]

• Probabilistic accuracy analysis for loop perforation [Misailovic et al. SAS ‘11, Chaudhuri et al. FSE ‘11]

Related WorkAccuracy-aware transformations• Empirical justification: training/test input set

[Rinard ICS ‘06, Rinard OOPSLA ’07, Ansel et al. PLDI ‘09, Misailovic et al. ICSE ’10, Baek & Chilimbi PLDI ‘10 Hoffmann et al. ASPLOS ‘11, Sidiroglou et al. FSE ‘11]

• Probabilistic accuracy analysis for loop perforation [Misailovic et al. SAS ‘11, Chaudhuri et al. FSE ‘11]

Ensuring safety of transformed programs• Separating critical and approximate parts of program

[Carbin & Rinard ISSTA ’10, Sampson et al. PLDI ’11]

• Verifying relaxed semantics of programs [Carbin et al. CSAIL-TR ‘11]

Analytic properties of programs [Majumdar & Saha RTSS ‘09, Chaudhuri et al. POPL’10, Ivancic et al. MEMOCODE ‘10 , Reed & Pierce ICFP ’10, Chaudhuri & Solar-Lezama PLDI ’10 , Chaudhuri et al. FSE ’11]

Summary

Model of Computation• Accuracy-aware program transformations• Effects on overall accuracy and execution time

Explore and Exploit Optimal Tradeoffs• Approximate optimal tradeoff curve construction• Polynomial, dynamic programming algorithm• Randomized program configurations to achieve tradeoffs

Envisioned Applications• Image and video processing, numerical algorithms,

queries on big data sets, machine learning, …• Optimization, fault tolerance, dynamic adaptation