Towards Shape Analysis for Device Drivers Hongseok Yang (Queen Mary, University of London) (Joint...

Towards Shape Analysis for Device

DriversHongseok Yang

(Queen Mary, University of London)

(Joint work with Josh Berdine, Cristiano Calcagno, Byron Cook, Dino Distefano, Peter O’Hearn and Thomas Wies)

Do deep shape analysis for real programs (i.e., programs of ~ 100K LOC).

1. Memory safety2. Absence of leaks3. Race freedom4. Reponsiveness

void XXX_CancelIrp(PDEVICE_OBJECT DeviceObject, PIRP Irp){ ……… PRESET_IRP ResetIrp,temp,tempnext; PDEVICE_EXTENSION de; ……… KeAcquireSpinLock(&de->ResetSpinLock, &Irql); ResetIrp = (PRESET_IRP)de->Flink2; while (ResetIrp !=NULL) { if (ResetIrp->Irp == Irp) { temp = (PRESET_IRP)de; tempnext = temp->Flink2; while (tempnext != ResetIrp) { temp = tempnext; tempnext = temp->Flink2; } temp->Flink2 = ResetIrp->Flink2; free(ResetIrp); break; } else if (ResetIrp->Flink2 == (PRESET_IRP)de) break; else ResetIrp = (PRESET_IRP)ResetIrp->Flink2; } KeReleaseSpinLock(&de->ResetSpinLock, Irql); …… IoCompleteRequest(Irp, IO_NO_INCREMENT);}

void XXX_CancelIrp(PDEVICE_OBJECT DeviceObject, PIRP Irp)

Pb 1: Extreme local analysis• Tied to OS kernel • Expensive shape analysis

KeReleaseSpinLock(&de->ResetSpinLock, Irql);

KeAcquireSpinLock(&de->ResetSpinLock, &Irql);

IoCompleteRequest(Irp, IO_NO_INCREMENT);

void XXX_CancelIrp(PDEVICE_OBJECT DeviceObject, PIRP Irp){ ……… PRESET_IRP ResetIrp,temp,tempnext; PDEVICE_EXTENSION de; ……… KeAcquireSpinLock(&de->ResetSpinLock, &Irql); ResetIrp = (PRESET_IRP)de->Flink2; while (ResetIrp !=NULL) { if (ResetIrp->Irp == Irp) { temp = (PRESET_IRP)de; tempnext = temp->Flink2; while (tempnext != ResetIrp) { temp = tempnext; tempnext = temp->Flink2; } temp->Flink2 = ResetIrp->Flink2; free(ResetIrp); break; } else if (ResetIrp->Flink2 == (PRESET_IRP)de) break; else ResetIrp = (PRESET_IRP)ResetIrp->Flink2; } KeReleaseSpinLock(&de->ResetSpinLock, Irql); …… IoCompleteRequest(Irp, IO_NO_INCREMENT);}

Pb1 : Extreme local analysis• A part of code in isolation.• No preconditions.• Only the memory footprint of the part.

void XXX_CancelIrp(PDEVICE_OBJECT DeviceObject, PIRP Irp){ ……… PRESET_IRP ResetIrp,temp,tempnext; PDEVICE_EXTENSION de; ……… KeAcquireSpinLock(&de->ResetSpinLock, &Irql); ResetIrp = (PRESET_IRP)de->Flink2; while (ResetIrp !=NULL) { if (ResetIrp->Irp == Irp) { temp = (PRESET_IRP)de; tempnext = temp->Flink2; while (tempnext != ResetIrp) { temp = tempnext; tempnext = temp->Flink2; } temp->Flink2 = ResetIrp->Flink2; free(ResetIrp); break; } else if (ResetIrp->Flink2 == (PRESET_IRP)de) break; else ResetIrp = (PRESET_IRP)BusResetIrp->Flink2; } KeReleaseSpinLock(&de->ResetSpinLock, Irql); …… IoCompleteRequest(Irp, IO_NO_INCREMENT);}

Pb 2: Challenging Data Structurestypedef struct {

DEVICE_OBJECT* StackDeviceObject; DEVICE_OBJECT* PortDeviceObject; DEVICE_OBJECT* PhysicalDeviceObject; …ADDRESS_DATA* Flink1; RESET_IRP* Flink2; CROM_DATA* Flink3; DETACH_DATA* Flink4; RESOURCE_DATA* Flink5;} DEVICE_EXTENSION, *PDEVICE_EXTENSION;

typedef struct DEVICE_OBJECT {…DEVICE_EXTENSION* DeviceExtension; … } DEVICE_OBJECT, *PDEVICE_OBJECT;

Drawn by Dino Distefano

Pb 3: Concurrency

KeReleaseSpinLock(&de->ResetSpinLock, Irql);

KeAcquireSpinLock(&de->ResetSpinLock, &Irql);

typedef struct {…DRIVER_CANCEL (*CancelRoutine)(…); …} IRP, *PIRP;

Status of Separation-logic based Shape Analysis

(London/Cambridge)Problem 1: Extreme local analysisExtreme local shape analysis[Calcagno, Distefano, O’Hearn, Yang]

Problem 2: Challenging data structuresAdaptive shape analysis[Berdine, Calcagno, Disetafano, Cook, Wies, O’Hearn, Yang]

Problem 3: ConcurrencyThread-modular shape analysis [Gotsman, Berdine, Cook, Sagiv]

Extreme Local Shape Analysis

• Analyze a part of code in isolation.• No preconditions.• Only the memory footprint of the part.

void XXX_CancelIrp(PDEVICE_OBJECT DeviceObject, PIRP Irp){ ……… PXXX_RESET_IRP XXXResetIrp,temp,tempnext; PDEVICE_EXTENSION deviceExtension; ……… KeAcquireSpinLock(&deviceExtension->ResetSpinLock, &Irql); ResetIrp = (PRESET_IRP)de->Flink2; while (ResetIrp !=NULL) { if (ResetIrp->Irp == Irp) { temp = (PRESET_IRP)de; tempnext = temp->Flink2; while (tempnext != ResetIrp) { temp = tempnext; tempnext = temp->Flink2; } temp->Flink2 = ResetIrp->Flink2; free(ResetIrp); break; } else if (ResetIrp->Flink2 == (PRESET_IRP)de) break; else ResetIrp = (PRESET_IRP)BusResetIrp->Flink2; } KeReleaseSpinLock(&deviceExtension->ResetSpinLock, Irql); …… IoCompleteRequest(Irp, IO_NO_INCREMENT);}

(de aD Flink2: x0) * ls (RESET_IRP,Flink2) (x0,de)

typedef struct { RESET_IRP* Flink2; IRP* Irp;…} RESET_IRP, *PRESET_IRP;

typedef struct { RESET_IRP* Flink2; …} DEVICE_EXTENSION;

(de aD Flink2: x0) * ls (RESET_IRP,Flink2) (x0,x1) * (x1aR Irp:Irp)

(de aD Flink2: x0) * ls (RESET_IRP,Flink2) (x0,0)

(de aD Flink2: x0) * ls (RESET_IRP,Flink2) (x0,de)(de aD Flink2: de)

(de a Flink2: x0) * ls (RESET_IRP,Flink2) (x0,de)

(de a Flink2: x0) * ls (RESET_IRP,Flink2) (x0,de)(de aD Flink2: de)

de aD Flink2: de

de aD Flink2: deÆ de = ResetIrp

ERROR: No IRP Field in DEVICE_EXTENSION

TraceGeneration

FootprintComputation

list t*;while (x!=0) {

t = x;

x = x->next;

free(t);

assert(x!=0);t=x;x=x->next;free(x);assert(x=0);

assert(x!=0);t=x;x=x->next;free(x);assert(x!=0);t=x;x=x->next;free(x);assert(x=0);

TraceGeneration

t = x;

x = x->next;

free(t);

TraceGeneration

t = x;

x = x->next;

free(t);

TraceGeneration

t = x;

x = x->next;

free(t);

Abstract preconditions

TraceGeneration

t = x;

x = x->next;

free(t);

SpaceInvader

{P2,P3,…,Pn}

{Q1,Q2,…,Qk}

{I1,I2,…,Im}

TraceGeneration

SpaceInvader

Sound precondition

Footprint only

Separation Logic

xay, ls (y,z)

xay * ls (y,z), emp

9y’. z!=0 Æ v=a Æ xay’ * ls (y’,z)

x y y z

Variable Convention

• Program variables: x,y,z,t,v,w• Ghost (or auxiliary) variables: a,b,c,d,….• Primed variables: x’,y’,z’,t’,v’,w’

9 w’,w’1.

x!=0 Æ z=a Æ w’!=w’1 Æ xaw’ * ls (w’,w’1) * yaw’1

Symbolic Heaps

Separation logic formulas of the form:

(x!=0 Æ z=a Æ w’!=w’1) Æ (xaw’ * ls (w’,w’1) * yaw’1)

SH = Set of all symbolic heapsGhostSH = Set of sym. heaps with ghost vars

Footprint Computation

Foot : Trace x SH ! GhostSH x SH

Correctness: If Foot(C,P)=(P0,Q0), then {P*P0}C{Q0} holds in Sep.

Log.Foot(t=x->next, x=aÆemp) = (aab, x=aÆt=bÆaab) {x=a Æ emp*aab} t=x->next {x=a Æ t=b Æ aab}

Foot(free(x),P*EaF) = (emp,P) if (P*EaF) ` E=xFoot(free(x), x=a Æ aa0) = (emp, x=a Æ emp)

Foot(free(x),P*EaF) = (emp,P) if (P*EaF) ` E=xFoot(free(x),P) = (aab,P) if P ` x=a, fresh b

Rewrite x to some ghost variable.

Foot(free(x), x=a Æ emp) = (aab, x=a Æ emp)

Foot(free(x),P*EaF) = (emp,P) if (P*EaF) ` E=xFoot(free(x),P) = (aab,P) if P ` x=a, fresh b

Foot(free(x),P) = (false,P) otherwise

Foot(free(x), emp) = (false, emp) != (xa b, emp)

Foot(A;C,P) = let (P0,Q0)=Foot(A,P) and (P1,Q1)=Foot(C,Q0)

in (P0*P1, Q1)

Q1P0 Q0

in (P0*P1, Q1) Foot(free(x);free(y), x=a Æ y=b Æ emp) = (aac * bad, x=a Æ y=b Æ emp)

Algorithm : Given a trace C. Supp FPV(C) = {x}.

P := (x=a Æ emp); (P0,Q0) := Foot(C, P);

return (P * P0);

List Disposal

list t*;while (x!

=0) { t = x; x = x-

>next; free(t);}

List Disposal

list t*;while (x!

=0) { t = x; x = x-

>next; free(t);}

List Disposal

list t*;while (x!

=0) { t = x; x = x-

>next; free(t);}

List Disposal

list t*;while (x!

=0) { t = x; x = x-

>next; free(t);}

List Disposal

list t*;while (x!

=0) { t = x; x = x-

>next; free(t);}

list t*;while (x!

=0) { t = x; x = x-

>next; free(t);}

t = x;

x = x->next;

free(t);

Discovered Precondition: x=a Æ emp

x=a Æ emp

t = x;

x = x->next;

free(t);

Discovered Precondition: x=a Æ a!=0 Æ emp

x=a Æ empx=a Æ a!=0 Æ emp

t = x;

x = x->next;

free(t);

Discovered Precondition: x=a Æ a!=0 Æ emp

x=a Æ a!=0 Æ t=a Æ emp

t = x;

x = x->next;

free(t);

Discovered Precondition: x=a Æ a!=0 Æ emp * aab

x=a Æ a!=0 Æ t=a Æ empx=b Æ a!=0 Æ t=a Æ emp * aab

t = x;

x = x->next;

free(t);

Discovered Precondition: x=a Æ a!=0 Æ emp * aab

x=a Æ a!=0 Æ t=a Æ emp

x=b Æ a!=0 Æ t=a Æ emp

x=b Æ a!=0 Æ t=a Æ emp * aab

t = x;

x = x->next;

free(t);

Discovered Precondition: x=a Æ a!=0 Æ b!=0 Æ emp * aab

x=b Æ a!=0 Æ t=a Æ b!=0 Æ emp

t = x;

x = x->next;

free(t);}

Discovered Precondition: x=a Æ a!=0 Æ b!=0 Æ emp * aab

x=b Æ a!=0 Æ t=a Æ b!=0 Æ empx=b Æ a!=0 Æ t=b Æ b!=0 Æ emp

t = x;

x = x->next;

free(t);

Discovered Precondition: x=a Æ a!=0 Æ b!=0 Æ emp * aab * bac

x=b Æ a!=0 Æ t=a Æ b!=0 Æ empx=b Æ a!=0 Æ t=b Æ b!=0 Æ empx=c Æ a!=0 Æ t=b Æ b!=0 Æ emp * bac

t = x;

x = x->next;

free(t);

Discovered Precondition: x=a Æ a!=0 Æ b!=0 Æ emp * aab * bac

x=b Æ a!=0 Æ t=a Æ b!=0 Æ empx=b Æ a!=0 Æ t=b Æ b!=0 Æ empx=c Æ a!=0 Æ t=b Æ b!=0 Æ emp * bacx=c Æ a!=0 Æ t=b Æ b!=0 Æ emp

t = x;

x = x->next;

free(t);

Discovered Precondition: x=a Æ a!=0 Æ b!=0 Æ c=0 Æ emp * aab * bac

x=c Æ a!= 0 Æ t=b Æ b!=0 Æ c=0 Æ emp

x=b Æ a!=0 Æ t=a Æ b!=0 Æ empx=b Æ a!=0 Æ t=b Æ b!=0 Æ empx=c Æ a!=0 Æ t=b Æ b!=0 Æ emp * bacx=c Æ a!=0 Æ t=b Æ b!=0 Æ emp

Abstraction

Abs : SH ! CanonicalSH

1. Existentially quantify ghost vars.2. Substitute away equalities d’=E.3. Drop disequalities.4. Apply list abstraction.

Abstraction

Discovered Precondition: x=a’ Æ a’!=0 Æ b’!=0 Æ c’=0 Æ emp * a’ab’ * b’ac’

Abstraction

Discovered Precondition: x!=0 Æ b’!=0 Æ emp * xab’ * b’a0

Abstraction

Discovered Precondition:

emp * xab’ * b’a0

Abstraction

emp * xab’ * b’a0

ls (x,0)

SpaceInvader

t = x;

x = x->next;

free(t);

Precondition: ls (x, 0)

Postcondition: x=0 Æ emp

Fixpoint Computation

Backward Footprint Computation

list t*;while (x!

=0) { t = x; x = x-

>next; free(t);}

Backward:

assert(x!=0);t = x;x = x->next;free(t);assert(x=0);

Forward:

assert(x!=0);t = x;x = x->next;free(t);assert(x=0);

Frame Rule and Correctness of Footprint

Computation Foot : Trace x SH ! GhostSH x SH

in (P0*P1, Q1)

Q1P0 Q0

Sound because of Frame Rule in Sep. Log.

{P*P0}A{Q0}

{P*P0*P1}A{Q0*P1} {Q0*P1}C{Q}

{P*P0*P1}A;C{Q}

Experiments with Firewire Device Driver

MacBook, 2GH Intel Core Duo. 2GB Mem. All fn calls are inlined.

Function LOC Time Result

F1 32 0.07 Alarm Real Err

F2 87 0.04 Yes

F3 130 0.43 Yes

F4 198 0.31 Yes

F5 (loop 1) 325 40.75 Yes

F5 (loop 2) 46 2.83 Yes

F5 (loop 3) 221 0.45 Yes

F5 (loop 4) 170 > 10 min

F5 (loop 5) 59 0.036 Yes

Future Direction

1. Weaving analysis results.2. Good abstraction for footprints.3. Better shape abstraction.

Foot(assert(x!=0),P) = (a!=0 Æ emp, a!=0 Æ P) if P ` x=a

Foot(assert(x!=0),P) = (a!=0 Æ emp, a!=0ÆP) if P ` x=a

Foot(assert(x!=0),P) = (false, P) otherwise

Towards Shape Analysis for Device Drivers Hongseok Yang (Queen Mary, University of London) (Joint...

Documents

Variance Analyses from Invariance Analyses Josh Berdine jjb@microsoft.com Microsoft Research, Cambridge Joint work with Aziem Chawdhary, Byron Cook, Dino

White Chocolate MacadamiaNut MM Kristin Miller Berdine Orelien Shannon Blier Eryk Williams Nick Ferraro

by Kyuho Son, Hongseok Kim, Yung Yi, Bhaskar Krishnamachari

Counterexample Generation for Separation-Logic-Based Proofs Arlen Cox Samin Ishtiaq Josh Berdine Christoph Wintersteiger

Separation and Information Hiding Peter W. O’Hearn (Queen Mary, University of London) John C. Reynolds (Carnegie Mellon University) Hongseok Yang (Seoul

Calcagno 2000 Fullness or the Nothing. Stein and Heidegger on Being

Axiomatic Methods for Software Verification Hongseok Yang

GRAN SASSO’S HADRON STOP Temperature’s behaviour under specified beam conditions Barbara Calcagno

Tutorial on Widening (and Narrowing) Hongseok Yang Seoul National University

Replicated Data Types: Specification, Verification, Optimality · 2014. 7. 30. · Replicated Data Types: ! Specification, Verification, Optimality Hongseok Yang! University of

LEEANN EVRON ANDREA CALCAGNO RESTAURANT EQUIPMENT 1

Shape Analysis by Graph Decomposition R. Manevich M. Sagiv Tel Aviv University G. Ramalingam MSR India J. Berdine B. Cook MSR Cambridge

© 2009 Microsoft Corporation. All rights reserved. Automatic Verification of Heap Manipulation using Separation Logic Josh Berdine with thanks to Byron

Computability and Complexity Results for a Spatial ... · Computability and Complexity Results for a Spatial Assertion Language for Data Structures Cristiano Calcagno12, Hongseok

Partially Disjunctive Shape Analysis Josh Berdine Byron Cook MSR Cambridge Tal Lev-Ami Roman Manevich Mooly Sagiv Ran Shaham Tel Aviv University Ganesan

Formal Semantics of Probabilistic Programming Languages: … · 2017-11-20 · Formal Semantics of Probabilistic Programming Languages: Issues, Results and Opportunities Hongseok

Josh Berdine, Cristiano Calcagno, Peter O’Hearn 1 July ...oliveras/espai/smtSlides/peter.pdf · Proof Procedures for Separated Heap Abstractions Josh Berdine, Cristiano Calcagno,

Adaptive Sampling Probabilities for Non-Smooth Optimizationproceedings.mlr.press/v70/namkoong17a/namkoong17a.pdf · Adaptive Sampling Probabilities for Non-Smooth Optimization Hongseok

WiMax 03-08 Calcagno

Finding Optimum Abstractions in Parametric Dataflow Analysis Xin Zhang Georgia Tech Mayur Naik Georgia Tech Hongseok Yang University of Oxford