The Science of Deep Specificationbcpierce/papers/deepspec-hcss... · 2016-05-11 · The Science of...

TheScienceofDeepSpecification

AndrewW.Appel,BenjaminPierce,StephanieWeirich,SteveZdancewic,Zhong Shao,AdamChlipala

Princeton Penn Yale MIT

Zero-vulnerabilitycriticalsoftware– Compilers,interpreters– Operatingsystems– Filesystems,networkingstacks– Distributedmiddleware– Databases– Crypto,securityprotocols

Apipedream?

Ahigh-value“niche”

Maybeuntilrecently!

Heroicproofsofconcept• CompCert (Ccompiler)• L4.verified(OS)

Proliferationof“pointsolutions”• CertiKos (hypervisor)

• Verdi(distributedalgorithmstoolkit)• RockSalt (softwarefaultisolation)• CakeML (MLcompiler)

• VeLLVM (LLVMoptimizations)

• HMAC+SHA(crypto)• …

Individuallyimpressive!Butdisconnected

TheRiseofIntegratedStacks• CompCert ecosystem• L4.verifiedecosystem• IronClad Apps• Bedrockwebserver• Everest(verifiedhttps)• … Whatmakesthischallenging?

(lotsofthings,but inparticular…)

SpecificationEngineering!

WhatwelearnedfromCompCert

6

Clanguage

CompCertCompiler

PowerPCISA

ProgramLogic

VerifiableCSystem

Clanguage

IBM’sCPU

Transistors

PowerPCISA

OSclientinterface

CertiKOShypervisorkernel

Clanguage AppelShao

PeterSewellUniv.ofCambridge

XavierLeroyInria

Whatwediscovered ...

7

CompCertCompiler

PowerPCISA

Clanguage

IBM’sCPU

Transistors

PowerPCISA

ProgramLogic

VerifiableCSystem

Clanguage

OSclientinterface

CertiKOShypervisorkernel

Clanguage AppelShao

PeterSewellUniv.ofCambridge

XavierLeroyInria

Solution:exercisespec.frombothsides(2006-2015)

8

CompCertCompiler

IBM’sCPU

Transistors

ProgramLogic

VerifiableCSystem

Clanguage

OSclientinterface

CertiKOShypervisorkernel

Clanguage

XavierLeroyINRIA

AppelShao

PeterSewellUniv.ofCambridge

Clanguage

PowerPCISA

PowerPCISA

Integration!

TheFutureofFormalMethods…

^

TheScienceofDeepSpecification

AnewNSFExpedition…

11

AndrewAppel

ZhongShao

StephanieWeirich

BenjaminPierce

SteveZdancewic

AdamChlipala

Princeton Penn YaleMIT

$10m5years

DeepSpecificationsareFORMAL,RICH,LIVE,and2-SIDED

12

RICH describecomplexbehaviorsindetail

FORMAL innotationwithaclearsemantics

LIVEmachine-checkedconnectiontoimplementations

2-SIDEDconnectedtobothimplementations&clients

DeepSpec goals1. Coreresearch2. Education3. Communitybuilding

13

14

CoreResearchTopics

Individualprojects,connectedatdeepspecs

15AdamChlipala

Verifiedprocessordesign

Individualprojects,connectedatdeepspecs

16

AndrewAppel

VerifiedtoolchainforverifyingconcurrentCprograms

Individualprojects,connectedatdeepspecs

17

SteveZdancewic

VerifiedLLVMcompiler

Individualprojects,connectedatdeepspecs

18

StephanieWeirich

Specificationoffunctionallanguage

Individualprojects,connectedatdeepspecs

19

ZhongShao

VerifiedhypervisorOSkernel

Individualprojects,connectedatdeepspecs

20

BenjaminPierce

Specification-basedrandom

testing

SpecificationandtestingPromisingdevelopment:Theriseofspecification-basedautomatedtestingtechniques– Property-based random testing(QuickCheck)– Model-based testing– Oracle-based testing– …

End-to-EndDemo(s)Leadingcandidates:– Votingsystems– Automotive software– Datacenterinfrastructure

22

Othersuggestions??

Educationandtraining

23

Textbooksandon-linematerialsSoftwareFoundationstextisusedatdozensofuniversities.Nowweknow:

Withgoodinstructionalmaterialsandinteractiveproofcheckers,specification&verificationcanbetaught…

...justlikeprogrammingandsoftwareengineeringcanbetaught!

BookDevelopment• Goal:UseSoftwareFoundations toseedanewseriesof“verifiedtextbooks”

• Firststep:

• Later:– Averifiedcompiler textbook?

24

VerifiedFunctionalAlgorithms

AndrewAppel

(fall2016)

CurriculumDevelopment

• Modularity⇒cleanpedagogical implementations• Precise(andcorrect!)descriptionofrelevantabstractions• Specifications⇒automatedtestharnesses/testcases/property-basedtesting(forgrading)

• Connects toformalmethodscoursethatteachesverificationtechniquesfortheseartifacts

25

NewCompiler&OSCoursesbasedon

CertiKOS

EducationSpecialistsBruceLenthallExecutiveDirectorPennCTL(CenterforTeaching&Learning)

EmilyElliottAssociateDirector,PennCTL

AnandaGunawardenaLecturer,PrincetonCS

Responsibilities:• determineappropriatemetricsforlearningoutcomes

• designassessmentplan• developdatacollectionplans

• helpdesignmeasurementinstruments

• analyzedata• workwithIRBs

Responsibilities:• manageimplementationofdatacollectionplan

• sendout,collect,andcompileassessments

• etc.

Assessmenttools1. ABETcourseoutcomes– Compare“pre-DS”to“DS-ified”versions of

courseatthesameuniversity (e.g.,Princeton),whereDS-ified versionswillbetestdriveninlateryearsoftheproject

2. Studentsurveys3. Instructorsurveys4. TrackingchangesbetweensuccessiveofferingsofDS-ified courses

CommunitybuildingGoalistoactasapointaroundwhichthingscrystallize…

• Workshops (everysummer)• Summerschools(beginningnextsummer)• Visitorprogram(acceptingapplications!)• IndustrialAdvisoryBoard• SupportforCoqdevelopment• Jobs forpostdocs,engineers,PhDstudents

28

Joinus!• DeepSpec isnotaboutbuildingasinglesystemorstack– It’saboutfindingouthowtomakeconnectionsbetweensystems

• Whowouldyou liketoconnectto?

29