Parfait Lessons Learnt - Karim Ali · product or service remains at the sole discreBon of Oracle. Any views expressed in this Any views expressed in this presentaon are my own and

Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|

ParfaitLessonsLearnt

CrisBnaCifuentes,NathanKeynes,ManuelValdiviezo*,JohnGough,DianeCorneyOracleLabsAustralia*OracleParfait17July2016


ThefollowingisintendedtoprovidesomeinsightintoalineofresearchinOracleLabs.ItisintendedforinformaBonpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfuncBonality,andshouldnotberelieduponinmakingpurchasingdecisions.OraclereservestherighttoalteritsdevelopmentplansandpracBcesatanyBme,andthedevelopment,release,andBmingofanyfeaturesorfuncBonalitydescribedinconnecBonwithanyOracleproductorserviceremainsatthesolediscreBonofOracle.AnyviewsexpressedinthispresentaBonaremyownanddonotnecessarilyreflecttheviewsofOracle.

3

Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.| 4

TodevelopastaBccodeanalysistoolthatisprecise(>=90%trueposiBves)yetscalabletomillionsoflinesofC/C++codeinanightlyrun


TheParfaitDesign2007


BuiltonTopofLLVM


SnapshotofParfaitResults

7

June 2009 Kernel Part LOCBuffer

overrunBug density Status Time (min)

OpenSolaris UTS b105

Core 2.1M 15 0.0069 Being fixed 5

Device drivers

1.2M 67 0.054 Being fixed

September 2010

ON Part LOC # bug types Memory Time (min)

OpenSolaris ON All 10.4M 9 10-20x .bc 90


TheBellsandWhistlestoEnableTechTransfer

Tracewitnessforeachbugreport

UniquebugidenBtyviahashes

21

ServerintegraBonwithbugtrackingsystem

4

ServertokeeptrackofmulBpleruns

3


TheTransfer

June2012

UsedbythousandsofdeveloperswithinOracleonadailybasis

•  ParfaitbecomesaninternalOracleproduct

•  UsedinternallybyRDBMS,Solaris,OEL,TimesTen,…


NewLanguageandAnalysisSupport

June2013

•  StartJavalanguagesupport

•  AnalysesfocusonvulnerabiliBesintheJavaplaiorm

•  UsedinternallybyJavaProductGroup

June2015

•  StartPL/SQLlanguagesupport

•  AnalysesfocusonwebvulnerabiliBes

•  TobeusedbyJEEandcloudorganisaBons

FocusonvulnerabiliBesratherthanbugs


WhatWorkedWell

11


•  “Looseningup”Clang– TosupportmulBpleCcompilersandoldversionsofC

•  TranslaBonoflanguageforanalysis– Java,PL/SQL

• MulB-languagesupportandreuseofanalyses

12

Frontend


• Demand-drivenanalysisscaleswell– Combinedwithextensivecaching– FuncBonsummarieshelp

• Backwardsreusableframeworks– Dataflow– Symbolicanalysis

• HavingabstracBonsalignwellwiththecodeunderanalysis– E.g.,bit-flagoperaBons

13

Analysis


• Usability

– ServertokeeptrackofmulBpleruns– Bughashesto

•  compareresultsfromdifferentruns,and•  groupbugs

– Tracewitnessforeachbugreport

14

PresentaBonFramework


•  LLVMworkswellastheunderlyinginfrastructure– IR– Analysissupport

15

Infrastructure


TheInBetween

16


•  Layeredanalysisworksbutnotfullyusedasoriginallyplanned– MostanalyseshavemulBpleexitpoints– PromoBonsofonebugtypetoanother

17

OriginalLayeredAnalysisDesign


•  Intermodulesupport– AnalysingoneLLVMmoduleataBmedoesn’tworkforlargemonolithiccodebases•  E.g.,200GBRAMtoprocessone.bcfile

– ReuseofresultsofanalysisofdynamiclibrarieslinkedintomulBplebinariesisneeded

•  IncrementalanalysisattheLLVMmoduledoesn’tworkforeveryone– Someteamswantincrementalatsubcomponentlevels

18

GranularityofAnalysis


• Replicatedworkduetoindependentdevelopmentoftheanalyses

• BughashesessenBalbuthardtokeepconsistent

19

ParfaitInfrastructure


WhatDidn’tWorkWell

20


• UseofopBmisaBonstosimplifyIR– Removedinfavourofusefulbugreports

• RequiresdatafromtheAST– Neededforusefulbugreports

• Cannotrepresentdynamicfeaturesoflanguages

21

IR


•  llvm-lddoesn’tscalewell•  .bcformatisnotindexable

– Nowusingfileformatthatsupportsrandomaccess

•  SupportforotherCcompilersnotofinteresttotheClangcommunity

22

LLVMInfrastructure


•  Technicaldebtexposedwhenimprovinganalysiscodecoverage

•  IncompletecallgraphduetofuncBonpointersandvirtualcalls

23

Analysis


•  “Expensive”analysesarenotdeployedinproducBon– IfrunBmeislargerthanallocatednightlyintegraBonwindow

24

UsabilityandDevelopmentOrganisaBon’sWorkflow


MainTakeaways

25


WorkedWell

•  Scalabilitythroughdemand-drivenanalyses+caching+funcBonsummaries

• Precisionthroughunsoundness+heurisBcs

•  LanguagetranslaBonforanalysis• UsabilitythroughuserandorganisaBonaldeploymentexperience

NeedsMoreWork

•  ExtensibilityonlypossiblethroughhandwrisenC++– Newlanguages– Newanalyses

•  InfrastructurechangesbecomechallengingasBmegoesby

26

ParfaitforC/C++,JavaandPL/SQL–MainTakeaways


ManyPeopleHaveWorkedonParfaitOvertheYears•  CrisBnaCifuentes•  BernhardScholz•  NathanKeynes•  LianLi•  ChenyiZhang•  EricaMealy

•  MichaelMounteney

•  SimonLong

•  NathanHawes•  MikeVanEmmerik

•  ChrisBanHoermann

•  ManuelValdiviezo

•  AndrewBrowne•  AdamHeron

•  JimmyTi

•  JacobZimmermann

•  AndrewCraik•  BradMoody

•  BenBarham

•  DouglasTeoh•  DucHoaiNguyen•  EdwardEvans•  DominicFerreira

•  IjazFaiz

•  BenDean•  BenJones•  DanielDawson•  AdamHeron

•  KostyantynVorobyov•  DianeCorney•  JohnGough•  DanielWainwright

•  NicholasAllen•  BrianModra

•  MashewJohnson

•  PaddyKrishnan

•  TomasKotal

•  VinceChiang•  LinGao•  RichardMarks

•  MinhtriPham

•  FrançoisGauthier•  AlexanderJordan•  VladimirSilchanka

•  TomKing

•  RamonMillsteed

27


Parfait:scalableandprecisebugdetecBonforstaBclanguages

28



ObservaBon1:somebugsareeasytofind,othersarehardtofind


ObservaBon2:cheapprogramanalysescanfindeasybugs,expensiveprogramanalysescanfindcomplexbugs


June2009BufferOverflowResultsOverOpenSourceOSKernels

KernelTime (min)

Part LOCBuffer

overrunBug density Status

OpenSolaris UTS b105

5 Core 2.1M 15 0.0069 Being fixed

Device drivers 1.2M 67 0.054 Being fixed

Linux 2.6.29* 13 Core 1.6M 12 0.0073 Fixed

Device drivers 4.1M 85 0.020 Submitted

OpenBSD 4.4 2 Core 0.5M 3 0.0060 Fixed

Device drivers 0.8M 26 0.029 Fixed

*LinuxhasthebenefitoftwoseparatescansalreadymadebyCoverityovertheirkernelcode


November2009–September2010CommonCBugsResultsOverOpenSolarisONCode

9.5 MLOC 10.3 MLOC 10.4 MLOC 10.4 MLOC 10.4 MLOC


ProvideinterfacetoDatalog Provideinterfacetootherlanguages

35

Extensibility–PossibleSoluBons


MemoryConsumpBon• Memoryusage:10x-20xsizeof.bc

36

Documents

Parfait Lessons Learnt - Karim Ali · product or service remains at the sole discreBon of Oracle. Any views expressed in this Any views expressed in this presentaon are my own and