16

Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi, Ramneet Kaur, James Weimer, Oleg Sokolsky, Insup Lee University of Pennsylvania July 13, 2018

Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

Download PDF Report

Upload
others
View
0
Download
0

Embed Size (px)

Citation preview

Page 1: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

Self-DrivingVehicleVerificationTowardsaBenchmark

NimaRoohi,RamneetKaur,JamesWeimer,OlegSokolsky,InsupLee

UniversityofPennsylvania

July13,2018

Page 2: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

• 

July13,2018 2

WinterofAIforAutonomousVehicles

Page 3: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

• 

July13,2018 3

WinterofAIforAutonomousVehicles

Page 4: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

•  SystemSpecification–  Finitesetofcars–  Onlymoveforward

–  Drivingonastraightroad•  FiveVerificationChallenges–  (Robust)Safety–  (Robust)Liveness–  ResponsibilitySensitiveSafety

•  CurrentStateoftheAutomaticFormalVerificationTools–  Unabletospecifyanyoftheseproblems

July13,2018 4

Outline

Page 5: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

• 

July13,2018 5

PositionandDelaySignals

Page 6: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

• 

July13,2018 6

MinimumLongitudinalDistanceSignal

Page 7: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

• 

July13,2018 7

DangerousLongitudinalSituationSignal

Page 8: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

• 

July13,2018 8

BlameTimeSignal

Page 9: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

• 

July13,2018 9

Policy

Page 10: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

•  SystemSpecification–  Finitesetofcars–  Onlymoveforward

–  Drivingonastraightroad•  VerificationChallenges–  (Robust)Safety–  (Robust)Liveness–  ResponsibilitySensitiveSafety

•  CurrentStateoftheAutomaticFormalVerificationTools–  Unabletospecifyanyoftheseproblems

July13,2018 10

Outline

Page 11: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

• 

July13,2018 11

Safety

Page 12: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

• 

July13,2018 12

RobustlySafeandRobustlyUnsafe

Page 13: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

•  (robust)Liveness:–  Somethinggoodwilleventuallyhappen

•  RSS:–  Incaseofanaccidenttherewillbearesponsibleparty

July13,2018 13

(Robust)LivenessandResponsibilitySensitiveSafety

Page 14: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

•  SystemSpecification–  Finitesetofcars–  Onlymoveforward

–  Drivingonastraightroad•  VerificationChallenges–  (Robust)Safety–  (Robust)Liveness–  ResponsibilitySensitiveSafety

•  CurrentStateoftheAutomaticFormalVerificationTools–  Unabletospecifyanyoftheseproblems

July13,2018 14

Outline

Page 15: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

July13,2018 15

Tools

•  Wecannotevenspecifythesechallengesusingthesetools–  markswhenatoolfails

– Whitecellsdonotguaranteeafeaturesupport

Page 16: Self-Driving Vehicle Verification Towards a Benchmarkiot.stanford.edu/nsf-final/slides/sitp-nsf-final-benchmark.pdf · Self-Driving Vehicle Verification Towards a Benchmark Nima Roohi,

•  Delayismadeexplicitindynamicsandpolicy

•  Computationofblametimesignalisslightlychanged•  Policyis(slightly)changed

July13,2018 16

AFewDifferences

Design Verification An Overview. Powerful HDL Verification Solutions for the Industry’s Highest Density Devices What is driving the FPGA Verification

Design Verification An Overview. Powerful HDL Verification Solutions for the Industry’s Highest Density Devices What is driving the FPGA Verification

Documents

NSF/ANSI ESTÁNDAR 42 NSF PROTOCOL P231 NSF/ANSI STANDARD 53 · NSF/ANSI STANDARD 53 ... NSF PROTOCOLO P231 (Rendimiento Microbiológico) Este Sistema ha sido probado de acuerdo con

NSF/ANSI ESTÁNDAR 42 NSF PROTOCOL P231 NSF/ANSI STANDARD 53 · NSF/ANSI STANDARD 53 ... NSF PROTOCOLO P231 (Rendimiento Microbiológico) Este Sistema ha sido probado de acuerdo con

Documents

RESTRITED DRIVING PERMIT ITD 3227 (Rev. 05 APPLIATIONWork and/or School Verification for Restricted Driving Permit . The work and/or school verification for restricted driving permit

RESTRITED DRIVING PERMIT ITD 3227 (Rev. 05 APPLIATIONWork and/or School Verification for Restricted Driving Permit . The work and/or school verification for restricted driving permit

Documents

Submission for NSF Protocol P352 · Submission for Verification of Eco-efficiency Analysis Under NSF Protocol P352, Part B U.S. Beef – Phase 2 Eco-efficiency Analysis September

Submission for NSF Protocol P352 · Submission for Verification of Eco-efficiency Analysis Under NSF Protocol P352, Part B U.S. Beef – Phase 2 Eco-efficiency Analysis September

Documents

Employee Information This form is for the use of Human ... · social security verification, motor vehicle records (“driving records”), verification of your education or employment

Employee Information This form is for the use of Human ... · social security verification, motor vehicle records (“driving records”), verification of your education or employment

Documents

NSF Interdisciplinary Researchresearch.ku.edu/sites/research.ku.edu/files/docs/NSF_Interdisciplinar… · NSF Interdisciplinary Research • NSF Goals: – Transform the Frontiers

NSF Interdisciplinary Researchresearch.ku.edu/sites/research.ku.edu/files/docs/NSF_Interdisciplinar… · NSF Interdisciplinary Research • NSF Goals: – Transform the Frontiers

Documents

to ANSI/NSF Standards 42 & 53 - Moen ANSI/NSF Standards 42 & 53 NSF

to ANSI/NSF Standards 42 & 53 - Moen ANSI/NSF Standards 42 & 53 NSF

Documents

WACCM Updates - CESM®€¦ · comparative aeronomy; upper atmosphere driving by the lower atmosphere processes. • NSF Coupling, Energetics, Dynamics of Atmospheric Regions (CEDAR)

WACCM Updates - CESM®€¦ · comparative aeronomy; upper atmosphere driving by the lower atmosphere processes. • NSF Coupling, Energetics, Dynamics of Atmospheric Regions (CEDAR)

Documents

Ani Aprahamian. Research Highlights - Examples NSF - External Home page NSF - Mathematical and Physics Sciences Directorate NSF -

Ani Aprahamian. Research Highlights - Examples NSF - External Home page NSF - Mathematical and Physics Sciences Directorate NSF -

Documents

SysInfoTools NSF Merge · SysInfoTools NSF Merge 2 1. SysInfotools NSF Merge 2. Overview The advanced NSF Merge tool of SysInfoTools Software can merge/combine your multiple NSF database

SysInfoTools NSF Merge · SysInfoTools NSF Merge 2 1. SysInfotools NSF Merge 2. Overview The advanced NSF Merge tool of SysInfoTools Software can merge/combine your multiple NSF database

Documents

Data-Driven Opportunities in the Smart Home and ...€¦ · • Driving location verification • Fleet management • Automated repair dispatch services • Real-time driving behavior

Data-Driven Opportunities in the Smart Home and ...€¦ · • Driving location verification • Fleet management • Automated repair dispatch services • Real-time driving behavior

Documents

Submission for NSF Protocol P352 · Submission for . Verification of Eco-efficiency Analysis Under . NSF Protocol P352, Part B . Incontinence Bed Pads Eco-Efficiency Analysis . Final

Submission for NSF Protocol P352 · Submission for . Verification of Eco-efficiency Analysis Under . NSF Protocol P352, Part B . Incontinence Bed Pads Eco-Efficiency Analysis . Final

Documents

NSF Astronomy and Astrophysics Postdoctoral Fellowships …NSF Astronomy and Astrophysics Postdoctoral Fellowships (AAPF) PROGRAM SOLICITATION NSF 16-575 REPLACES DOCUMENT(S): NSF

NSF Astronomy and Astrophysics Postdoctoral Fellowships …NSF Astronomy and Astrophysics Postdoctoral Fellowships (AAPF) PROGRAM SOLICITATION NSF 16-575 REPLACES DOCUMENT(S): NSF

Documents

SmartLicence - Online driving licence verification with DVLA

SmartLicence - Online driving licence verification with DVLA

Automotive

NSF News Denise Caldwell NSF Physics Division Director

NSF News Denise Caldwell NSF Physics Division Director

Documents

NSF PST Converter Tool to Recover NSF & Export NSF to PST

NSF PST Converter Tool to Recover NSF & Export NSF to PST

Software

Submission for NSF Protocol P352€¦ · Verification of Eco-efficiency Analysis Under . NSF Protocol P352, Part B . Micro Surfacing Eco-efficiency Analysis . Final Report - July

Submission for NSF Protocol P352€¦ · Verification of Eco-efficiency Analysis Under . NSF Protocol P352, Part B . Micro Surfacing Eco-efficiency Analysis . Final Report - July

Documents

Submission for NSF Verification to Protocol P352...Submission for Verification of Eco-Efficiency Analysis Under NSF Protocol P352, Part B Retail Shopping Carts Final Report – January

Submission for NSF Verification to Protocol P352...Submission for Verification of Eco-Efficiency Analysis Under NSF Protocol P352, Part B Retail Shopping Carts Final Report – January

Documents

NSF (National Standard Format) USER MANUALmanuals.momed.com/edb_pdf/NSF (National Standard Format) User...NSF (National Standard Format) User Manual 1 NSF ... FA0 Service Line Detail

NSF (National Standard Format) USER MANUALmanuals.momed.com/edb_pdf/NSF (National Standard Format) User...NSF (National Standard Format) User Manual 1 NSF ... FA0 Service Line Detail

Documents

NSF EDA Workshop Evolution and Basic Science …web.eecs.umich.edu/~mazum/NSF-EDA.pdfPinaki Mazumder, NSF/ECCS & Univ. of Michigan NSF EDA Workshop July 8-9, 2009 NSF EDA Workshop

NSF EDA Workshop Evolution and Basic Science …web.eecs.umich.edu/~mazum/NSF-EDA.pdfPinaki Mazumder, NSF/ECCS & Univ. of Michigan NSF EDA Workshop July 8-9, 2009 NSF EDA Workshop

Documents

Submission for NSF Protocol P352info.nsf.org/Certified/Sustain/ProdCert/C0023412-004.pdf · Submission for Verification of Eco-efficiency Analysis Under NSF Protocol P352, Part B

Submission for NSF Protocol P352info.nsf.org/Certified/Sustain/ProdCert/C0023412-004.pdf · Submission for Verification of Eco-efficiency Analysis Under NSF Protocol P352, Part B

Documents

A full-year, NSF-funded physics curriculum developed for ...activatelearning.com/wp...Physics-Overview-Flyer.pdf · the physics of driving by making a presentation to a board of driving

A full-year, NSF-funded physics curriculum developed for ...activatelearning.com/wp...Physics-Overview-Flyer.pdf · the physics of driving by making a presentation to a board of driving

Documents

Modular Verification of Higher-Order Methods in JML Gary T. Leavens, Steve Shaner, and David A. Naumann Support from US NSF grant CCF-0429567

Modular Verification of Higher-Order Methods in JML Gary T. Leavens, Steve Shaner, and David A. Naumann Support from US NSF grant CCF-0429567

Documents

Product Category Rule - NSF International...To initiate your LCA, receive your EPD verification, or have questions on where to start, contact NSF Sustainability at sustainability@nsf.org

Product Category Rule - NSF International...To initiate your LCA, receive your EPD verification, or have questions on where to start, contact NSF Sustainability at [email protected]

Documents

Environmental Technology Verification ReportEMAIL: SALES@VILLAGEMARINE.COM ; NSF International (NSF) manages the Drinking Water Systems (DWS) Center under the U.S. Environmental Protection

Environmental Technology Verification ReportEMAIL: [email protected] ; NSF International (NSF) manages the Drinking Water Systems (DWS) Center under the U.S. Environmental Protection

Documents

Automatic Driving License Verification System Through ...Title Automatic Driving License Verification System Through Android Smartphones Keywords Engineering Created Date 6/8/2017

Automatic Driving License Verification System Through ...Title Automatic Driving License Verification System Through Android Smartphones Keywords Engineering Created Date 6/8/2017

Documents

NSF/ANSI Standard Acceptance: Drinking Water Treatment ... Kit/Product...• NSF/ANSI 58 • NSF/ANSI 62 • NSF/ANSI 177 • NSF p231 • WQA S-100 • WQA S-200 • WQA S-300 •

NSF/ANSI Standard Acceptance: Drinking Water Treatment ... Kit/Product...• NSF/ANSI 58 • NSF/ANSI 62 • NSF/ANSI 177 • NSF p231 • WQA S-100 • WQA S-200 • WQA S-300 •

Documents

Introduction and NSF Overview September 2006. Main Topics Origins of NSF The National Science Foundation The NSF FY 2007 Budget Request NSF Initiatives

Introduction and NSF Overview September 2006. Main Topics Origins of NSF The National Science Foundation The NSF FY 2007 Budget Request NSF Initiatives

Documents

Functional Flaws Driving Need for Re-Spins - klabs.orgklabs.org/mapld06/seminars/verification/verification_seminar_mapld... · • Testbench Automation ... Ethernet Controller SDRAM

Functional Flaws Driving Need for Re-Spins - klabs.orgklabs.org/mapld06/seminars/verification/verification_seminar_mapld... · • Testbench Automation ... Ethernet Controller SDRAM

Documents

ATAC NSF Proposal Notes - Massachusetts Institute of ...people.csail.mit.edu/jim/temp/ATAC NSF Proposal, v19.doc · Web viewThis is because most of the power is consumed in driving

ATAC NSF Proposal Notes - Massachusetts Institute of ...people.csail.mit.edu/jim/temp/ATAC NSF Proposal, v19.doc · Web viewThis is because most of the power is consumed in driving

Documents