Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Computer Science and Information Technology (I) Connected and Autonomous Vehicles
Chung-Wei [email protected]
CSIE Department
National Taiwan University
Introduction to Myself
❑ B.S. Student➢ 2001.09--2005.06, CSIE Department, NTU
❑ M.S. Student➢ 2005.09--2007.06, GIEE (EDA Group), NTU
❑ Ph.D. Student➢ 2009.08--2015.08, EECS Department, UC Berkeley
• Some interesting teaching experience here
❑ Researcher➢ 2015.09--2018.07, Systems and Software Division, Toyota
InfoTechnology Center (Mountain View, CA)
❑ Assistant/Associate Professor➢ 2018.08--2021.07/2021.08--, CSIE Department, NTU
2
Outline
❑ What are connected and autonomous vehicles?
❑ What are the roles of software people?
❑ A closer look at connected and autonomous vehicles
❑ Some specific problems
3
Video (20170822-2)
Intelligent Vehicles
❑ Connected and Autonomous Vehicles (CAV)➢ A good application may need both of "connectivity" and "autonomy"
➢ What if the intersection management does not have connectivity?
➢ What if the intersection management does not have autonomy?
4
Intersection Management
Levels of Autonomy
❑ Levels of driving automation [SAE J3016 / Wikipedia]➢ Level 1 (Drive Assistance): Adaptive Cruise Control (ACC), Parking
Assistance, etc.
➢ Level 2 (Partial Automation)
• The vehicle takes full control of accelerating, braking, and steering
• The driver must monitor the driving
➢ Level 3 (Conditional Automation)
• The driver can safely turn his/her attention away from the driving tasks
• The driver must be prepared to intervene, when called upon by the vehicle
➢ Level 4 (High Automation)
• No driver attention is ever required for safety
• Self driving is supported only in limited spatial areas or under special circumstances
➢ Level 5 (Full Automation)
5
Levels of Autonomy
6
https://www.birmingham.ac.uk/news/thebirminghambrief/items/2016/11/driving-the-revolution.aspx
Connected Applications
❑ Communication standards➢ Dedicated Short-Range Communications (DSRC)
➢ C-V2X (Cellular Vehicle-to-Everything)
7
!
IntersectionManagement
Emergency Vehicle Warning
Side Road MergingSharp Curve Assistant
Cooperative Adaptive Cruise Control
Video (20160325-0)
Emergency Vehicle Warning (1/2)
8
Video (20160324-1)
Video (20160324-0)
Emergency Vehicle Warning (2/2)
9
Outline
❑ What are connected and autonomous vehicles?
❑ What are the roles of software people?
❑ A closer look at connected and autonomous vehicles
❑ Some specific problems
10
Software Design
❑ Various applications including Advanced Driver AssistanceSystems (ADAS) and autonomous functions
❑ Various software programs forsensing, signal processing, control,decision making, etc.➢ Values to vehicle's total value
• Embedded software: 2% → 13% from 2000 to 2010
• Electronics system: expected to be 50% in 2030
➢ Number of lines of code
• 1 → 10+ → 100 million from 2000 → 2010 → 2020
❑ Due to the safety-critical nature,correctness and quality ofsoftware are extremely important
11http://www.toyota.com/safety-sense/
Pre-Collision System (PCS)
Lane Departure Alert (LDA)
Automatic High Beams (AHB)
Pre-Collision System with Pedestrian Detection Function (PCS w/PD)
Dynamic Radar Cruise Control (DRCC)
Hardware Design
❑ Number of Electronic Control Units (ECUs)➢ 20 → 50+ in the past decade
❑ Integrated architecture➢ One function can be distributed over multiple ECUs, and multiple
functions can be supported by one ECU
• More sharing and contention among software functions
• Traditional federated architecture: each function is deployed to one ECU and provided as a black-box by its supplier
❑ New computational components➢ Field Programmable Gate Array (FPGA)
➢ Graphical Processing Unit (GPU)
❑ Next-generation communication protocols➢ Ethernet-based protocols
12
https://en.wikipedia.org/wiki/Ethernet
Design-Time vs. Runtime
❑ Runtime software is executed during vehicle operation➢ It is usually designed during design-time
➢ Examples of runtime software / functions
• Localization algorithm
• Object detection algorithm
• Vehicle control algorithm
❑ Design-time software is executed during the design stage➢ It is more and more difficult to manually design a vehicle
➢ Examples of design-time software / tools
• Modeling
• Design including optimization
• Analysis including simulation, verification, and testing
13
Outline
❑ What are connected and autonomous vehicles?
❑ What are the roles of software people?
❑ A closer look at connected and autonomous vehicles
❑ Some specific problems
14
Subsystems
❑ Body
❑ Chassis
❑ Suspension
❑ Control
❑ Engine
❑ Transmission
❑ Braking
❑ Wiring
❑ Electronics
❑ And many others
15
http://www.axiscades.com/automotive.html
Layered View of Autonomous Vehicles
16
Hardware Platform, e.g., Electronic Control Unit (ECU)
(Operating System)
GPS
Camera
Lidar
Radar
Sensing
Localization
ObjectTracking
Object Detection
Segmentation
Perception
Prediction
MotionPlanning
BehaviorPlanning
RoutePlanning
Planningand Decision
EnergyManagement
EmissionControl
TorqueControl
SteeringControl
Control
Light
Wheel
Brake
Engine
Actuation
Electronic Control Unit (ECU)
❑ Is an ECU like a Central Processing Unit (CPU)?➢ Yes? It does some computation
➢ No? It is not centralized
17
http://www.denso.co.id/Electro_ecu.html
https://www.indiamart.com/proddetail/denso-engine-electronic-control-unit-18672409991.html
Layered View of Connected Vehicles
❑ From multiple vehicles to components in a single vehicle
18
ExternalNetwork
(( ))
https://slideplayer.com/slide/10709648/
CAN: Controller Area NetworkLIN: Local Interconnect NetworkMOST: Media-Oriented Systems Transport
Discussion
❑ When will autonomous driving become reality?➢ Technology
➢ Cost
➢ Law and regulation
➢ Human comfort
➢ Philosophy
19
Video (20170214)
Homework
❑ Question 1➢ List and explain (e.g., 3 sentences)
• Three benefits of connected vehicles and/or autonomous vehicles?
• Three concerns of connected vehicles and/or autonomous vehicles?
❑ Question 2➢ How to measure risk and safety?
➢ List two reasons that people may not choose the transportation method with the lowest risk (defined by themselves) to travel
❑ Visit NTU COOL for details or questions
20
Outline
❑ What are connected and autonomous vehicles?
❑ What are the roles of software people?
❑ A closer look at connected and autonomous vehicles
❑ Some specific problems
21
Mapping from SW to HW (1/2)
❑ Software (functional model): task graph
❑ Hardware (architectural platform): distributed Electronic Control Units (ECUs) connected by a network
22
ECU CAN bus
fun
ctio
nal
mo
de
l
τ3
σ1
μ1,1
μ1,2
τ2
τ1
τ6
τ5
τ4
τ8
τ7
σ2
σ3
σ4
σ5
σ6
ε2ε1
μ2,1
μ2,2
ε3
arch
itec
ture
pla
tfo
rm
task allocation signal packing
broadcast
task signal
message
Mapping from SW to HW (2/2)
❑ Decide task allocation, signal packing, and priority assignments (tasks on ECUs and messages on CAN bus)
❑ Satisfy timing constraints for tasks, signals, and paths
23
ECU CAN bus
fun
ctio
nal
mo
de
l
τ3
σ1
μ1,1
μ1,2
τ2
τ1
τ6
τ5
τ4
τ8
τ7
σ2
σ3
σ4
σ5
σ6
ε2ε1
μ2,1
μ2,2
ε3
arch
itec
ture
pla
tfo
rm
task allocation signal packing
broadcast
task signal
message
Intersection Management (1/2)
❑ Every vehicle will periodically optimize its own moving path
❑ The conflict-resolution problem is reduced to a cycle-removal problem in a graph➢ If vehicles (even from different automotive companies) agree how to
remove a cycle, then "conflict-free" can be guaranteed
➢ Cycle-removal algorithm is the"agreement" between vehicles(e.g., four-way stop sign)
24
Cycle-Removal
Vehicles will go through the conflict zone (e.g., intersection) with the order in this graph
Intersection Management (2/2)
❑ The payment-based approach supports prioritized intersection management where truthfulness is guaranteed
❑ An intersection becomes "more expensive" when there are more vehicles requesting the intersection
25
Video (20170825)
Cooperative Adaptive Cruise Control (CACC)
❑ Two simplified CACC modes➢ Gap control mode
• The following vehicle (F) decidesacceleration based on the gap, speeds,and accelerations of the two vehicles
➢ Collision avoidance mode
• The following vehicle (F) decelerates with its maximum deceleration
❑ Information sources➢ Gap and speeds are obtained by sensors
➢ Accelerations are broadcasted with V2X messages
26
PFGap
CollisionAvoidance
Mode
GapControlMode
Gap is larger than gsafe
Gap is smaller than gsafe
gsafe: gap threshold
Video (20161021)
Jamming of Wireless Network (1/2)
27
Jamming of Wireless Network (2/2)
❑ An intersection manager receives requests from vehicles, schedule them, and sends confirmations to them
28
Video (20170203-1) Video (20170203-3)
Video (20170203-2) Video (20170203-4)
SomeMitigation
MoreMitigation
Without Jamming
WithJamming
Plug-and-Play Systems
❑ What if a vehicle is like a smart cellphone that applications can be downloaded, updated, and activated?➢ At dealership (customized vehicles)
➢ Before driving or even during driving?
29
Order withBSM+LKA
BSM+LKA
Configure LKA(at Dealership)
LKA
[Dealership]
[Factory] Order
[Dealership]
[User]
[Factory]
Deliver to DealershipDeliver to User Deliver to DealershipDeliver to User
[User]
BSM+LKA?
LKA? LKA?
OK
Lane Keeping Assistance (LKA)Blind Spot Monitor (BSM)
Edge Computing (1/2)
30From Automotive Edge Computing Consortium
Cloud
Data DataData
V2C2VIntelligent
Driving
High Definition
Map
too much traffic
too muchdata processing
Edge Computing (2/2)
31From Automotive Edge Computing Consortium
Cloud
Data DataData
LocalData
LocalData
LocalData
LocalData
V2C2VIntelligent
Driving
High Definition
MapV2C2V
IntelligentDriving
High Definition
Map
Edges
Software Integrity (1/3)
❑ ISO 26262 is recognized as the state-of-the-art standard for automotive systems
❑ Examples➢ Some software structures are NOT recommended for highest Safety
Integrity Level (SIL)
• Dynamic objects and variables
• Multiple uses of variable names
• Implicit type conversions
• Unconditional jumps
• Recursions
32
Management
DevelopmentSystem Level
Hardware LevelSoftware Level P
rod
uct
ion
& O
pe
rati
on
Co
nce
pt
Ph
ase
Supporting Processes
Software Integrity (2/3)
❑ A potential conflict between Original Equipment Manufacturers (OEMs) and software suppliers (developers)➢ A certification process represents a systematic way to inspect the
source codes
➢ Some source codes of software suppliers (developers) are confidential
❑ Desired properties➢ Authenticity
• Only authenticated results from compilers and analysis tools (verification, simulation, and/or testing) are considered by the certification issuers
➢ Confidentiality
• Sensitive source codes of the software suppliers and developers are not released to certification issuers
33
Software Integrity (3/3)
❑ Trusted third-party
➢ Run a certification program which consists of a compiler and an analyzer
➢ Maintain a router which controls the input and the output
❑ Certification program
➢ All of the compiler, the analyzer, and the private key are updated by the OEM
➢ The updating process must be unidirectional to guarantee confidentiality
❑ Router
➢ Only the corresponding developer can be the receiver
34
Trusted 3rd-PartyCertification Program (Installed or Updated by OEM)
Compiler
Executable
Analyzer
Log forDeveloper
Log forOEM
Signing Signature
Router
Public andPrivate Key
Developer
Censoring
Public Key
Supplier
SignatureChecking
Public Key
OEM
Public andPrivate Key
SignatureChecking
SourceCodes
Physically Located at theTrusted 3rd-Party's Site
Wire Routing
❑ The wiring weight of a system can be up to 30kg➢ The third heaviest and costliest component in an automotive system
(after the chassis and the engine)
➢ Netlist
• A set of "parts" to beconnected
➢ Splice
• Used for connectingmore than two wires
• Steiner vertex!
➢ Where to put splices?
• Steiner tree problem
35
Intrusion Detection
36
Physics RulesPrincipal Component
AnalysisHidden Markov Model
FeaturesSimple and Quick
(No Training Needed)Catch Implicit Relationships
Catch Time-Series Data
In-Car Applicable Complexity Concern Complexity Concern
Edge(Roadside Unit)
Applicable Applicable Applicable
Cloud Latency Concern Latency Concern Latency Concern
Attack A1(Lie about Acceleration)
Detected Detected Detected
Attack A2(Lie about Velocity)
Not Detected Detected Detected
Attack A3(Lie about Position)
Not Detected Not Detected Detected
Attack A4(Lie about Velocity + Position)
Not Detected Not Detected Detected
Consensus Algorithms
❑ Vehicles, road side units, edge servers, and cloud servers may have different opinions➢ Intrusion detection
➢ Dynamic map creation
➢ Event report checking
• Examples: location, speed,and acceleration of a vehicle
❑ Challenges➢ If A says that B is wrong, is
A or B actually wrong?
➢ Timing-critical information
➢ Vehicles are moving
37
Adversary Classification
❑ Adversary classification
❑ How if we add barcodes to traffic signs?
❑ A
38
[CVPR '18]
MachineLearningClassifier
There is nostop sign!
➢ "Selecting the battlefield strategically against an adversary is just likein the Battle of Thermopylae. King Leonidas of Sparta and his 300soldiers could face Persian Empire of Xerxes and his far larger armylegendarily at the strategically selected narrow coastal pass ofThermopylae, where the numerical imbalance did not make a hugedifference."
Q&A
39
Course Roadmap of UC Berkeley CS
70
61A
61B 61C
184 186188
189
169160 162 164 161
168
170
191174 176172
150
152
core
hardware
software
theory
applications
required
recommended
CS70: Discrete Mathematics and Probability Theory
Topics in CS70
❑ Propositions and Quantifiers
❑ Proofs
❑ Induction
❑ Stable Marriage
❑ Modular Arithmetic
❑ Bijections and RSA
❑ Polynomials
❑ Error Correcting Codes
❑ Counting
❑ Introduction to Discrete Probability
41
❑ Conditional Probability
❑ Random Variables
❑ Variance
❑ Two Killer Applications
❑ Chebyshev's Inequality
❑ Some Important Distributions
❑ Continuous Probability
❑ Graph Theory
❑ Infinity and Uncountability
❑ Self-Reference andUncomputability
Teaching Experience at Berkeley
❑ CS70: Discrete Mathematics and Probability Theory➢ Spring 2013, Spring 2014: teaching assistant
➢ Summer 2015: instructor (http://www-inst.eecs.berkeley.edu/~cs70/su15/)
• 300+ students on the first day
• The total tuition is about $450,000
• 11 teaching assistants --- all undergraduates
• 14 readers (to grade homework) --- all undergraduates
❑ Background➢ Some students want to meet the GPA requirement to "declare" (i.e.,
major) CS
➢ The class GPA is (was) expected to be 2.5--2.9 (3 is B and 2.7 is B-)
❑ Instructor's evaluation from students will be posted publicly
42
Weekly Schedule (8-Week Long)
43
How-To
❑ Answer students' questions➢ Homework parties
➢ Online forum
• Participation score
❑ Synchronize TAs and readers➢ Two additional forums
❑ Grade homework and exams➢ Gradescope
❑ Proctor exams (3 in 8 weeks)➢ Separated rooms
➢ Timing conflicts
➢ Special cases
44
Observations
❑ Are/do students at Berkeley…➢ Very smart?
• Pick 3 cards from 52 cards
– Sampling without replacement, order does matter
– Sampling without replacement, order does not matter
– Sampling with replacement, order does matter
– Sampling with replacement, order does not matter
➢ Study very hard?
➢ Love to ask questions?
➢ Care about grade?
• Regrading policy
➢ Have a lot of pressure?
• Special office hour
❑ How about TAs?
❑ And …45
Q&A
46