Embed Size (px)
Citation preview
Edge Computing with Jetson TX2 for Monitoring Flows of Pedestrians and Vehicles
Dr J. Barthélemy, Dr N. Verstaevel, Dr H. Forehead, Senior Prof. P. Perez
Edge Computing with Jetson TX2 for Monitoring Flows of Pedestrian and Vehicles
At SMART, we believe that People with good
information and good tools will make good
Decisions and change our world
2
Technology and community: DLL
3
Solving problems: The case of Liverpool
CBD is growing fast: new UoW campus, airport development,…
What does it mean for the city and its community? What are the problems?
Smart Cities and Suburbs Program: How can we use IoT to solve the problems?
4
Which problems?
Let’s ask the people! Pedestrians- Where are they going?- What are the most popular routes?- What are the most congested locations?- Impact of city activity?
Cyclists- Which route are they taking?- How can we improve bike usage?
Cars- Live traffic?
5
Sensors locations
20 sensors
Image credit: OpenStreetMap
6
How about using CCTV?
7
Privacy!
Sensor requirements
• Mobile units + leveraging CCTV infrastructure
• Privacy is important!
– On board video analytics
– Only indicators transmitted (no raw data!)
• Real-time image processing
• LoRaWAN network
– Long range, low bandwidth (200 bytes/message)
– Free to use by the community8
History of the prototypes
10
The sensorAn edge computing device
Components
— NVIDIA Jetson TX2 for onboard processing
— Pycom LoPy 4 for data transmission on The Things Network
— Camera (USB webcam / existing CCTV)
11
Architecture of the solution
Fixed counters
Air quality x20
Noise level x20
x5
x15
Mobile counters
Sensors IoT Core ApplicationsTransport
+Private and Public APIs Traffic
modelling
Analytics
Dashboard
Citizens app
12
From the input to the dashboard
Image Acquisition
• CCTV/Webcam (OpenCV)
Detection
• Deep Convolutional Neural Network
Tracking
• Kalman Filtering
Data transmission
• LoRaWAN/OneM2M
Dashboard/Database
Image credit: NVIDIA Corporation
Image credit: Pycom
13
Detection: YOLO v3
Inference: Jetson TX2 @ 10 fpsTraining: Titan Xp
• Fully convolutional DNN• 106 hidden layers• Detections at 3 scales• 3 classes: person, bicycle, vehicles• Pascal VOC and COCO datasets
cuDNN
FP16+ +
14
NVIDIA GPU Grant
Detection: YOLO v3• Detecting locations of pedestrians and vehicles
• Number of objects of each type
15
VIRAT Video Dataset
Tracking: Kalman Filtering• Associating IDs with the detections
• Trajectories
16
VIRAT Video Dataset
Final output
No image Privacy OK!
17
Dashboard
18
DashboardHeatmap of the maximum number of detections
19
DashboardTrajectories of the detections
20
DashboardTrajectories of the detections (inside a building)
21
DashboardTrajectories of the detections (inside a building)
22
Dashboard
23
Next step: inferring network dynamics
Image credit: OpenStreetMap Image credit: Google Maps
24
ConclusionsIt’s only the beginning!
• Jetson TX2 for real time object detection and tracking
• Privacy compliant… but meaningful information
• Open data for people centric approach
– citizen applications
– city and traffic planners
• Scalability and interoperability
• Framework can integrates other sensors
– air quality, noise
Traffic modelling
AnalyticsDashboardCitizens app
25
An ecosystem around the Jetson TX2
26
Keep In Touch
linkedin.com/company/smart-
infrastructure-facility-university-
of-wollongong
@SMART_Facility
smart.uow.edu.au
uowblogs.com/smartinfrastructureSMART Infrastructure Facility
27
