View
3
Download
0
Category
Preview:
Citation preview
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 1 / 25
IJCNN 2017
Anchorage, AK, USA
How to Get Pavement Distress Detection Ready for Deep Learning? A Systematic Approach Markus Eisenbach
Ilmenau University of Technology (Germany)
Neuroinformatics and Cognitive Robotics Lab markus.eisenbach@tu-ilmenau.de www.tu-ilmenau.de/neurob
R. Stricker, D. Seichter, K. Amende, K. Debes, H.-M. Gross
Ilmenau University of Technology
Neuroinformatics and Cognitive Robotics Lab
Germany
M. Sesselmann,
D. Ebersbach
LEHMANN + PARTNER GmbH
Germany
U. Stoeckert
German Federal Road Research Institute (BASt)
Germany
ASINVOS Funded by BMBF under grant agr. no. 01IS15036
Project
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 2 / 25
IJCNN 2017
Anchorage, AK, USA
Motivation Why do we need Deep Learning for road condition assessment?
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 3 / 25
IJCNN 2017
Anchorage, AK, USA
What do you want?
Safe roads of course!
But this implies...
But this is the last thing I want
What do you want?
Safe roads of course!
But this implies...
But this is the last thing I want
If you would ask a driver...
Condition assessment necessary
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 4 / 25
IJCNN 2017
Anchorage, AK, USA
Is this road in good condition?
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 5 / 25
IJCNN 2017
Anchorage, AK, USA
Is this road in good condition?
Easy recognition due to focusing
No need to have a perfect detection system
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 6 / 25
IJCNN 2017
Anchorage, AK, USA
4 year cycle
13,000 km + 40,000 km 8,000 mi 25,000 mi
German road system
~ 7,500 km 4,700 mi
= 30,000 lane-km per year
(2,000 HD-images per lane-km)
? ? 18,600 lane-mi
= 30,000 lane-km per year
(2,000 HD-images per lane-km)
? ? 18,600 lane-mi
4 year cycle
~ 7,500 km 4,700 mi
13,000 km + 40,000 km 8,000 mi 25,000 mi
German road system
How many roads to analyze?
Analyze 60 M images / year
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 7 / 25
IJCNN 2017
Anchorage, AK, USA
How is it done at the moment?
manual labor lots of
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 8 / 25
IJCNN 2017
Anchorage, AK, USA
Outline What can you expect?
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 9 / 25
IJCNN 2017
Anchorage, AK, USA
Outline – What can you expect?
What do you need to train a useful distress detection systems?
GAPs dataset
Goal: Assist operators by Deep Learning
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 10 / 25
IJCNN 2017
Anchorage, AK, USA
Outline – What can you expect?
How good are the trained classifiers in crack detection?
Deep Learning techniques
First results
Goal: Assist operators by Deep Learning
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 11 / 25
IJCNN 2017
Anchorage, AK, USA
GAPs German Asphalt Pavement
distress dataset
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 12 / 25
IJCNN 2017
Anchorage, AK, USA
Standardized Data Acquisition Certified by the German Federal Highway Research Institute (BASt)
Standardized Data Acquisition Certified by the German Federal Highway Research Institute (BASt)
German Asphalt Pavement distress dataset (GAPs)
RMA*-specified labeling (coarse for ~1x1m cells)
and detailed labeling (tightly enclosing Bounding Boxes)
* RMA ... Road Monitoring and Assessment
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 13 / 25
IJCNN 2017
Anchorage, AK, USA
German Asphalt Pavement distress dataset (GAPs)
5 distress types (of 6 in FGSV-regulation*)
Crack 83%
Pothole 9%
Inlaid patch 2%
Applied patch 2%
Open joint 4%
* 6th distress type: Bleeding (was not present in data)
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 14 / 25
IJCNN 2017
Anchorage, AK, USA
1m Lane
3 German federal roads 1969 HD-images (8 bit) labeled
High resolution 1 Pixel = 1.2 mm x 1.2 mm
1m Lane
3 German federal roads 1969 HD-images (8 bit) labeled
High resolution 1 Pixel = 1.2 mm x 1.2 mm
German Asphalt Pavement distress dataset (GAPs)
Patches extracted 64x64 pixels ~ 8 cm x 8 cm (3.2 in x 3.2 in)
4 M intact road (subsampled)
1.3 M distress (mainly from )
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 15 / 25
IJCNN 2017
Anchorage, AK, USA
Installation (90GB!!):
pip install gaps-dataset
Python (MNIST-like import):
>>> from gaps_dataset import gaps
>>> gaps.download(login=‘your_login‘)
>>> x, y = load_chunk(0, ‘train‘)
Installation (90GB!!):
pip install gaps-dataset
Python (MNIST-like import):
>>> from gaps_dataset import gaps
>>> gaps.download(login=‘your_login‘)
>>> x, y = load_chunk(0, ‘train‘)
Website: (Version 2.0 will appear soon: 50k dataset, patch sizes)
https://www.tu-ilmenau.de/neurob/data-sets-code/gaps/
German Asphalt Pavement distress dataset (GAPs)
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 16 / 25
IJCNN 2017
Anchorage, AK, USA
Experiments How good are the trained
classifiers in crack detection?
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 17 / 25
IJCNN 2017
Anchorage, AK, USA
Detection of JPEG alignment
How hard is distress detection?
Detection of distress
vs
Is this patch aligned with a JPEG block?
Does this patch contain a crack?
Easier for
machines
Easier for
humans
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 18 / 25
IJCNN 2017
Anchorage, AK, USA
How did the Deep Learning approaches perform? Visual results
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 19 / 25
IJCNN 2017
Anchorage, AK, USA
How did the Deep Learning approaches perform? Visual results
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 20 / 25
IJCNN 2017
Anchorage, AK, USA
Performance Evaluation
Results in numbers
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 21 / 25
IJCNN 2017
Anchorage, AK, USA
8 Conv layer, 3 Max-pooling layer, 3 Fully connected layers 4 M weights
VGG-like
vs labeled distress
intact
3x
Which nets did we use?
8 Conv layer, 3 Max-pooling layer, 3 Fully connected layers 4 M weights
3x 3x VGG-like
For comparison: RCD* net (4x Conv, 4x Pool, 2x FC) 300k weights
* RCD ... Road Crack Detection
AlexNet-like
vs labeled distress
intact
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 22 / 25
IJCNN 2017
Anchorage, AK, USA
Features +
Boosting
0,3121
Machine Learning
CrackIT
0,4882
Compu. Vision
Features +
Boosting
0,3121
Machine Learning
CrackIT
0,4882
Compu. Vision
RCD 99x99
RCD 64x64
0,7184 0,6642
Deep Learning
0,7246
ASINVOS 64x64
Deep Learning outperforms classical approaches
F1-Score
Results in numbers
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 23 / 25
IJCNN 2017
Anchorage, AK, USA
Evaluation of regularization
Dropout
Batch normalization
Batch normalization + Dropout
Weight decay + Dropout
Max-norm regularization + Dropout
None ...not in the paper:
Adversarial Training
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 24 / 25
IJCNN 2017
Anchorage, AK, USA
Conclusion
How to Get Pavement Distress Detection Ready for Deep Learning?
M. Eisenbach et al. – Ilmenau University of Technology, Neuroinformatics & Cognitive Robotics Lab 25 / 25
IJCNN 2017
Anchorage, AK, USA
Conclusion How to Get Pavement Distress Detection Ready for Deep Learning? A Systematic Approach
Demand for Deep Learning assisted distress detection
GAPs dataset Regularization, Performance
D BN A
Recommended