View
2
Download
0
Category
Preview:
Citation preview
12/10/2016 1
1Intelligent Infrastructure Systems Lab
12/10/2016 1
1
(1) Departments of Civil Engineering, Purdue University, United States
(2) Departments of Mechanical Engineering, Purdue University, United States
(3) Luna Innovation Inc., United States
Shirley J. Dyke1, Chul Min Yeum1, Christian Silva2, and Jeff Demo3
12/10/2016 1
12/10/2016 1
Applications of Computer Vision in
Structural Health Monitoring
12/10/2016 2
2Intelligent Infrastructure Systems Lab
Presentation Outline
1. Big Picture of Vision based Structure Health Monitoring
2. Vision based Automated Visual Inspection of Large-scale Infrastructure
• Object Recognition based Crack Detection
• Optimal Design and Identification of Fiducial Markers
3. Vehicle Classification on a Mobile Bridge
4. Conclusion
12/10/2016 3
3Intelligent Infrastructure Systems Lab
Processing
Movement
• Feature extraction and object detection
• Optimal decision making
• High performance computing
Big Picture of Vision based Structural Health Monitoring
Documentation
Sensing
• High resolution acquisition in 2D/3D
• 2D photo stitching/blending
• 3D point cloud construction
• High accessibility and coverage
• Communication with sensor network
• Autonomous flying
• Indexing and searching
• Descriptive metadata schema
• Data visualization
12/10/2016 4
4Intelligent Infrastructure Systems Lab
Previous Research Works
1. Mohammad R Jahanshahi, Purdue University, USA
2. Ioannis Brilakis, Georgia Institute of Technology, USA
3. Michael O’Byrne, Trinity College Dublin, Ireland
4. Alberto Ortiz, University of Balearic Islands, Spain
Crack detection
and quantification
Image stitching
for defect
detection
Spalling
detection
Post earthquake
evaluation
1 2
3 4
3D recovery for
underwater inspection
Vessel inspection using UAV
Pavement defect
detection and
quantification
brick counting
for façade
construction
Surface damage segmentation Corrosion detection
12/10/2016 5
5Intelligent Infrastructure Systems Lab
Presentation Outline
1. Big Picture of Vision based Structure Health Monitoring
2. Vision based Automated Visual Inspection of Large-scale Infrastructure
• Object Recognition based Crack Detection
• Optimal Design and Identification of Fiducial Markers
3. Vehicle Classification on a Mobile Bridge
4. Conclusion
12/10/2016 6
6Intelligent Infrastructure Systems Lab
Drone
Fatigue crack
Objective
Development of a vision-based visual
inspection technique using a large volume
of images collected by aerial cameras
Proposed Approach
Dangerous
Traffic blockAccessibility
Advantage
• Fully automated visual inspection
• Use of images taken under
uncontrolled circumstance
• Robust detection and minimizing
false-positive detection and
misdetection
12/10/2016 7
7Intelligent Infrastructure Systems Lab
Problems of Current Vision based Damage Detection
• Many false-positive alarms and misdetections
→ Detection of damage-sensitive areas (object)
• Visibility depending on viewpoints
→ Use of many different viewpoints of object images
Non-crack area Images of a fatigue crack from different view points
12/10/2016 8
8Intelligent Infrastructure Systems Lab
Overview of the Proposed Techniques
··· Object 2
Object 1
Object 3
Same bolts
Object n
Damage
●●●
: Object patch(obj)
obj1obj2
obj3
obj1
obj3obj2
Step 1. Image Acquisition
··· image 1image 2
Structure : Object
Step 2. Object Detection
Step 3. Object Grouping Step 4. Damage Detection
Crack-like edge
Object
boundary
Strong crack-
like edge
12/10/2016 9
9Intelligent Infrastructure Systems Lab
Experimental Setup and Results
• # of images : 72 (Nikon D90)
• Image resolution : 4288 x 2848
• # of object (bolts) : 68
• # of artificial cracks : 2 (A and B)
• Working distance : 2~3 m
• # of training images : 5 (68 positive and
204 negative image patches)
Location A Location B
Crack-like edge
Object
boundary
Strong crack-
like edge
Damage detection
12/10/2016 10
10Intelligent Infrastructure Systems Lab
Presentation Outline
1. Big Picture of Vision based Structure Health Monitoring
2. Vision based Automated Visual Inspection of Large-scale Infrastructure
• Object Recognition based Crack Detection
• Optimal Design and Identification of Fiducial Markers
3. Vehicle Classification on a Mobile Bridge
4. Conclusion
12/10/2016 11
11Intelligent Infrastructure Systems Lab
Motivation of Marker-based Structural Health Monitoring
Drone fleets could monitor bridge safety*
* Reference: http://spectrum.ieee.org/tech-talk/robotics/aerial-robots/drones-could-monitor-bridge-safetyResearchers: Usman Khan (Tufts University), Babak Moaveni (Tufts University)
In reality
Problem: Marker corruption (dirt, torn, shadow, …)
Engineer
May not
feasible
12/10/2016 12
12Intelligent Infrastructure Systems Lab
Proposed Error-correctable Marker Design and Detection
Objective
• Error-correctable design and detection
of fiducial markers under permanent
occlusion (corruption)
• Development of configurable optimal
marker design
Contribution
• Advanced error-correctable capability
under permanent occlusion
(corruption)
• Probabilistic evaluation of error-
correctable capability
Installed markers
• How to design markers for correcting
errors
• How to estimate original markers from
corrupted markers
…
Original
(no corruption)
8bit errors
(black)7bit errors
(white)
Corrupted markers
or
12/10/2016 13
13Intelligent Infrastructure Systems Lab
Demonstration of the Proposed Technique (Video)
Marker 1 Marker 2 Marker 3 Marker 4
• Design of 30 markers (Rotation invariant)
• 13 minimum hamming distance
• 98% of true-positive (allowable 8 bits of
errors, which is 2 bit higher than the best
error-correctable markers in the literature )
12/10/2016 14
14Intelligent Infrastructure Systems Lab
Presentation Outline
1. Big Picture of Vision based Structure Health Monitoring
2. Vision based Automated Visual Inspection of Large-scale Infrastructure
• Object Recognition based Crack Detection
• Optimal Design and Identification of Fiducial Markers
3. Vehicle Classification on a Mobile Bridge
4. Conclusion
12/10/2016 15
15Intelligent Infrastructure Systems Lab
Motivation of Developing Vehicle Classification Algorithm
Objective
Development of an algorithm to accurately monitor usage patterns of the bridge,
recording the classes of vehicles traversing a mobile bridge
Rapidly Emplaced Bridge (REB)
Remaining Service Life
indicators (RSLI)
RSLI has four fatigue fuses;
each fuse is designed to break
as the bridge incurs an
approximate number of
crossings (full-load cycles)
12/10/2016 16
16Intelligent Infrastructure Systems Lab
Fine-grained image categorization Vehicle classification on mobile bridges
Similarity between Object Image Categorization and Vehicle Classification
…
Class 1 Class 2
Images from BMW-10 data set
What class?
Class 10
Which vehicle?
Accelerometer
0 1 2 3 4 5
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
Time (s)
No
rmalized
am
p.
0 1 2 3 4 5
-0.05
0
0.05
Time (s)
No
rmalized
am
p.
0 1 2 3 4 5
-0.06
-0.04
-0.02
0
0.02
0.04
Time (s)
No
rmalized
am
p.
0 1 2 3 4 5
-0.05
0
0.05
Time (s)
No
rmalized
am
p.
0 1 2 3 4 5
-0.04
-0.02
0
0.02
0.04
Time (s)
No
rmalized
am
p.
0 1 2 3 4 5
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
Time (s)
No
rmalized
am
p.
0 1 2 3 4 5-0.06
-0.04
-0.02
0
0.02
0.04
0.06
Time (s)
No
rmalized
am
p.
0 1 2 3 4 5
-0.05
0
0.05
Time (s)
No
rmalized
am
p.
0 1 2 3 4 5
-0.04
-0.02
0
0.02
0.04
0.06
0.08
Time (s)
No
rmalized
am
p.
12/10/2016 17
17Intelligent Infrastructure Systems Lab
Overview of the Proposed Technique
Training
Step 1. Acceleration signal acquisition
Step 2. Estimation of vehicle exit time
Step 3. Signal resampling
Step 4. Spectrogram computation
Step 5. Integral image computation
Step 6. Feature extraction
Step 7. Learning vehicle classifiers
Testing
Step 1. Running 1~6
Step 2. Training data set estimation using a
reference vehicle
Step 3. Applying vehicle classifiers learned from
corresponding training data set
0 1 2 3 4 5 6 7 8 9 10
-2
-1
0
1
2
Time (s)
Accele
rati
on
(m
/s2)
0 1 2 3 4 5
-0.04
-0.02
0
0.02
0.04
Time (s)
No
rmalized
am
p.
Time sample
Fre
qu
en
cy s
am
ple
100 200 300 400 500 600 700 800 900 1000
20
40
60
80
100
120
0.2
0.4
0.6
0.8
1
Step 1. raw sig.
Step 3. resamp. sig.
Step 4. spectrogram
12/10/2016 18
18Intelligent Infrastructure Systems Lab
Preliminary Full Scale Experimental Testing
Predicted class
Actual
classV1 V2 Accuracy
V1 15 0 (15/15) 100 %
V2 1 7 (7/8) 88 %
• Installation of 12 Acc.
• 1024 Hz sampling
• Wood supports and ramps
• Starting from outside of the bridge
• 276 sample data (23 * 12)
V1: 15 runs (slow, middle, fast speed)
V2: 8 runs (slow, fast speed)
Confusion matrix
Run 1
V1
V2
Run 2
12/10/2016 19
19Intelligent Infrastructure Systems Lab
Experimental Setup (Lab-scale)
• Installation of 8 Acc.
• 1024 Hz sampling
• Drawing vehicles from three different
people
• Starting from outside of the bridge
• 864 sample data (8 x 6 x 3 x 6)
6 vehicle (V1, V3, V4, V5, V6)
6 Run (3 forward, 3 backward)
8 Sensors
3 Boundary (BG, BR, BW)
B1 (gravel) B2 (rubber) B3 (wood)Bridge installation
Testing vehicles
12/10/2016 20
20Intelligent Infrastructure Systems Lab
Experiment Video (Lab-scale)
12/10/2016 21
21Intelligent Infrastructure Systems Lab
Vehicle Classification Results (Lab-scale)
Predicted class
Actual
classV1 V3 V4 V5 V6 V7 Accuracy
V1 17 0 0 1 0 0 (17/18) 94.4 %
V3 0 18 0 0 0 0 (18/18) 100 %
V4 0 0 18 0 0 0 (18/18) 100 %
V5 0 0 0 18 0 0 (18/18) 100 %
V6 0 0 0 0 18 0 (18/18) 100 %
V7 0 0 0 0 0 18 (18/18) 100 %
Predicted class
Actual
classV1 V3 V4 V5 V6 V7 Accuracy
V1 17 0 0 1 0 0 (17/18) 94.4 %
V3 1 17 0 0 0 0 (17/18) 94.4 %
V4 6 0 11 1 0 0 (11/18) 61.1%
V5 7 0 1 10 0 0 (10/18) 55.6%
V6 1 0 2 3 12 0 (12/18) 66.7%
V7 0 0 0 0 0 18 (18/18) 100%Run 1 Run 2 Run 3
V7
V6
V5
V4
V3
V1
Confusion matrix (B1-B1,B2-B2,B3-B3)
Confusion matrix (B1-B1B2,B2-B2B3,B3-B1B3)
12/10/2016 22
22Intelligent Infrastructure Systems Lab
Presentation Outline
1. Big Picture of Vision based Structure Health Monitoring
2. Vision based Automated Visual Inspection of Large-scale Infrastructure
• Object Recognition based Crack Detection
• Optimal Design and Identification of Fiducial Markers
3. Vehicle Classification on a Mobile Bridge
4. Conclusion
12/10/2016 23
23Intelligent Infrastructure Systems Lab
Conclusion
• Visual data provides crucial and abundant information regarding the condition of a structure,
such as change detection,
• Recent advances in the various sensors and sensing systems achieve remarkable visual
sensing capabilities in time and space using automated methods. Moreover, The field of
computer vision is devoted to such problems of interpreting the world through the analysis of
visual images.
• The opportunities associated with automated processing and advanced sensing systems
have accelerated the work to develop autonomous visual methods for SHM.
• This study successfully shows implementation of computer vision technology to solve two
different problems in SHM.
• It is anticipated that such repurposing of computer vision technology can address many
problems in SHM with intelligent ways.
12/10/2016 24
24Intelligent Infrastructure Systems Lab
Thanks You
Recommended