View
215
Download
1
Category
Preview:
Citation preview
5/15/2009
1
Slide 1ILLINOIS - RAILROAD ENGINEERING
Development of Machine Vision Technology for Inspection of Railroad Trackfor Inspection of Railroad Track
William W. Hay Railroad Engineering Seminar Series, 15 May 2009
Steven V. Sawadisavi, J. Riley Edwards, Esther Resendiz*John M. Hart*, Christopher P. L. Barkan, Narendra Ahuja*John M. Hart , Christopher P. L. Barkan, Narendra Ahuja
Railroad Engineering Program and Beckman Institute*University of Illinois at Urbana-Champaign
5/15/2009
2
Slide 2ILLINOIS - RAILROAD ENGINEERING
Presentation Outline
• Backgroundg
• Selection of Inspection Tasks
• Camera View SelectionCamera View Selection
• Data Collection
• Algorithm DevelopmentAlgorithm Development
• Panoramic Image Generation
• Future Work• Future Work
• Summary
5/15/2009
3
Slide 3ILLINOIS - RAILROAD ENGINEERING
Backgroundg
• Inspection of track and track components is a critical, but labor intensive task resulting in large annual operating expenditures
• There are limitations in speed, quality, objectivity, and scope of the current inspection methodsscope of the current inspection methods
• Machine vision provides a robust solution to facilitate• Machine vision provides a robust solution to facilitate more efficient and effective inspection of many track components
5/15/2009
4
Slide 4ILLINOIS - RAILROAD ENGINEERING
What is Machine Vision?• Machine vision consists of recording digital images and videos
and using algorithms to detect certain attributes in these images
• Has advantages and disadvantages as compared to manual visual inspection
– Advantages:Advantages:
• Greater objectivity and reliability
• Increased speed, precision and repeatability
• Data archiving and trending capabilities
– Disadvantages:
• Challenges in providing controlled lighting conditions
• Difficulties coping with unforeseen events
• Higher initial capital cost• Higher initial capital cost
5/15/2009
5
Slide 5ILLINOIS - RAILROAD ENGINEERING
System Outliney
P ti tImageAcquisition
System
MachineVision
Algorithms
Railroad Track
Data AnalysisSystem
PertinentInformation
forRailroadsRailroads
5/15/2009
6
Slide 6ILLINOIS - RAILROAD ENGINEERING
Top Track Related Accident Causes from 2001 2005 (Cl I M i & Sidi )2001-2005 (Class I Mains & Sidings)
1200
1400
1000
1200
Acc
iden
ts
600
800
mbe
r of A
200
400
Tota
l Num
0Broken Rail Wide
GaugeCross Level
Buckled Track
Switch Point
Turnouts (Other)Gauge Level Track Point (Other)
Primary Cause of Accident
5/15/2009
7
Slide 7ILLINOIS - RAILROAD ENGINEERING
Survey of Track Inspection Technologiesy p g
• Existing technologies
– Ultrasonic rail flaw detection
– Eddy current
– Radiography
– Split-load axle
• Emerging technologies• Emerging technologies
– Inertial accelerometers
– Ground Penetrating Radar (GPR)g ( )
– Light Detection And Ranging (LIDAR)
– Machine vision
5/15/2009
8
Slide 8ILLINOIS - RAILROAD ENGINEERING
Survey of Closely Related Technologiesy y g
• Light Detection And Ranging (LIDAR)
– Measures distance by analyzing properties of reflected light
– Used for measurement of tunnel clearances and shoulder ballast profilesballast profiles
• Other Machine-vision systems for track inspection inspect:
– Joint bars
– Elastic rail clips
– Ties
– Rail, tie plates and gauge
5/15/2009
9
Slide 9ILLINOIS - RAILROAD ENGINEERING
Defect Severity Levelsy• Critical
– Detection of such defects constitutes aDetection of such defects constitutes a hazard
– i.e. Buckled track
• Non-critical
– Individual defects are not a hazard, however, detection of several can constitute a critical defect
– i.e. Low crib ballast between a pair of ties
• Symptomatic• Symptomatic
– Do not represent defects, but indicative of a potential problem
– i.e. Shiny spots on rail base not a defect, but indicative of longitudinal rail movement
5/15/2009
10
Slide 10ILLINOIS - RAILROAD ENGINEERING
Inspection Focus• Cut spikes
– Missing– Raised– Inappropriate patterns
• Rail anchors– Missing– Shifting– Inappropriate patterns
• Ballast– Insufficient crib ballast– Identify improper breathing
on curves• Turnouts
– Missing bolts– Missing cotter pins– Switch point inspection
5/15/2009
11
Slide 11ILLINOIS - RAILROAD ENGINEERING
Virtual Track Model• Used to model track
components and their defects for determination of initialfor determination of initial camera views
• Utilized the AREMA Manual and Class I Railroad standardsand Class I Railroad standards
• Incorporated AAR clearance plates
5/15/2009
12
Slide 12ILLINOIS - RAILROAD ENGINEERING
Initial Camera Views
• Spike pattern and rail anchor view
Optimal view for measuring the distance– Optimal view for measuring the distance between the ties and rail anchors
– Used to verify spike heights measured in other viewsother views
• Raised spike view
– Used to determine spike height with respectUsed to determine spike height with respect to the base-of-rail
• Ballast view
f f f– Used for determining the profile of the shoulder ballast
– Also used for measuring the crib ballast level with respect to the top of the ties
5/15/2009
13
Slide 13ILLINOIS - RAILROAD ENGINEERING
Over-the-rail View
• Used for measuring the distance between the spike head and the b f il d if i iki ttbase-of-rail and verifying spiking patterns
• Also used to measure crib ballast level with respect to the top of the tie
5/15/2009
14
Slide 14ILLINOIS - RAILROAD ENGINEERING
Lateral View
• Optimal view for measuring the distance between the rail anchors and th d f th ti d if i h ttthe edge of the ties and verifying anchor patterns
• Used to verify spike heights measured in the over-the-rail view
• Conducive to panorama generationConducive to panorama generation
5/15/2009
15
Slide 15ILLINOIS - RAILROAD ENGINEERING
Data Collection
• Initially used handheld cameras to take imagesg
– Obtained insights on later challenges, such as variability in component appearanceappearance
• Developed Video Track Cart for field data acquisition
– Used on low density track for verifying camera views and experimenting with lighting
• Optimizing lighting is a challenging task
– Reviewed lighting solutions from existing machine-vision systemsexisting machine vision systems
5/15/2009
16
Slide 16ILLINOIS - RAILROAD ENGINEERING
Video Track Cart
• Equipped with two camera mounts for the over-the-rail view and a gauge-side lateral view
• Geared tripod heads for precise adjustment of camera views
• Deep-cycle marine battery supplies power for all electronics
• Rugged laptop resistant to environmental conditions
5/15/2009
17
Slide 17ILLINOIS - RAILROAD ENGINEERING
Video Data Collection• Visit to Advanced Transportation Research and Engineering
Laboratory (ATREL) in Rantoul, IL
14’ t k l– 14’ track panel
– Experimented with differing focal lengths
– Developed video capture procedure for future field testingDeveloped video capture procedure for future field testing
5/15/2009
18
Slide 18ILLINOIS - RAILROAD ENGINEERING
Video Data Collection• Monticello Railway Museum (MRM) visits
– Recorded video from both camera views
• Improved lateral view and over-the-rail view mounts
– Manually measured and catalogued track defects for algorithm verificationalgorithm verification
5/15/2009
19
Slide 19ILLINOIS - RAILROAD ENGINEERING
Lighting Considerationsg g
• Reviewed lighting systems of similar machine-vision systems
University of Central Florida’s track inspection system– University of Central Florida s track inspection system
• Lasers and timed strobe lights
• Sun shields
– ENSCO’s joint bar inspection system
• High-powered xenon lights
• No sun shields
– Georgetown Rail’s Aurora system
L ith li ht filt• Lasers with light filters on cameras
• Lighting solutions are unique to each system’s data collection requirements, with our system requiring even illumination over a large area of track
5/15/2009
20
Slide 20ILLINOIS - RAILROAD ENGINEERING
Algorithm Developmentg p• Initial development used Virtual
Track Model
Simulation of component defects– Simulation of component defects
• Development using still-images
– Changed from tie-based d t ti t il b d d t tidetection to rail-based detection for reliability
– Began using texture information to make algorithms more robustto make algorithms more robust
5/15/2009
21
Slide 21ILLINOIS - RAILROAD ENGINEERING
Track Component Isolationp
• Delineate ties, ballast, and rail using edge detection
• Incorporate texture classification to make edge detection robust
Original image Edge image (base-of-rail highlighted)
• Incorporate texture classification to make edge detection robust– Extract patches of texture from a hand-labeled training image
– Compare new texture patches against known textures
• Use Gabor filter outputs to classify patches from new video frames
Wood patches Ballast patches Metal patches
5/15/2009
22
Slide 22ILLINOIS - RAILROAD ENGINEERING
Base-of-Rail Delineation
Original image from field video
5/15/2009
23
Slide 23ILLINOIS - RAILROAD ENGINEERING
Base-of-Rail Delineation
Strong horizontal edges are detected (candidate base-of-rail edge is shown)
5/15/2009
24
Slide 24ILLINOIS - RAILROAD ENGINEERING
Base-of-Rail Delineation
Surrounding pixels above and below candidate edge are examined for expected textures
5/15/2009
25
Slide 25ILLINOIS - RAILROAD ENGINEERING
Base-of-Rail Delineation
Metal Metal Metal aboveedge
aboveedge
Ballast belowedge
Ballast belowedge
Base-of-rail is confirmed by confirming ballast below the edge and metal above the edge
5/15/2009
26
Slide 26ILLINOIS - RAILROAD ENGINEERING
Tie, Tie Plate, and Spike Detection, , p• Detect ties and tie plates using similar edge/texture method
– Edge detection, verified with texture classification
Original image Delineated components
• Hypothesize the spike locations (and missing spike locations) using prior knowledge of tie plate structure
S ik d ik h l– Spikes and spike holes are located with template filtersS ik h i ht i d– Spike height is measured relative to the base-of-rail
Spikes and spike holes
5/15/2009
27
Slide 27ILLINOIS - RAILROAD ENGINEERING
Rail Anchor Detection
• Uses prior knowledge of anchor location (since tie plate and rail b tt h b d li t d)bottom have been delineated)
• Find parallel edges to achieve anchor detection
– Robust to changes in anchor orientationRobust to changes in anchor orientation
Isolated anchorsIsolated anchors
5/15/2009
28
Slide 28ILLINOIS - RAILROAD ENGINEERING
Component Detection on Field Videop
Field data video
5/15/2009
29
Slide 29ILLINOIS - RAILROAD ENGINEERING
Defect Detection
Original image for spike detection and anchorOriginal image for spike detection and anchor displacement measurement
5/15/2009
30
Slide 30ILLINOIS - RAILROAD ENGINEERING
Defect Detection
Tie plate area identified and tie boundaries located
5/15/2009
31
Slide 31ILLINOIS - RAILROAD ENGINEERING
Defect Detection
Spikes, anchors and spike holes are located
5/15/2009
32
Slide 32ILLINOIS - RAILROAD ENGINEERING
Defect Detection
Defects detected if anchor displacement exceeds a defined threshold
5/15/2009
33
Slide 33ILLINOIS - RAILROAD ENGINEERING
Panoramic Generation
5/15/2009
34
Slide 34ILLINOIS - RAILROAD ENGINEERING
Track Panorama• Visualization of cumulative track components using
panoramic stitching
– Minimizes errors due to lens distortion
Ti / Ti l t d li ti T t PTie / Tie plate delineation on Test Panorama
C t d t ti T t PComponent detection on Test Panorama
5/15/2009
35
Slide 35ILLINOIS - RAILROAD ENGINEERING
System Implementationy p
• Future system likely to be mounted on either
– Track geometry car
– Rail defect detector car
– High-rail vehicle
• Possible initial implementation on defect detector car
Inspections occur with acceptable frequency– Inspections occur with acceptable frequency
– Crews better trained with advanced inspection equipment
– Already equipped for massive data storage and uploady q pp g p
• The system will be adapted for high-rail vehicle use
5/15/2009
36
Slide 36ILLINOIS - RAILROAD ENGINEERING
Future Work• Plan for 2009
– Continue testing spike and anchor algorithms on field-acquired video
– Develop an approach to detecting adequate crib ballast level
– Initiate study on methods to inspect for turnout defects
Begin lighting experimentation– Begin lighting experimentation
– Record more videos at MRM and on other local tracks
• Approach for 2010
– Investigate cameras which can run at higher speeds and in a greater variety of environmental conditions
– Develop and test crib ballast profile inspection algorithmsp p p g
– Develop and test algorithms for turnout defects
– Adapt acquisition system for trial runs on a track vehicle
5/15/2009
37
Slide 37ILLINOIS - RAILROAD ENGINEERING
Summary and Discussiony• Current inspection tasks
– Raised or missing cut spikesg p
– Displaced or missing rail anchors
• Future inspection tasks
– Low crib ballast and curve breathing
– Turnouts and other special trackwork
S t b fit• System benefits
– Detailed trending of track health over time and space for predictive maintenance planning
– Improved understanding of the contributions of individual track components on track behavior
• Can be used to help develop an advanced failure• Can be used to help develop an advanced failure prediction model
5/15/2009
38
Slide 38ILLINOIS - RAILROAD ENGINEERING
Acknowledgementsg
Research Sponsors:AAR Technology Scanning Program and the NEXTRANS Centergy g g
Computer Vision and Robotics Laboratory:Narendra Ahuja, Professor, Beckman Institute
John M. Hart, Senior Research Engineer, Beckman InstituteJohn M. Hart, Senior Research Engineer, Beckman Institute
Esther Resendiz, Graduate Student, Electrical and Computer Engineering
Railroad Engineering Program:Chris Barkan Associate Professor Director of Railroad Engineering ProgramChris Barkan, Associate Professor, Director of Railroad Engineering Program
Riley Edwards, Lecturer, Railroad Engineering Program
Mike Wnek, Undergraduate Student, Railroad Engineering Program
Brennan Caughron Undergraduate Student Railroad Engineering ProgramBrennan Caughron, Undergraduate Student, Railroad Engineering Program
Adam Borhart, Undergraduate Student, Railroad Engineering Program
Th k l t l f CN NS BNSF UP d th l fThanks also to people from CN, NS, BNSF, UP and many other people for their input and assistance, as well as the Monticello Railway Museum for data collection assistance
Recommended