Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Boston Imaging and Vision Group Infrared Vision
Image Processing in Infrared Cameras
Marc Norvig Principal Engineer FLIR Systems, Inc.
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Hashagen, J. (September 2014). SWIR Applications and Challenges: A Primer. EuroPhotonics http://www.photonics.com/Article.aspx?AID=56646
Reflected IR Thermal IR
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Reflected Infrared
Photos by Nick Spiker https://www.facebook.com/invisiblelightimages/
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Long-Wave (8 – 14 μm)
Photos Courtesy FLIR Systems, Inc.
Thermal Infrared Camera Comparison Mid-Wave (3 – 5 μm)
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Long-Wave (8 – 14 μm) + Room temperature operation
Photos Courtesy FLIR Systems, Inc.
Thermal Infrared Camera Comparison Mid-Wave (3 – 5 μm) – Cyrogenic (77˚K to 155˚K)
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Long-Wave (8 – 14 μm) + Room temperature operation + Compact + Lightweight + Lower power + Lower price + Instant imaging + 20-plus year service interval + Very good radiometric accuracy
Thermal Infrared Camera Comparison Mid-Wave (3 – 5 μm) – Cyrogenic (77˚K to 155˚K) – Size – Weight – Power – Cost – Cooldown time – 10 to 15,000 hour service interval + Best thermal image quality + Best sensitivity + High contrast + Long distance viewing + Higher frame rates
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
offset(x,y) gain(x,y)
Sensor
Temp
Image
Memory
Vshutter (x,y)
OV(T3)
1
GV(T2)
Raw
Image
Data-
+
+
+ -Corrected
Infrared
Image
Shutter
Sensor Corrections
- Mechanical shutter captures “dark image” for subtraction
- Each pixel has a different bias point and sensitivity due to manufacturing process variation
- Corrected using per-pixel offset and gain tables
- All pixels now have uniform response to relative temperatures
- For accurate temperature measurement we need to correct for sensor temperature variation
Corrected Image
Sensor Corrections
Raw Image
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Sosnowski, T. et al. (2010). Processing of the Image from Infrared Focal Plane Array Using FPGA-based System. MIXDES Proceedings of the 17th International Conference, 581-586
Corrected Image
Sensor Corrections
Raw Image
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Budzier, H. & Gerlach, G. (2015). Calibration of uncooled thermal infrared cameras. Journal of Sensors and Sensor Systems,4(1), 187-197
Defective Pixel Replacement
- Some pixels can’t be made to match others (dead, excessive gain, hot, blinking)
- These pixels are marked as defective at factory calibration
- The bad pixels must be replaced with a better value for image viewing
- Approach this as either a denoising problem or an inpainting problem
- Simplest solution is nearest neighbor replacement - Need method to choose between equidistant neighbors
- Spatial filter over a small local neighborhood - If any neighborhood pixel is also defective it must be excluded (remember to renormalize the kernel)
- Simple mean of a few neighbors is surprisingly effective
- Median filter never seems to turn out as well as you want it to
- Adaptive, gradient-based, and patch-based methods preserve image structure - Probably not be worth the computational effort
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Image Enhancement
- Corrected sensor output is 14-bit but display is 8-10 bits
- Typical scenes have high dynamic range (hot car engine on a winter day) - Most of the pixels in this scene will be the same temperature (low contrast)
- Combined bit length reduction and contrast enhancement is required
- Contrast Limited Adaptive Histogram Equalization (CLAHE) and its derivatives have emerged as the best solutions.
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Image Enhancement
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Image Enhancement
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Original Histogram EQ
Image Enhancement
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Original CLAHE
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Visible–Infrared Image Fusion: Registration
- Image-driven registration requires features that appear in both visible and IR images - Finding good joint feature detectors is an open problem
Lui, F. & Seipel, S. (2015). Infrared-visible image registration for augmented reality-based thermographic building diagnostics. Visualization in Engineering,3(16)
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Visible–Infrared Image Fusion: Image Combining
- Combining can occur at various levels - Pixel-level
- Feature-level
- Object-level
- Transform Domain Techniques - Pyramids (Gaussian, Laplacian, Gradient)
- Wavelets, Curvelets
- Spatial Domain Techniques - Weighted blending (global or locally adaptive weights)
- PCA
- Gram-Schmidt
- High-pass filter
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Feature-based Fusion Example
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Infrared Camera 50% Blend Visible Camera
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Vis + IR Features
Infrared Camera Shared Features
Visible Features Infrared Features
Visible Camera
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Infrared Camera 50% Blend
Visible Features Infrared Features
Visible Camera
Grayscale Fused
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Infrared Camera False-Color Fused
Visible Features Infrared Features
Visible Camera
Grayscale Fused
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
High-pass Filter Fusion Example (FLIR MSX®)
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Visible Camera Infrared Camera High-pass filter
Boston Imaging and Vision Group Infrared Vision
[email protected] 06/02/2016 Image Processing in Infrared Cameras
Visible Camera Infrared Camera MSX
®
Visible Camera Infrared Camera MSX
®
Boston Imaging and Vision Group Infrared Vision
Thank you
[email protected] 06/02/2016 Image Processing in Infrared Cameras