PASSIVE MILLIMETER WAVE IMAGING WITH SUPER-RESOLUTION: Application to Aviation safety

in extremely poor visibility

Dr. Isaiah M. Blankson, Research &Technology

DirectorateNASA Glenn Research Center,

Cleveland, OH

Presented at Institute of Mathematics and its Applications, University of Minnesota: May 5, 2001

PASSIVE MILLIMETER-WAVE IMAGING (PMMWI) PROJECT: OBJECTIVES &GOALS

• Explore the potential application of Radiometric sensors to alleviate atmospheric hazards to aviation.

• Develop/design an all-weather Radiometer operating at 94 GHz which employs Super-Resolution Algorithms for a Real -Time rapid image inversion processing, and is capable of producing very high resolution images ( recover scene-spatial frequencies ~or >10 X {Rayleigh Limit}).

• Construct a functioning system capable of Ground and Airborne Applications

Aeronautics & Space Transportation TechnologyStrategic Roadmap

Source: Aeronautics & Space Transportation Technology / Strategic Roadmap, NASA GRC

Space Applications

Remote Sensing of Planetary Surfaces

• Structurally Embeddable• Low Power Applications• Payload Reduction• Compact

Pillar One:

Global Civil

Aviation

Safety 2000 2010Human-Related Factors

Increase AirportCapacity

Improve Navigational Aids

Reduce AccidentRates 10X

Millimeter Wave Radiometryat 94 GHz with

Super-Resolution

Electromagnetic Spectrum

1102104106108101010121014101610181020

Millimeter & Sub-Millimeter Wave Region

0.03

Å

3 Å

300

Å

0.3

m

3 m

300

m

3 cm

3 m

300

m

30 k

m

300

km

3 km

30 m

30 c

m

0.3

cm

30

m

30 Å

0.3

Å0.4 m - 0.7 m

Visible

Infrared

Gamma Ray X Ray Radar

Radio Bands Audio AC

UV Microwave

Wavelength

Black Represents Atmospheric Attenuation

= 1

= 0

Frequency (Hz)

Heating Heating

Dissociation

PhotoIonization

PhotoDissociation

ElectronShifts

Electromagnetic Field Fluctuations

Rain & FogAttenuation

Spherics

Interaction Mechanismsor Phenomena Detected

MolecularVibration

MolecularVibration

Cosmic NoiseRadioAstronomy

Source: Manual of Remote Sensing, Vol. 1, First Edition, 1975

Black Body RadiationS

pec

tral

Exi

tan

ce

(W c

m-2

m-1 )

Wavelength (m)

1 2 3 4 5 6

30

50

10

40

20

0

1000 °K

1200 °K

1400 °K

1600 °K

1800 °K

2000 °K

22

2 22 kT

c

kTfBbb

Rayleigh-Jeans

Approximation Holds

Microwave

Infra-RedNear-Infrared

MillimeterSub-millimeter

1015 1013 1011 109 107 105

Frequency (Hz)

Rel

ati

ve

Rad

ian

ce

Why Passive Millimeter-Wave Imaging?

• All natural objects whose temperatures are above absolute zero emit passive millimeter-wave radiation.

• Millimeter-waves are much more effective (lower attenuation) than infrared in poor weather conditions such as fog, clouds, snow, dust-storms and rain. Also, images produced by passive millimeter-waves have natural appearances.

• The amount of radiation emitted in the millimeter-wave range is 108 times smaller than the amount emitted in the infrared range.

• However, current millimeter-wave receivers have at least 105 times better noise performance than infrared detectors and the temperature contrast recovers the remaining 103.

• This makes millimeter-wave imaging comparable in performance with current infrared systems.

• Electromagnetic radiation windows occur at 35 GHz, 94 GHz, 140 GHz, and 220 GHz.

• Choice of frequency depends on specific application

APPLICATIONS

Advances in Inverse Problem Solutions for :

— Geological Explorations

— Remote Sensing of Vegetation & Soil Conditions

— Non-Invasive Brain Volumetric Mapping

— Airport Safety

— All-Weather Vision

— Fused Sensor Imaging - Component

— Medical Diagnostics

— Plasma Diagnostics

— In-Situ Non-Destructive Testing

( Composites : Voids, Delaminations )

— Defense Applications

— Environmental

REMOTE SENSING

(Terrestrial & Extra-terrestrial)

DIAGNOSTICS

GENERAL

SIDE - BENEFITS

RADIOMETER CONCEPT

ELECTRONICS

BEAM Controller

COLLECTORANTENNA

……..

……….

[[[[**33

SUPER-RESOLUTIONSoftware

COMPUTER

Aviation Safety Application

Sky Radiation

Ground and VegetationEmissions

Metal Reflections of Cold Sky Radiation

Passive Radiometric Sensing - Concept

Side LobeAtmosphericContributions

Atmosphere

Antenna

Beam Width

RadiometerReceiver

VO

Side LobeBackgroundContribution

UpwardAtmospheric

Emission

Antenna PowerPattern

BUPScattered Radiation

Atmospheric Loss

Target Observation Cell

BB Self Emission

DownwardAtmosphericEmission

BDN

BSC

LATM

LATM

BB

LATM

BSC

Conceptual Diagram of 2-D Phased Array Radiometer

1 complete scan 1 video frame

ImageProcessor Receiver

Radiating Element

Low Noise Amplifier

PhaseShifter

Beam SteeringComputer

Direct Measurement Result

GOAL: Best true “Scene “ R e c o v e r y

INVERSE Problem Solution

EMR-Properties of Propagation media

Mathematical Processing of Measured Data

TRUE Scene

“True” Scene..Recovery

Why Super-Resolution?

• Images acquired from practical sensing operations usually suffer from poor resolution due to the finite size limitations of the antenna, or the lens, and the consequent imposition of diffraction limits.

• The fundamental operation underlying the sensing operation is the “low-pass” filtering effect due to the finite size of the antenna lens.

• The portions of the scene that are lost by the imaging system are the fine details (high frequency spatial spectral components) that accurately describe the object in the scene.

• For super-resolution, spatial spectral extrapolation is needed.

• Some studies have indicated that the cost of an imager increases as (1/Resolution) raised to the power 2.5.

• Hence, a possible two-fold improvement in resolution by super-resolution processing, roughly translates into a cost reduction of an imager by more than 5 times.

Regularization/Reduction MethodNoise

PSFA

Imaging System

Hypothetical operator

RI0

Original image

IDegraded Image

Î0

The Best Fit Image

I = AI0 + Î0 = R AI0 + RRI

The mathematical model of the method is shown above where: I0 = is the original image,I = is the degraded image = AI0 + A = the measured PSF of the imaging system, is the measured mean square noise of the system, R = is a hypothetical operator to be found in order to obtain the best fit image.RAI0 = the noise-free output.

The main idea of this method of reduction is to find an operator R such that the following conditions are satisfied:

=maximum required mean square noise of the system =maximum allowed error for the mean square difference between RA and E (the identity matrix) , A* is the transposed matrix of A. is a parameter to be found such that the set of conditions (2.3), (2.4) and (2.5) is satisfied. These conditions form a constrained optimization problem.

(2.5) and

(2.4)

(2.3)

22

*1*

2

min

RERAR

AEAAR

R

Mathematics of Inversion

Tikhonov - Pytiev Regularization

Tikhonov-Regularization

CONSTRAINT

LINEAR INVERSION :f = (A* A + H )-1 A* g

f = (A* A + I )-1 A* g

Errors: e1, e2, e3,…em

equal weights

related by

k

kee 22

f = ( A* R-1 R-1 A + I ) A* R-1 R-1 g

R operator is chosen so as to make

the elements of the transformed error

vector “e “ mutually independent and

possessing the same weight

R A f = R g + R e

A f = g + e

Minimize (f * ·H ·f )) subject to constraint

| Re | 2 =constant = 2

R operator is subject to 3 conditions :

1..angle target size within limits ( Fourier image size

within the recording system’s MTF)

2..signal to noise ratio no more than 3:1(preferably 5:1)

3..antenna scanning limited to avoid Gain degradation

and Grating Lobes

TIKHONOV - PYTIEV Regularization

Assembling the Building Blocks...

Antenna Design

f = ( A* R-1 R-1 A + I ) A* R-1 R-1 g

Tikhonov - Pytiev Regularization

0 200 400 600 800

0

-5

-10

-15

Spatial Frequency M

TF

Mag

nitu

de (

dB)

fRayleigh = 45 fspatial = 450

Post - Facto Determination of “ “

Constraints

Knowledge of Measurement Errors { Including Error correlations }

Precise Modeling of Media { Characterization of EM-Interactions } Optimization

Multi-Disciplinary Approach

PHYSICS

EM-Media Properties,

Field Measurements

MATHEMATICS

Forward / Inverse Problem

Regularization

MATERIALS

Radome, Substrates,

Bond-Films

COMPUTERS &

INFORMATION

SCIENCES

Neural Network

Adaptive Algorithms,

Parallel Processing,

Simulations, Optimization

ANTENNA

Design / FabricationHIGH FREQUENCY I.C

Modules, Receiver

STRUCTURAL

MECHANICS

Thermal Cycling,

Rigidity,

Stress, Durability..

RADIOMETER

Design /Fabrication

LAB. Measurements

Radiometer Noise

N.F. Measurements

Antenna Characterization

Future Radiometer Trends: Multi-Layered Integrated System

Source: University of Michigan http://www.eecs.umich.edu/RADLAB/katehi.dir/cism/index.html

Integration

• Each Layer:• Amplification• Phase Shifting• Combining

Has An Integrated Function• High Monolithic

• Permanent Layer Bonding

• Device Integration• Monolithically• Flip-Chip• Lift-Off

• Monolithic Packaging

• Integrated Processors

Combining Network

Heat Sink

Scanning ArrayAntenna

LNAsPhase Shifters

Multi-layer Microstrip PatchArray Assembly

Substrate(Silicon Wafer,Teflon)

Microstrip Patch

Aperture CouplingSlots

Stripline Feed

Combining Network

MMICs - LNAs Phase Shifters

RESOURCES: SELECTED BIBLIOGRAPHY

• Charles W. Groetsch, “Inverse Problems in the Mathematical Sciences”. VIEWEG (Bertelsman Publishing Group International) 1993

• S. Twomey, “Introduction to Mathematics of Inversion in Remote Sensing and Indirect Measurements”. Dover Publications Inc., 1977

• “Ill-Posed Problems in the Natural Sciences”. Advances in Science and Technology in the USSR, Mathematics and Mechanics Series. (Edited by A. N. Tikhonov and A.V. Goncharsky) MIR Publishers, Moscow. 1987

• “Inverse Problems in scattering and Imaging”. NATO Advanced Research Workshop, 1991. ( Edited by: E. R. Pike and M. Bertero) Adam Hilger Publishers (IOP), Bristol, England.

• M. Bertero and P. Boccacci, “Introduction to Inverse Problems in Imaging”. Institute of Physics Publishing Ltd., 1998

• “Mathematics of Profile Inversion”. Workshop Proceedings (Edited by: L. Colin) NASA Ames Research Center, Moffett Field, California. July 12 - 16, 1971. (NASA TMX-62-150)