4
1 An All-COTS Automotive Radar for SAR Imaging and Target Classification Shane Flandermeyer, Russell Kenney, Hyeri Kim, Kurt Konyalioglu, Alex Pham, Callin Schone Department of Electrical and Computer Engineering Advanced Radar Research Center University of Oklahoma Norman, OK 73019 Email: [email protected] Abstract—In this concept paper, a radar concept developed by a team from the University of Oklahoma using a fully COTS FMCW automotive radar module produced by Texas Instruments is presented. The radar system is designed to perform synthetic aperture radar (SAR) as a method for locating targets in three- dimensional space. The basics of FMCW processing are given, along with the fundamentals of SAR processing. A control system and mechanical positioner are formulated for the purpose of positioning the radar with precision. Additionally, the basics of deep-learning networks for radar target classification are discussed. I. I NTRODUCTION A S vehicles become more and more sophisticated, em- ploying features such as blind-spot detection (BSD), lane-change assist (LCA), adaptive cruise control (ACC), and other autonomous functions, sensors monitoring the road for vehicles and other obstacles must become more and more robust [1]. Because sensing and classification of these ob- stacles potentially requires fine sensing resolution, a large radar signal bandwidth is desirable. In lower frequency RF bands, the spectrum is becoming more and more congested as new radar and communications devices compete for more bandwidth. This has motivated the automotive radar industry to jump to higher frequency bands, starting with a narrow ISM band at 24 GHz and more recently a 4 GHz band from 77- 81 GHz. This band is ideal for automotive radar because the relatively short scale of distance required for target sensing negates the path loss and attenuation penalty of moving to higher frequency while allowing for wideband signals that can give range resolutions as precise as 4 cm. For similar reasons, automotive radar operating in the 77-81 GHz band is also ideal for the Radar2020 Radar Challenge. Because the scene size will be small, path loss and attenuation of the signal will not reduce radar performance nearly as much as a large scene size would. The ability to operate with a wide bandwidth will also allow targets to be determined in location with much finer precision, and targets closer together have a higher likelihood of being distinguished from one another. Additionally, the wideband automotive radar remains compliant with FCC Chapter 15 [2]. In this concept paper, the radar overview produced by the University of Oklahoma team at the Advanced Radar Research Center is provided. A COTS automotive radar module has been selected for use in this competition (Sections II and III). The radar will be used for 2D and 3D image formation using synthetic aperture radar (SAR) with assistance from a mechanical positioner and control system used to form the synthetic aperture (Sections IV and V). The radar data can also be co-processed in a ground-moving target indication (GMTI) algorithm to obtain information about the location and velocity of moving targets within the scene (Section IV). Finally, using a deep-learning method for target classification being researched at the University of Oklahoma, the radar data will be processed to attempt to classify the targets as either spherical or corner reflectors (Section VI). II. FMCW OPERATION The University of Oklahoma will be using a Frequency Modulated Continuous Wave (FMCW) radar system for the IEEE Radar 2020 Challenge. FMCW was chosen to ensure the system will be able to operate and collect useful data in the close proximity scene since it is capable of collecting range and doppler information without a blind range due to the nature of its continuous operation. A Linear Frequency Modulated (LFM) waveform is used to transmit a chirp over the desired bandwidth. This LFM waveform, at bandpass, is given by x(t)= exp ( β τ t 2 ) exp ( j Ω(t - t b ) ) , (1) where β is the bandwidth of the signal, t is the fast-time axis, τ is the pulsewidth of the transmitted pulse, and Ω is the carrier frequency. The received signal is a time delayed version of the transmitted waveform, arriving at a later time of t b . The received signal, given by ˜ x(t)= ρ exp ( β τ (t - t b ) 2 ) exp ( j Ω(t - t b ) ) , (2) where ρ is the reflectivity of the scatterer, is then mixed with a reference signal, typically chosen to be the signal that would be received from some reference range, received at time t 0 . The resulting signal is said to be ”deramped” and is given by y(t)= ρ exp ( -j 4πR b λ ) exp ( -2β τ δt b (t-t 0 ) ) exp ( β τ (δt b ) 2 ) . (3) Here, λ is the wavelength of the carrier, R b is the range to the return received at t b , and δt b is the time difference between the target and reference range. The second exponential in equation 3 represents a constant frequency complex sinusoid

An All-COTS Automotive Radar for SAR Imaging and Target ...ieee-aess.org/sites/ieee-aess.org/files/08_Schone.pdfSAR processing will be used to achieve high cross-range resolution to

  • Upload
    others

  • View
    7

  • Download
    0

Embed Size (px)

Citation preview

Page 1: An All-COTS Automotive Radar for SAR Imaging and Target ...ieee-aess.org/sites/ieee-aess.org/files/08_Schone.pdfSAR processing will be used to achieve high cross-range resolution to

1

An All-COTS Automotive Radar for SAR Imagingand Target Classification

Shane Flandermeyer, Russell Kenney, Hyeri Kim, Kurt Konyalioglu, Alex Pham, Callin SchoneDepartment of Electrical and Computer Engineering

Advanced Radar Research CenterUniversity of Oklahoma

Norman, OK 73019Email: [email protected]

Abstract—In this concept paper, a radar concept developedby a team from the University of Oklahoma using a fully COTSFMCW automotive radar module produced by Texas Instrumentsis presented. The radar system is designed to perform syntheticaperture radar (SAR) as a method for locating targets in three-dimensional space. The basics of FMCW processing are given,along with the fundamentals of SAR processing. A control systemand mechanical positioner are formulated for the purpose ofpositioning the radar with precision. Additionally, the basicsof deep-learning networks for radar target classification arediscussed.

I. INTRODUCTION

AS vehicles become more and more sophisticated, em-ploying features such as blind-spot detection (BSD),

lane-change assist (LCA), adaptive cruise control (ACC), andother autonomous functions, sensors monitoring the road forvehicles and other obstacles must become more and morerobust [1]. Because sensing and classification of these ob-stacles potentially requires fine sensing resolution, a largeradar signal bandwidth is desirable. In lower frequency RFbands, the spectrum is becoming more and more congestedas new radar and communications devices compete for morebandwidth. This has motivated the automotive radar industryto jump to higher frequency bands, starting with a narrow ISMband at 24 GHz and more recently a 4 GHz band from 77-81 GHz. This band is ideal for automotive radar because therelatively short scale of distance required for target sensingnegates the path loss and attenuation penalty of moving tohigher frequency while allowing for wideband signals that cangive range resolutions as precise as 4 cm.

For similar reasons, automotive radar operating in the 77-81GHz band is also ideal for the Radar2020 Radar Challenge.Because the scene size will be small, path loss and attenuationof the signal will not reduce radar performance nearly as muchas a large scene size would. The ability to operate with awide bandwidth will also allow targets to be determined inlocation with much finer precision, and targets closer togetherhave a higher likelihood of being distinguished from oneanother. Additionally, the wideband automotive radar remainscompliant with FCC Chapter 15 [2].

In this concept paper, the radar overview produced by theUniversity of Oklahoma team at the Advanced Radar ResearchCenter is provided. A COTS automotive radar module hasbeen selected for use in this competition (Sections II andIII). The radar will be used for 2D and 3D image formation

using synthetic aperture radar (SAR) with assistance from amechanical positioner and control system used to form thesynthetic aperture (Sections IV and V). The radar data canalso be co-processed in a ground-moving target indication(GMTI) algorithm to obtain information about the locationand velocity of moving targets within the scene (Section IV).Finally, using a deep-learning method for target classificationbeing researched at the University of Oklahoma, the radar datawill be processed to attempt to classify the targets as eitherspherical or corner reflectors (Section VI).

II. FMCW OPERATION

The University of Oklahoma will be using a FrequencyModulated Continuous Wave (FMCW) radar system for theIEEE Radar 2020 Challenge. FMCW was chosen to ensurethe system will be able to operate and collect useful datain the close proximity scene since it is capable of collectingrange and doppler information without a blind range due tothe nature of its continuous operation.

A Linear Frequency Modulated (LFM) waveform is usedto transmit a chirp over the desired bandwidth. This LFMwaveform, at bandpass, is given by

x(t) = exp(jπβ

τt2)

exp(jΩ(t− tb)

), (1)

where β is the bandwidth of the signal, t is the fast-timeaxis, τ is the pulsewidth of the transmitted pulse, and Ω isthe carrier frequency. The received signal is a time delayedversion of the transmitted waveform, arriving at a later timeof tb. The received signal, given by

x(t) = ρ exp(jπβ

τ(t− tb)

2)

exp(jΩ(t− tb)

), (2)

where ρ is the reflectivity of the scatterer, is then mixed witha reference signal, typically chosen to be the signal that wouldbe received from some reference range, received at time t0.The resulting signal is said to be ”deramped” and is given by

y(t) = ρ exp(−j 4πRb

λ

)exp

(−2jπ

β

τδtb(t−t0)

)exp

(jπβ

τ(δtb)

2).

(3)Here, λ is the wavelength of the carrier, Rb is the range to

the return received at tb, and δtb is the time difference betweenthe target and reference range. The second exponential inequation 3 represents a constant frequency complex sinusoid

Page 2: An All-COTS Automotive Radar for SAR Imaging and Target ...ieee-aess.org/sites/ieee-aess.org/files/08_Schone.pdfSAR processing will be used to achieve high cross-range resolution to

2

Fig. 1: Stretch processing.

with frequency dependent on the target’s distance from thereference. This range-dependent beat frequency can be usedto create a range profile by taking a Fourier Transform of thereceived signal. A thorough derivation of stretch processing isgiven in [3]. Fig. 1 shows the received signals from a target atthe near edge of the scene and one at the far edge of the scene.These two signals are mixed with the reference signal, inblack, which downconverts them to baseband where they havea constant frequency related to their range. In order to gatherinformation on the Doppler shift of a target, several pulseswill be transmitted and processed using a discrete FourierTransform (DFT).

III. TI AUTOMOTIVE RADAR MODULE

The hardware we are using is a commercially available au-tomotive FMCW radar system (Texas Instruments AWR1642)packaged in its evaluation module (AWR1642BOOST). Thesystem is small enough to be easily mounted to a linearactuator system that will move the radar to create the syntheticaperture required for SAR processing (see Section IV).

The AWR1642, shown in Fig. 2, is an integrated single-chip FMCW radar sensor capable of operation in the 76- to81-GHz band. The AWR1642 was chosen due to its low-power consumption, small form factor, and high bandwidth.It integrates the DSP subsystem, which contains TI’s high-performance C674x DSP for the Radar Signal processing.The device includes an ARM R4F-based processor subsystem,which allows radio configuration, control, and calibration.Custom programming model changes can enable a wide vari-ety of sensor implementation with the possibility of dynamicreconfiguration for implementing a multimode sensor. Theradar module is equipped with 2 transmit channels and a 4receive channels in a linear array. Having multiple receivechannels allows for digital beamforming, as well as the im-plementation of several gmti techniques.

To interface with the AWR1642, the Texas InstrumentsDCA1000EVM capture card will be used. The capture card en-ables streaming of raw ADC data over an ethernet connectionto a PC for post-processing. The design of this capture card

Fig. 2: DCA1000EVM capture card (left) and theAWR1642BOOST module (right).

is based on the Lattice FPGA LFE5UM85F-8BG381I withDDR3L. The raw ADC data is stored to the computer as a .binfile, which is then parsed and post-processed in MATLAB.

IV. SAR IMAGING AND MOVING TARGET DETECTION

A radar’s range-resolution is governed by the bandwidthof the transmit waveform, whereas the angular resolution islimited by the physical size of the antenna aperture. Typicallyin airborne radar applications, hardware size is limited by thephysical space allotted to the radar system. This limitationgives rise to the use of synthetic aperture radar, which usesthe inherent motion of the airborne platform to create asynthetic array by sending pulses over the duration of somecoherent processing interval (CPI). The pulsed data can then beprocessed using an imaging algorithms such as Backprojectionor Polar Format Algorithm [4][5], giving rise to high-fidelityimages of the scene of interest. In order for the algorithmsto coherently process data, precise knowledge of the antennalocation is required. For airborne applications, this requiresthe use of high quality IMUs and GPSs. For a ground-basedsystem imaging a small volume, the platform can be affixedto a linear actuator, driven by high quality motors with somemethod of tracking precise position (such as encoders or astep motor driver) to deliver the precision needed for coherentprocessing. Yanik and Torlak [6] have shown the feasibility ofusing an automotive radar module for high resolution imagingof concealed targets using SAR processing. To complementhigh resolution imaging of the scene, moving target indicationwill be performed to detect moving targets within the scene.

SAR processing will be used to achieve high cross-rangeresolution to complement the radar’s high range-resolution.Range-resolution for an lLFM waveform is given by

∆R =c

2β(4)

where c is the speed of light and β is the bandwidth ofthe waveform. Based on this equation, the automotive radarmodule has a maximum range-resolution of roughly 3.75centimeters. Cross-range resolution for a SAR system is givenby

∆CR =λR

2D(5)

where λ is the wavelength of the carrier, R is the range tothe scatterer of interest, and D is the length of the synthetic

Page 3: An All-COTS Automotive Radar for SAR Imaging and Target ...ieee-aess.org/sites/ieee-aess.org/files/08_Schone.pdfSAR processing will be used to achieve high cross-range resolution to

3

aperture. A synthetic aperture length of one meter will resultin a cross-range resolution of roughly 3.8 centimeters atthe scenes furthest point of 20 meters, resulting in squarepixels. Due to the range dependence in equation 5, cross-rangeresolution is spatially variant, meaning that the resolutioncloser to the radar will be significantly smaller than 3.8centimeters. In traditional SAR systems, the range to thescene center is large enough that the spatial variation canbe ignored, but here, some consideration must be made onhow to handle this issue. Using the radar module affixed toa single linear actuator will result in high range-resolutionand high resolution in the horizontal plane of the target area,but the vertical plane will have low resolution. If the radarmodule were positioned such that the array were orientedvertically, beamforming could be performed to achieve higherresolution in the vertical dimension, but this resolution wouldstill be significantly worse than the horizontal dimension asthe beamwidth of the patch array on-board the radar is toolarge. As Yanik and Torlak demonstrated, the use of twoorthogonal linear actuators provides high resolution in bothcross-range dimensions. Using multiple actuators, we will beable to achieve approximately equal resolution in all threedimensions, allowing for a volumetric image of the scene tobe created.

One major advantage of the automotive radar’s built-inantenna array is the ability to perform one of several differentGMTI algorithms. GMTI techniques work by filtering outclutter, leaving returns that only contain a signal from movingtargets. The two GMTI techniques of interest for this projectare the displaced phase center antenna (DPCA) technique andalong-track SAR interferometry (AT-InSAR) [3][7]. DPCAand AT-InSAR require accurate knowledge of antenna loca-tions on each pulse, as well as the ability to send out each pulseat exactly the desired location on each pulse. DPCA and AT-InSAR both require that the physical array is aligned along thedirection of motion, so the scheme for transmitting pulses willhave to be chosen such that this is satisfied while still movingin two dimensions. Implementation of these techniques willalso be limited by the control system’s ability to provide thenecessary precision and accuracy.

V. MECHANICAL POSITIONING AND CONTROL SYSTEM

As discussed in Section IV, focusing of a SAR imagerequires that the position of the radar be known with anaccuracy on the order of the wavelength of the transmitfrequency at every pulse. The transmit frequency of the TIautomotive radar is 77+ GHz, resulting in a wavelength ofless than 4 mm. Consequently, the position of the radar mustbe known at each pulse with an accuracy of less than 1 mm.In many systems, the position of the radar can be determinedusing an inertial measurement unit (IMU). However, evenwith expensive, state-of-the-art IMUs, positional accuracy isdependent on another ”ground truth” measurement (usuallyGPS) to correct for gyroscope and accelerometer biases, andthe position measurements made by the IMU even with GPScorrection are typically only accurate on the centimeter orderof magnitude [8].

Fig. 3: The linear actuator with slide to be used in constructingthe synthetic array positioner.

Fig. 4: The TB67S249FTG stepper motor controller.

Because of the stringent requirements on the precision ofthe position measurement, an IMU with GPS correction isnot suitable for this application. Fortunately, the aperture sizerequired to generate an image with the desired resolution isquite small, on the order of one meter. This smaller scale ofmotion allows the radar to be moved using a fixed positionerrather than an independently moving vehicle. The solutionwe plan to investigate is to connect the radar module to a1.5 meter-long linear actuator with a slide, similar to theone shown in Fig. 3. Depending on time considerations, 3of these actuators could be constructed into a 2-D positioner,allowing the radar to be arbitrarily moved to any point ina 1.5×1.5 m2 grid. The linear actuators will be operatedusing stepper motors, which allow for highly predictable andaccurate positioning without using an encoder for feedback.

The stepper motor allows for the rotation angle of the motorshaft to be set at precise increments (often in increments of1.8°). However, they are more challenging to control thanordinary DC motors. To simplify the design of the positioner,stepper motor drivers, such as the TB67S249FTG (see Fig.4) will be employed. The motor driver abstracts away thesomewhat complex control of the phases within the motor andallows the motor to be fully controlled by two digital inputs:a direction pin, changing the direction of rotation, and a stepclock, which causes the motor to rotate by a single step oneach rising edge in the desired direction.

Depending on the measured open-loop fine positioningcapability of the stepper motors, an IMU may be incorporatedto the positioning system to allow for some sensor feedback.The IMU measurement can be fused with the stepper motorstep count using an algorithm like the Kalman Filter. The

Page 4: An All-COTS Automotive Radar for SAR Imaging and Target ...ieee-aess.org/sites/ieee-aess.org/files/08_Schone.pdfSAR processing will be used to achieve high cross-range resolution to

4

Kalman Filter is a state-space control algorithm that developsthe maximum likelihood estimate of a system state (in thiscase, the position of the radar in a 2-D space) using possiblyinaccurate sensors and a system that slightly deviates from amathematical model. The mathematics of the Kalman Filterare developed extensively in [9].

The digital control of the stepper motors and the datacollection from an IMU, if it is required, will be performedby a high-performance microcontroller unit (MCU) such asthe TI LAUNCHPAD.

VI. DEEP LEARNING TARGET RECOGNITION

In this project, Convolutional Neural Networks (CNNs) willbe employed to classify targets. Unlike traditional program-ming that requires a known algorithm developed by a human,machine learning develops a model by training itself usingdatasets that consist of input data and pre-determined outputlabels. Once trained, the model is able to determine the outputlabel autonomously using only input data, or an ”image”.

CNNs are one of the machine learning methods that arewidely employed in image classification. In general, the CNNsmodel is the combination of convolutional layers, poolinglayers, an activation function, and fully-connected layers.The convolutional layer is utilized to extract the features,the pooling layer is employed to down-sample the data,the activation function applies a threshold to the data, andthe fully-connected layer computes the class scores for eachclassification category. The final label can then be determinedusing the maximum computed score.

One training iteration process consists of a forward pass,loss calculation, backward pass, and weight update. Randomweights and biases are assigned to initialize the network.Loss is then calculated using a cost function such as crossentropy. Back propagation is performed to reduce the lossand weights and biases are updated for each layer. Throughthis iterative process, the final model would be determinedas the weights and biases converge to some point [10]. Inthis method, collecting adequate training datasets is crucialfor constructing a high performance model.

In this study, using the FMCW radar, baseband signals canbe obtained by beating the received signal with the transmittedsignal. The short time Fourier Transform (STFT) can thenbe applied to the deramped signal. After the STFT, twospectrograms corresponding to the up-ramp and down-rampsignals can be extracted [11]. With these signatures, it can beapplied to the CNNs model as an image, and the model canclassify the target. Training datasets should be provided beforemaking the model and good datasets will give better modelperformance. In this study, several objects and conditionswould be tested to optimize the classification model for bothcorner and spherical reflectors, including size, distance, andangle.

VII. CONCLUSION

In this paper, the concept for a radar system utilizing modernCOTS automotive radar is presented. The radar will be capableof high-resolution sensing of targets in the competition scene

due to its range resolution enabled by the wideband FMCWsignal. This system will include a high-precision mechanicalpositioner enabling the formation of 2D and 3D images ofreflectors in the scene using SAR, and the data can be inputto a GMTI algorithm to allow information about movingtargets to be extracted. Finally, a deep-learning model will bedeveloped using training data of many different target typesand scenes to allow the radar system to classify the detectedtargets.

REFERENCES

[1] K. Ramasubramanian, K. Ramaiah, and A. Aginskiy, “Moving fromlegacy 24 ghz to state-of-the-art 77 ghz radar,” 2017.

[2] “Radar services in the 76-81 ghz band,” June 2017.[3] M. A. Richards, Fundamentals of radar signal processing. Tata

McGraw-Hill Education, 2005.[4] L. A. Gorham and L. J. Moore, “Sar image formation toolbox for

matlab,” in Algorithms for Synthetic Aperture Radar Imagery XVII, vol.7699. International Society for Optics and Photonics, 2010, p. 769906.

[5] A. W. Doerry, “Synthetic aperture radar processing with tiered subaper-tures,” Sandia National Labs., Albuquerque, NM (United States), Tech.Rep., 1994.

[6] M. E. Yanik and M. Torlak, “Near-field 2-d sar imaging by millimeter-wave radar for concealed item detection,” in 2019 IEEE Radio andWireless Symposium (RWS). IEEE, 2019, pp. 1–4.

[7] R. W. Deming, “Along-track interferometry for simultaneous sar andgmti: Application to gotcha challenge data,” in Algorithms for SyntheticAperture Radar Imagery XVIII, vol. 8051. International Society forOptics and Photonics, 2011, p. 80510P.

[8] I.-L. datasheet, “Span imu-ln200,” 2017.[9] R. G. Brown, P. Y. Hwang et al., Introduction to random signals and

applied Kalman filtering. Wiley New York, 1992, vol. 3.[10] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature, vol. 521,

no. 7553, pp. 436–444, 2015.[11] S. Capobianco, L. Facheris, F. Cuccoli, and S. Marinai, “Vehicle

classification based on convolutional networks applied to fmcw radarsignals,” in Italian Conference for the Traffic Police. Springer, 2017,pp. 115–128.