1

Automated Cyclone Tracking using Multiple Remote Satellite Data via Knowledge Transfer

Shen-Shyang Ho and Ashit Talukder Jet Propulsion Laboratory

California Institute of Technology 4800 Oak Grove Ave 300-123

Pasadena, CA 91109 {sho, ashit.talukder}@jpl.nasa.gov

Abstract—Cyclone tracking using a single orbiting satellite in a continuous manner is impractical as it has limited spatial and temporal coverage. One solution is to use multiple orbiting satellites for cyclone tracking. However, data from some orbiting satellites do not provide features as useful as other satellites in identifying cyclones. Moreover, satellite data containing strong cyclone discriminating features may be affected by coarse temporal resolution and object occlusion. In this paper, we propose a knowledge transfer methodology based on a Kalman filter for cyclone tracking using multiple satellite data sources containing a mixture of strong and weak features. This approach minimizes the negative effect of coarse temporal resolution and occlusion if only the satellite data containing strong cyclone discriminating features were used. Experimental results are presented to demonstrate the feasibility and usefulness of our knowledge transfer approach for cyclone tracking.12

TABLE OF CONTENTS

1. INTRODUCTION.................................................................1 2. REVIEW ............................................................................2 3. CYCLONE TRACKING VIA KNOWLEDGE TRANSFER .........2 3. METHODOLOGY ................................................................3 4. EMPIRICAL RESULTS.........................................................4 5. CONCLUSIONS ..................................................................5 6. ACKNOWLEDGEMENTS .....................................................6 REFERENCES ........................................................................6 BIOGRAPHY ..........................................................................6

1. INTRODUCTION Tropical and extra-tropical cyclones are key manifestations of the oceanic air-sea interaction and contribute to regional heat exchanges, which affects ocean and atmosphere dynamics. A cyclone landfall causes great devastation, incurs fatality, and affects people's livelihood. To identify and track tropical weather system, the Tropical Prediction Center/National Hurricane Center (TPC/NHC) uses conventional surface and upper-air observations and reconnaissance aircraft reports from field missions [9]. These observations, however, are concentrated in the North American coasts and in Japan/Europe to some degree.

1 1 978-1-4244-2622-5/09/$25.00 ©2009 IEEE. 2 IEEEAC paper #1313, Version 3, Updated January 9, 2008

Coverage on a global basis, especially in under-developed and developing nations such as large portions of Asia and Africa is limited or lacking which results in disastrous consequences in many of these regions. In recent years, some studies have used satellite images that are manually retrieved and analyzed to improve the accuracy of cyclone tracking. This procedure is currently slow, tedious, and requires close analysis by teams of experts. Satellite sensor data from the QuikSCAT (Quick Scatterometer) mission providing wind speed and direction measurement has been extremely powerful in identifying a cyclone, which is an “organized deep convection and a closed surface wind circulation about a well-defined center”3 of high sustaining wind speed. Recent research showed that QuikSCAT data could be useful in early identification of tropical depression [6] and early detection of tropical cyclones [9, 10]. Moreover, QuikSCAT data has been used in the three-dimensional variational data assimilation technique for better cyclone tracking and intensity forecasting [7]. Our recent work [4] showed the feasibility of using QuikSCAT wind measurements for automated cyclone identification. However, due to the polar orbiting nature of the QuikSCAT satellite, it has limited spatial and temporal coverage. In particular, the time interval between two consecutive observations of a cyclone is approximately around 12 hours. Sometimes the sensor can only capture partial measurement of the cyclone due to the orbiting path. To alleviate these problems, one can use sensor measurements from multiple orbiting satellites. However, sensors from other satellites measure different earth and atmospheric properties. These measurements are not as powerful in identifying cyclone as the wind properties measured by the Scatterometer on QuikSCAT satellite. For example, the precipitation rate measured by the Tropical Rain Measurement Mission (TRMM)4 has been used to trace the cyclone track manually either during a cyclone event or as a reanalysis after a cyclone event. However, the precipitation rate cannot be used for cyclone identification since heavy rainfall does not imply a cyclone event. In this paper, we describe our methodology (first proposed in [5]) for transferring local spatial-temporal knowledge between satellite data from different data sources (i) to enable robust event detection using weaker satellite data

1 3 http://www.research.noaa.gov/weather/t_hurricanes.html 4 http://trmm.gsfc.nasa.gov/

2

features and (ii) to reduce temporal resolution for event tracking in detail. The paper is organized as follows. In the next section, we briefly review some common transfer learning scenarios. Then, we describe in detail the problem of applying knowledge transfer for cyclone detection using multiple satellite data sources. After that, we describe our methodology for spatial-temporal knowledge transfer. Finally, we apply our proposed transfer learning methodology to cyclone detection is described in detail and we demonstrated the feasibility and usefulness of our approach to cyclone tracking.

2. REVIEW Inductive transfer or transfer learning refers to “the problem of retaining and applying the knowledge learned in one or more tasks to efficiently develop an effective hypothesis for a new task”5. The key characteristic of transfer learning is that knowledge is transfer across domains, tasks, or/and distributions that are similar but not identical. Transfer learning capitalizes on previously acquired domain knowledge or data model (from other problems) to benefit the handling of current related task. Multi-task learning [3] is one form of transfer learning which attempts to improve generalization performance by “leveraging the domain-specific information contained in the training signals of related task”. The training examples from the related task, sharing a common representation, serve as an inductive bias6. Another setting of transfer learning is called the “domain adaptation” [1]. In such a setting, one specified a source domain and a target domain such that the source and target distributions are different but are defined over a particular feature space. To put it in a simpler way, it is basically learning when the training and the testing distributions are different. A more specific setting is when the distributions are different but the conditional distribution of the label given the unlabeled examples is identical for both training and testing distributions. This is also known in the statistical community as the “covariant shift adaptation” problem [2, 11].

3. CYCLONE TRACKING VIA KNOWLEDGE TRANSFER

The problem of using multiple data sources for a single target task can be studied in a general framework for transfer learning. For a classification problem, classifiers based on data from multiple sources perform differently as a variety of discriminating features are used in the discrimination process.

2 5 NIPS 2005 Workshop “Inductive Transfer: 10 Years Later” 6 Inductive bias of a machine learning algorithm is defined to be the assumptions that the algorithm’s predictions based on the observed data (training examples) are logical deductions.

For cyclone tracking, one has the additional time and space constraints on the data from multiple satellite sources. In other words, a satellite data matrix from source A is collected at time t1 on some (cyclone) event in region R1 while the next data from source B is collected at time t2 on the same (cyclone) event in region R2 such that t1 is not equal t2, R1 is not equal R2, and Area(R1) is not equal Area(R2) since cyclone is a dynamic event (see Figure 1). In other words, the data from multiple satellite observations is not registered spatially or temporally. Hence, one cannot simply merge/fuse/combine data from source A and B using prior data fusion solutions that assume co-registered observations.

Figure 1. Data from multiple satellite data sources e.g. the QuikSCAT and the TRMM satellites for cyclone tracking.

We introduce the concept of knowledge transfer to overcome the issues related to using multiple data sources for (near) real-time object tracking. Towards this end, we put the cyclone tracking problem into the knowledge transfer framework as follows. The source task is “identify cyclone from the QuikSCAT wind data”. The target task is “identify cyclone from the merged precipitation data from TRMM satellite. The task relatedness criteria between the source and target task are namely:

1. Task objective is identical 2. Cyclone (strong wind) implies rain. 3. Overlapping “feature”: Space and Time.

In Figure 2, we illustrate the concept of knowledge transfer from Detector A built using training data with more discriminating power (QuikSCAT) to detector B which uses the spatial-temporal knowledge from Detector A and prior knowledge from data with lesser discriminating power (TRMM) for cyclone tracking.

Figure 2. Spatial-temporal Knowledge Transfer between the TRMM data source and the QuikSCAT data source for cyclone tracking.

3

To avoid negative transfer, which reduces the detection and tracking performance, two conditions need to be met: 1. Detector A built using training data with more

discriminating power must be as powerful as possible so that the spatial-temporal knowledge generated is useful for the weaker Detector B.

2. The much weaker Detector B should have a high true positive rate, i.e. high recall.

A high false positive rate, i.e. low precision, for Detector B does not affect the performance of tracking as long as accurate spatial-temporal knowledge is available from Detector A. The knowledge transfer enables the application of TRMM data for automated cyclone tracking which was previously not possible. One direct effect of using multiple satellite data sources is a reduction in the temporal resolution of observing two consecutive cyclone event images from about 12 hours using a single QuikSCAT satellite to a maximum of 3 hours using TRMM and QuikSCAT (and also the Aqua) satellites (See Figure 3).

Figure 3. Data availability timeline for TRMM (3B42 data), QuikSCAT (L2B data) and Aqua (MODIS) on 18 Aug 2007 for Hurricane Dean.

In the next section, we describe our knowledge transfer methodology in detail.

3. METHODOLOGY The knowledge transfer methodology is driven by a Kalman filter [12]. Based on the output from a strong detector, a spatial-temporal bias is induced to provide knowledge to localize (“predict”) the search region for detection using data that contains weaker discriminating feature. The output from the weak detector, again, will induce a spatial-temporal bias (“correction”) to be used by the weak detector if during the next iteration data with weak discriminating feature is available (see Figure 4). Let A be the state transition model (motion model), Q be the process noise covariance, and H is the observation model which maps the true state space into the observed state

space. The state vector is of the form

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

ΔΔ

=

yx

yx

x . The motion

model

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

ΔΔ

=

k

k

tt

A

00000000100001

where ktΔ is the time

interval between the current observation and the next

observation. The observation model ⎟⎟⎠

⎞⎜⎜⎝

⎛=

00100001

H .

The “predict” search region process consists of 1. Estimating the state at the next step:

kk xAx ˆˆ 1 =+

2. Estimating the error covariance at the next step:

QAAPP Tkk += ++ 11

ˆ

3. Define the search region based on 1ˆ +kx and 1ˆ

+kP

The “correction” process consists of 1. Computing the Kalman gain

1111 )ˆ(ˆ −

+++ += RHPHHPK Tk

Tkk

2. Updating estimate with measurement 1+kz

)ˆ(ˆ 11111 +++++ −+= kkkkk xHzKxx

3. Updating the error covariance

111ˆ)( +++ −= kkk PHKIP

One notes that Kalman filter assumes that the posterior density of the state estimation and noise model at every time step is Gaussian. Hence, the Kalman filter is parameterized by a mean vector and a covariance metrics. Moreover, in a system described by a linear motion model and a linear observation model under the Gaussian distribution assumption, the Kalman filter achieved the optimal solution (in the sense of least square error) to the tracking problem. This knowledge transfer process continues as one tracks the object with two data sources, one with weak features and one with strong features. This knowledge transfer process can be further generalized to the two following scenarios: 1. Multiple data sources with either strong or weak

discriminating features, and 2. Occurrence of consecutive strong/weak feature

observations.

The knowledge transfer process shown in Figure 4 can be easily extended to Scenario 1. For Scenario 2, knowledge transfer can be ignored when there are consecutive strong feature observations. However, when there are consecutive weak feature observations, one needs to use the previous weak feature observation to make search region prediction for the next weak feature observation. In our current implementation, cyclone tracking occurs in Scenario 2. We note that the tracking performance is not affected as long

4

asa strong feature observation is measured within a reasonable number of weak observations. We point out here again that in our cyclone tracking problem weak feature observations are more readily available and it helps to reduce the temporal resolution of cyclone tracking.

Figure 4. Knowledge transfer between data sources containing strong and weak features.

Figure 5. Knowledge transfer based on Kalman filter using QuikSCAT

and TRMM satellite data.

Our automated cyclone tracking using knowledge transfer based on Kalman filter is shown in Figure 5. Initially, QuikSCAT data is retrieved from the database or from real-

time streaming information, and is input into the cyclone discovery module (strong detector) to locate/identify possible cyclones. The cyclone location is then used to predict the regions that are likely to contain a cyclone at the next incoming data stream retrieved using a linear Kalman filter predictor. If the next data stream is the 3B42 TRMM data, a constrained search is carried out around the region most likely to contain the cyclone as identified by the Kalman Filter predictor. This constrained tracking via the Kalman Filter predictor is especially important for the 3B42 TRMM precipitation data as it is not a definitive indicator of cyclones and is susceptible to high false alarms. The estimated search region localizes the region that is most likely to contain cyclone based on past cyclone tracks and hence the incidence of false alarms is minimized by a large margin. A cyclone is localized by applying a threshold to the TRMM precipitation rate measurement (T6 = 0). After a cyclone is located in the TRMM data, the Kalman filter measurement update (“correction”) is applied to obtain an estimate of the new state vector or the predicted location of the cyclone in the next TRMM (or QuikSCAT) observation cycle after 3 hours.

4. EMPIRICAL RESULTS Our knowledge transfer solution leverages the strength of each remote sensor type for cyclone tracking. QuikSCAT wind measurement has excellent information for accurate cyclone detection but lacks sufficient temporal resolution (each pass-through is repeated every 12 hours). TRMM precipitation measurement on the other hand has excellent temporal resolution of 3 hours, but lacks good discriminative ability for accurate cyclone detection. Therefore, we employ (strong feature) QuikSCAT wind measurement for cyclone detection (every 12 hours), and knowledge transfer to (weak feature) TRMM precipitation measurement for detection/tracking (every 3 hours). This solution therefore ensures a high detection rate for cyclones while maintaining fine temporal resolution during cyclone tracking. Figure 6 demonstrates the feasibility of the knowledge transfer-based tracking methodology using both Level 2B QuikSCAT data and 3B42 TRMM data for Hurricane Gonu (reaching Category 5 wind speed level) in the North Indian Ocean in 2007 for two days. It is the strongest tropical cyclone since record keeping begun in 1945 for the North Indian Ocean and the Arabian Sea. Hurricane Gonu is an interesting weather event as tropical cyclones developed in the Arabian Sea very rarely exceed the tropical storm intensity, i.e. becoming a hurricane.

Cyclone Discovery Module

Kalman Filter “Predict” Search

Region

Cyclone Location Prediction:

Precipitation > T6 TRMM

QuikSCAT

Kalman Filter “Correction”

5

Figure 6. Two days tracking of Hurricane Gonu in 2007 using the QuikSCAT and the TRMM data. A red box bounds the predicted cyclone. The region bounded by a black box is a region correctly identified as a non-cyclone region.

5. CONCLUSIONS Knowledge sharing between different satellites is a challenging and as yet unresolved problem, and an efficient solution such as ours that taps the information from such disparate sources will greatly improve science data understanding in various domains in environmental and space science. To date, we have tested our technique on isolated hurricanes. We aim to test our implementation over a longer time scale in a region where there are occurrences of multiple cyclones that will allow us to demonstrate the global tracking capability of our implementation. In our current implementation, the merged TRMM 3B42 data are used for motion/location prediction and cyclone tracking. In the future, we would like to deploy (i) TRMM 2B25 swath data to construct a vertical profile of reflectivity for cyclone events to be used in a Cyclone Discovery Module (CDM) for the TRMM data. We plan to include the TRMM 3B40RT gridded data with an hourly temporal resolution to further improve the quality and accuracy of cyclone detection and tracking. Further knowledge sharing will also include the use of MODIS atmospheric data from the Aqua satellite and sensor measurements from other satellites to further refine the detection and temporal tracking accuracy. We would also extend and improve our knowledge transfer methodology by replacing Kalman filter with particle filter. This would remove the restrictive assumptions of Kalman filter discussed in Section 3. Furthermore, we would

6

construct a theoretically rigorous framework to justify our knowledge transfer methodology for sequential spatial-temporal data.

6. ACKNOWLEDGEMENTS This work was carried out at the Jet Propulsion Laboratory, California Institute of Technology with funding from the NASA Applied Information Systems Research (AISR) Program. The first author is supported by the NASA Postdoctoral Program (NPP) administered by Oak Ridge Associated Universities (ORAU) through a contract with NASA. The authors thank Andrew Bingham and Eric Rigor for their help in obtaining the Level 2B QuikSCAT data.

REFERENCES [1] Ben-David, Shai, John Blitzer, Koby Crammer, and Fernando Pereira. “Analysis of Representations for Domain Adaptation.” NIPS. MIT Press, 2006. 137-144.

[2] Bickel, Steffen, Michael Bruckner, and Tobias Scheffer. “Discriminative learning for differing training and test distributions.” ICML. Corvalis, Oregon: ACM, 2007. 81-88.

[3] Caruana, Rich. “Multitask Learning.” Machine Learning, no. 28, 1997, 41-75.

[4] Ho, S.-S., and Talukder, A. “Automated Cyclone Identification from Remote QUIKSCAT Satellite Data”, IEEE Aerospace Conference, 2008. [5] Ho, S.-S. and A. Talukder, “Cyclone Tracking Using Multiple Satellite Data Sources via Spatial-Temporal Knowledge Transfer”, AAAI-08 workshop, Transfer Learning for Complex Tasks, Chicago, IL, 2008. [6] Katsaros, K. B.; Forde, E. B.; Chang, P.; and Liu, W. T. “QUIKSCAT’s Seawinds facilitates early identification of tropical depressions in 1999 hurricane season.” Geophysical Research Letters, Vol. 28, No. 6, 2001, 1043–1046. [7] Liang, X.; Wang, B.; Chan, J. C.; Duan, Y.; Wang, D.; Zeng, Z.; and Ma, L. “Tropical cyclone forecasting with model-constrained 3D-VAR. ii: Improved cyclone track forecasting using AMSU-A, QUIKSCAT and cloud-drift wind data.” Q.J.R. Meterol. Soc. 133, 2007, 155–165. [8] Lungu, T., and et. al. “QUIKSCAT science data product user’s manual”. Version 3.0, D-18053-Rev A, 2006. [9] Pasch, R. J.; Stewart, S. R.; and Brown, D. P. Comments on “Early detection of tropical cyclones using Seawinds-derived vorticity”. Bulletin of the American Meteorological Society, Vol. 85, No. 10, 2003, 1415–1416. [10] Sharp, R. J.; Bourassa, M. A.; and O’Brien, J. J. “Early

detection of tropical cyclones using Seawinds derived vorticity.” Bulletin of the American Meteorological Society Vol. 83, No. 6, 2002, 879–889. [11] Sugiyama, Masashi, Matthias Krauledat, and Klaus-Robert Muller. "Covariate Shift Adaptation by Importance Weighted Cross Validation." Journal of Machine Learning Research, No. 8, 2007, 985-1005. [12] Welch, G., and Bishop, G. “An introduction to Kalman filter.” TR 95-041, Department of Computer Science, University of North Carolina at Chapel Hill, 2006.

BIOGRAPHY

Shen-Shyang Ho received his BS degree in mathematics and computational science from the National University of Singapore in 1999, and the MS and PhD degrees in computer science from George Mason University in 2003 and 2007, respectively. He is currently a NASA Postdoctoral Program (NPP) fellow affiliated to the Jet Propulsion Laboratory at the California Institute of Technology. His research interests include machine learning, online learning from data streams, adaptive learning, pattern recognition, data mining and optimization methods. He is a reviewer for IEEE Transaction on Pattern Analysis and Machine Intelligence, IEEE Transaction on Aerospace and Electronic Systems, and IEEE Transaction on Knowledge and Data Engineering. He is a technical program committee member for the IEEE International Conference on Data Mining (ICDM-2008) and the Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD-09).

Ashit Talukder graduated with a Ph.D. in Electrical and Computer Engineering from Carnegie Mellon University in 1999. His research interests and expertise include pattern recognition, image and signal processing, machine learning, AI, controls, computer vision, optimization, and mathematics, applied to data

7

mining, robotics, autonomous systems and sensors, sensor networks, biometrics, ATR, and human-machine interaction. He is currently a Senior Researcher (Level A) at Jet Propulsion Laboratory / NASA, California Institute of Technology, in Pasadena, California, and also holds a joint appointment as a research adjunct faculty member at University of Southern California and senior researcher at CHLA. He is a principal investigator, project manager and technical lead on several NASA, NIH, NSF, DoD and JPL-funded projects. He has over 55 refereed journal and conference publications and he has authored a book chapter. Dr. Talukder is on the organizing committee of the Annual SPIE Defense and Security Symposium and chair of the ESSA-2009 workshop on sensor networks for Earth and Space Science under IPSN-2009. He is on the Technical Organizing committee of the IJCAI-09 Workshop on Artificial Intelligence in Space held at the International Joint Conference on Artificial Intelligence (IJCAI-09). He is a Technical Program Committee member of the International Conference on Digital Telecommunications ICDT 2006 and The First International Conference on Advances in Computer-Human Interaction ACHI 2008. He has a patent for a biomedical data analysis and visualization system which was nominated for the NASA software of the year, and holds a provisional patent for a remote biometric identification system. He is a recipient of the NASA Space Act Award and a PACE award at Iowa State University. He has chaired several SPIE conference sessions and given three invited conference talks. He is a reviewer on several technical journals.