Physical retrieval (for MODIS)

Andy HarrisJonathan MittazPrabhat Koner

(Chris Merchant, Pierre LeBorgne)

Satellite data – pros and consMain advantages of satellite data

– Frequent and regular global coverage (cloud cover permitting for IR)

– Not a direct measure. A retrieval process is required

– ‘Single’ source of data– Many observations

Challenges

– Single source + many observations means that data must be accurate, or risk swamping the conventional record with erroneous values

– Lack of other sources in remote regions to cross-compare

Reprocessing of historical data

• Unless we have one of these…

• …we must reprocess old data to the standard required for climate monitoring

Expected SST trend is ~0.2 K/decade

Hence requirement is that observing system must be stable to <0.1 K/decade

1975 1980 1985 1990 1995 2000 2005

AVHRR

(A)ATSR

MODIS

El Chichon Mt PinatuboEl Niño El Niño

• The chief advantage of radiative transfer (RT) is that it allows specification of the retrieval algorithm without bias towards the data-rich regions• The in situ data can then act as a random independent sampling of the retrieval conditions• If the observed errors agree with the modeled ones, then high confidence can be placed on the modeled errors in data-sparse regions

Radiative transfer-based retrievals

Is it necessary? Let’s consider the likely errors in the empirical ‘state-of-the-art’ AVHRR Pathfinder SST…

Early theory required

SST – Ti = kiF(atm)

This allowed

SST = k2T1 – k1T2 ———————————

(k2 – k1)

And hence the “split-window” equation, mystique about channel differences, etc.

Only need to assume SST – Ti SST – Tj to get SST = a0 + aiTi

Some refinements to account for non-linearity, scan angle:

SST = a0 + a1T11 + SSTbga2 (T11 – T12) + (SZA-1)a3 (T11 – T12)

Some issues w.r.t. regression-based SST retrievals

• Does it matter what form the scan-angle correction takes?– Sec(ZA) term multiplying a channel difference (e.g.

NLSST)– Separate SZA-dependent term for each independent

variable

• How much signal is coming from the surface?– Dependence on water vapor– Geophysical coupling

Scan-angle dependence• Simulation using Modtran & 1,358 globally sampled

ECMWF profiles, sec(ZA) = 1.0, 1.25, 1.5… …2.5• In-sample testing of algorithm accuracy

Scan-angle dependence• Simulation using Modtran & 1,358 globally sampled

ECMWF profiles, sec(ZA) = 1.0, 1.25, 1.5… …2.5• In-sample testing of algorithm accuracy

How much signal?• Simulation using Modtran & 1,358 globally sampled

ECMWF profiles, sec(ZA) = 1.0, 1.25, 1.5… …2.5• The mid-IR ~4 μm channel(s) are powerful…

How much signal?• Simulation using Modtran & 1,358 globally sampled

ECMWF profiles, sec(ZA) = 1.0, 1.25, 1.5… …2.5• The mid-IR ~4 μm channel(s) are powerful…• Use all surface-sensitive channels…

Simulated Pathfinder Retrieval Errors

• The effect can be modeled using the following– Pathfinder matchup dataset– Full-resolution ERA-40 atmospheric and surface data– Fast-forward radiative transfer model (CRTM)

• Process is as follows– Pathfinder matchup information is used to select the

appropriate atmospheric profiles for each month– SSTs and profiles are used to generate ToA radiances

which are then used to generate algorithm coefficients– Retrieval coefficients are then applied to full month of

radiances generated from cloud-free ERA-40 data

• AVHRR Pathfinder Oceans is current “state-of-the-art” SST– Distribution & quantity of matchup data change over time– How does this affect SST retrieval performance?

Pathfinder Mean Bias 1985 – 1999

Pathfinder Mean Bias 1985 – 1999

Changes in sampling 1985 – 1999

Physically-based methodology models retrieval effects explicitly – apply to whole AVHRR record to improve consistency of SST record for climate studies

Using Radiative Transfer in SST Retrieval

• Since algorithm biases can be modeled, just subtract them– Has been employed operationally (LeBorgne et al.,

2011)– Has little impact on S.D., no impact on sensitivity,

but does improve bias– Dependent on model input (!)

• Accuracy of NWP profile• Not in model state not corrected (e.g. aerosol)

Direct removal of local retrieval bias

• Example - Methodology of Meteo-France:

[From presentation by Pierre LeBorgne @GHRSST XIII, Tokyo]

“Physical” Retrieval:Estimate adjustment to “guess” SST

• Based on local linearization, i.e. Δy = KΔx

– Δy = yo – yg , K, yg from RTM + NWP– x is reduced state vector, at least [SST, TCWV]T

– K is Jacobian of partial derivatives of yg w.r.t. x

• Ordinary unweighted least squares solution:Δx = (KTK)-1KTΔy [ = GΔy ]

• Potential issues:– G may be ill-conditioned (noise amplification)– K may be “incorrect”– Δy may be biased, i.e. Δy ≠ 0 when Δx = 0

• e.g. RTM, calibration

Incremental Regression

• Only use Δy and simply derive a regression coefficient-based operator to retrieve ΔSST (Petrenko et al.)– Do not retrieve a water vapor adjustment– Side-step issues of calibration, errors in RTM calculation, etc.

• Essentially no control over algorithm sensitivity– Noisy input will suppress coefficients

• No means of estimating uncertainty– Cf. other physical methods

• No means of iterating solution– No adjustment to model state (apart from SST)

• Revision of other processing elements requires recalculation of coefficients

The “Traditional” ApproachOptimal Estimation

• E.g. Merchant et al. for SST• Primary issue is dealing with the ill-conditioned nature of gain matrix

G– Use prior estimates of uncertainty in RTM, model state and instrument

noise– RTM + satellite errors combined into one covariance matrix Se

– Error in model state propagated through covariance for reduced state vector Sa

Δx = (KTSe-1K + Sa

-1)-1KTSe-1Δy

• Error covariance matrices function as a regularization operator on the gain matrix– Reduces the “condition number”– Any regularization will reduce sensitivity to true SST change

Extended OESST[From presentation by Chris Merchant @GHRSST XIV, Woods Hole]

Extended OESST

Extended OESST - Results

MODIS - OSTIA, Feb 2012

[SST (night) – OSTIA] cf. modeled

• Modeled clear-sky NLSST bias is February average for 1985 – 1999− No aerosols

• Several features common to both observed and modeled biases− Boundary currents over-estimate of gamma− Cold tongue under-estimate of gamma− High SST but low water vapor− (Decoupling of SST and air temperature)

• Modeling shows annual cycle of bias

• Hence Lat-band coefficients, etc.• Employ a physical retrieval methodology

− MTLS

Modified Total Least Squares:A Deterministic Regularization

• Based on local linearization, i.e.

G = (KTK + rI)-1KT

– r is dynamically calculated regularization strength

r = (2 log(κ)/||Δy||)σ2end

– κ is the condition number of K– σ2

end is the lowest singular value of [K Δy]

– ||Δy|| is L2-norm of Δy

• Regularization uses observation-based estimate of noise amplification– Closer to “actual” error for retrieval/observation conditions on a

case-by-case basis

MTLS applied to MODIS

• Validation against iQUAM– MTLS: -0.02±0.36 K SST(night): -0.21±0.47 K – Reduction of ~0.3 K of independent error– (SST4: -0.24±0.44 K)– (Difference in bias due to skin effect)

Uncertainty Estimation

• OE uses “goodness of fit”χ2 = (KΔx – Δy)T[Se(KSaKT + Se

-1)Se]-1(KΔx – Δy)

• MTLS formulation for total error, e||e|| = ||(MRM – I)Δx|| + ||G||||(Δy - KΔx)|| KT

– Δx should be true value, but is substituted by retrieved quantity

– MRM is model resolution matrix (a.k.a. averaging kernel)

Summary

• Inherent limitations in NLSST• MODIS has 16 TIR channels

– Currently, only a very few used for SST retrieval• 1st cut MTLS physical retrieval shows promise

– Initial result subject to MODIS cloud mask• Extra channels permit more complex retrieval vector

– WV scale height, air temperature, aerosol, etc.• Multiple iterations• Smoothed inputs (Merchant)• OE and MTLS allow prospect for direct uncertainty

estimation