CrIMSS DR discussion-Input from LaRC

Xu LiuNASA Langley Research Center

Susan KizerSSAI, Hampton, VA 23681

25 April 2013 CrIMSS EDR Telecon

DR 4088 (from NGAS slide)• Description:

– The current operational code (GSIT) for CrIMSS (setcovback) does not match the ATBD, paragraph 5.2.6

• 1. The number of covariance matrices is listed as 6, code has 8• 2. The tests in Figure 5 (35?) are not used - instead a stratification by brightness temperature

of BT(23) with tunable thresholds appears to be used.• 3. There is no mapping accessible that indicates what surface types correspond to igeo values

of 1-8• 4. The code uses a channel difference (demmw) and the figure 35 uses a ratio. The ATBD

needs to be updated as part of document convergence to reflect the code so that we can understand how to use the tunable parameters

• Investigation:– The ATBD (Rev E) has been updated to say there are 8 different surface types– Figure 35 is a flow chart describing the precipitation detection algorithm, not the

surface type stratification– Fig 38 describes surface typing algorithm. There seems to be some inconsistency

with the code implementation (in part due to post-launch updates)• Recommendation:

– Update ATBD to be consistent with the latest code (including the recent updates)

2

LaRC Input

• The logic is in the code, however– Need to add the follow two line of code after the

statement “IF (clw > 0.3 .OR. siw > 9.0) iprcp = 2”Else

iprcp = 1

– The Blue boxes in figure 35 of the ATBD need to be modified

DR 7116 (NGAS slide)

• Description:– Noise Amplification factor coding error (details?)

• Investigation:– LaRC is working on the fix

• Recommendation:– Candidate for MX8.0 implementation

4

LaRC recommended fix to cloud-clearing noise amplification factor (1 of 2) Currently, the IDPS code does not give cloud-clearing Noise Amplification Factor (ccNAF) proper values under 2 conditions: 1). When the scene is identified as clear2). When the algorithm skipped combined IR+MW retrieval (e.g. overcast, no good IR data, etc.) Therefore, as it stands now, the cloud-clearing noise amplification factor is not a good QC parameter for the current IDPS code. Also, it is important to note that ccNAF is not the true noise amplification factor. It is the square of the noise amplification factor. The true noise amplification factor is calculated according to the following line (in calcCrimssProfiles.f): crimssRetrLvlDataPtr%IrNAFactor(currRet) = sqrt(ccNAF) The variable “crimssRetrLvlDataPtr%IrNAFactor” is written out to the IDPS EDR file as a variable named “IR_NoiseAmplification”. To fix the issue of improper values under the above mentioned conditions, we propose to change “calcCrimssProfiles.f” as follows: 1). Initialize ccNAF to -999.9 (or something appropriate): Add the line of code ccNAF=-999.9 before these two lines! Start IR iteration: irLoop: DO iter = 1, maxIrIter 2). Assign ccNAF to its proper value when performing clear-sky retrieval: Add the line of code ccNAF=1.0/9 after the following code segment

LaRC recommended fix to cloud-clearing noise amplification factor (2 of 2) IF (clrflg(ic)) THEN IF (iter <= 1) THEN CALL avgrad(ym0,ic,nfovcc, & indxfov,kchan,kchanFOV,devnoise,ym,ef) 3). To avoid the square root of a negative value of ccNAF, change the following line crimssRetrLvlDataPtr%IrNAFactor(currRet) = sqrt(ccNAF) to if (ccNAF > 0.0) crimssRetrLvlDataPtr%IrNAFactor(currRet) = sqrt(ccNAF) Note, the variable “crimssRetrLvlDataPtr%IrNAFactor” has been initialized to “CRIMSS_FLOAT32_FILL” in subroutine “initRetrievals”. 3). The current CrIMSS algorithm sets the ccNAF to 1.0 when the cloud-clearing procedure finds it less than one. This will make the cloud-cleared radiance have a larger noise than it should be. To address this issue, we propose to comment out the following line in the “calc_cc_rad.f”. IF (ccnaf<1.0) ccnaf = 1.0

DR 7118 (NGAS slide)

• Description:– H2O super-saturation thresholds (details?)

• Investigation:– LaRC is working on the fix

• Recommendation:– Candidate for MX8.0 implementation

7

LaRC recommendation to constrain water vapor mixing ratio between 300-0.005 mb The current CrIMSS algorithm checks for super-saturation from surface to 300 mb, but not from 300 mb to the top of atmospheric (0.005 mb). Occasionally, the retrieval puts a lot of water in all levels (probablly due to precipitation), but the code forces the values from 1100-300 mb to be less than saturation mixing ratio (with a tunable relaxation factor, 1.6), leaving the values above 300 mb large. The MW OSS optical depth calculation uses linear interpolation for water, while the IR uses quadratic interpolation. Since the water above 300 mb can be orders of magnitude larger than the saturation water mixing ratio, the quadratic interpolation will cause overflow or under flow when converting the water optical depth to transmittance [exp(-optical depth)]. We propose to change the following segment of code in “chkges.f” from: !-- check mixing ratios for super-saturation DO i = 64, nh2og - 1 satvpre = svp(prof(1+i)) satvmix = 622.*satvpre/(pref(i+1)-satvpre)*crimssConfigParamPtr%htol To: !-- check mixing ratios for super-saturation DO i = 1, nh2og - 1 satvpre = svp(prof(1+i)) IF (i .ge. 64) THEN satvmix = 622.*satvpre/(pref(i+1)-satvpre)*crimssConfigParamPtr%htol ELSE satvmix = 622.*satvpre/(pref(i+1)-satvpre)*10.0 END IFBasically, the code modification will force the water mixing ratio above 300 mb to be less than or equal to the water saturation mixing ratio multiplied by a factor of 10. The reasons for using such a large factor are: 1) the saturation water mixing ratio calculation for low pressure is much less accurate and meaningful, we want to make sure that we are not forcing the retrieved water above 300mb to some unphysical value, 2) the factor of 10 is small enough that the OSS IR forward model will not cause numerical overflow/underflow, and 3) the ChisqMW QC will filter out the bad retrievals.

DR 7119 (NGAS slide)

• Description:– Cloud path mislabel (details)

• Investigation:– LaRC is working on the fix ?

• Recommendation:– Candidate for MX8.0 implementation

9

LaRC recommendation on scene classification logic (1 of 3) Currently, the CrIMSS OPS code classifies the cloudiness as follows: Cloudy: (ncldfm = 0 & clrFlg(1)=False) OR ncldfm >=4Partly-cloudy: (ncldfm > 0 & ncldfm <=3)Clear: other scenes that are neither cloudy nor partly-cloudy This logic puts a scene as “clear” when there is no good quality CrIS spectrum (i.e. when cc=false and clrflg=true). The CrIMSS algorithm does not use ncldfm variable to determine retrieval strategy, it uses cc and clrflg. Proposed solution: The following table lists how the CrIMSS algorithm behaves when cc and clrflg are set at different values: Table 1: How CC and clrflg are used to determine the retrieval logic

LaRC recommendation on scene classification logic (2 of 3)

We propose to change the following segment of code in calcCrimssProfiles.f. From the current: ! Set retrieval cloudiness, default is clear. ! If 0<ncldfm<=3, set to partly cloudy. If clrflg(1) is false and ! nocldfm = 0, or ncldfm >=4 then set to cloudy IF (ncldfm >0 .AND. ncldfm <= 3) THEN crimssRetrLvlDataPtr%qualityFlags(currRet,1) = & ior(crimssRetrLvlDataPtr%qualityFlags(currRet,1), & PARTLY_CLOUDY_SCENE) ELSE IF ((ncldfm == 0 .AND. clrflg(1) .EQV. .false.) .OR. & ncldfm >= 4) THEN crimssRetrLvlDataPtr%qualityFlags(currRet,1) = & ior(crimssRetrLvlDataPtr%qualityFlags(currRet,1), & CLOUDY_SCENE) END IF To proposed: ! Set retrieval cloudiness, default is clear IF( cc(1) .EQV. .false. .or. ncldfm>=4 ) THEN crimssRetrLvlDataPtr%qualityFlags(currRet,1) = & ior(crimssRetrLvlDataPtr%qualityFlags(currRet,1), & CLOUDY_SCENE) ELSE IF (clrflg(1) .EQV. .false.) THEN crimssRetrLvlDataPtr%qualityFlags(currRet,1) = & ior(crimssRetrLvlDataPtr%qualityFlags(currRet,1), & PARTLY_CLOUDY_SCENE) ENDIF

LaRC recommendation on scene classification logic (3 of 3)

For our golden day data (05/15/2012), there are only 4 cases with ncldfm > 4. We can remove the condition of ncldfm>=4 in the "if" sentence listed above. With the above modification, the case 2 listed in table 1 (no good IR data) is categorized as a cloudy scene so MW retrieval will be used for the final product.

Discussions: With the modification above, it pretty much covers all the situations and the current OPS code already puts appropriate retrieval and QC flags into the final products. Please note that we should not confuse our off-line netcdf output files with the final products written out by the OPS code. We can always change what goes into the netcdf files in a separate discussion. For now, we are going to answer the question “if the OPS has the right logic for writing out the MW-only and IR+MW retrievals (and associated qc flags”. The answer is yes. The key logic for selecting MW-only or MW+IR retrieval for the final product is really simple: If (chisqIR <= thr1 & chisqMw2 <= thr2) then

IR+MW product is used for final productElse

MW-only retrieval is used as final productEnd ifFor example, if we don’t have IR observation, the chisqIR will be chisq_init_fill= 999999.0 and the final product will be MW-only according to the above logic (because chisqIR will be larger than 1.0 (thr1)).

DR 4923 (from NGAS slide)• Description:

– The CrIMSS EDR team has discovered that the surface pressure ancillary input to the CrIMSS EDR appears to have large errors and sometimes (10's of cases per day) exceeds reasonable values. Relative to an off-line calculation using our own DEM there are large differences, mostly over land. The fact that the ocean cases are reasonable we feel that the NWP component of surface pressure is probably computed properly. We suspect (but cannot prove) that there is some kind of error related to the computation of terrain altitude within the IDPS system on the CrIMSS field of regard. Large differences tend to be positive (IDPS values exceed our calculation by 50 mbar 0.1% of the time) and sometimes these differences are unreasonable (~20 cases greater than 1050 mbar in 0.1% of the time, 2 cases on 5/15/2012 exceed 1100 mbar) and occur in locations that are not low terrain). In extreme cases this causes the AVTP and AVMP to have derived values below the surface. The impact of this error on AVTP and AVMP is small; however, it does directly impact the AVPP product, which is a delivered EDR. Also, the EDR reporting layers are derived from surface pressure and in many cases this leads to unreasonable profile shapes in CrIMSS KPP products (the retrieval constrains top of atmosphere radiance and, as such, an error in surface pressure forces the retrieval to assign AVTP and AVMP to the wrong pressure levels). This is related to DR4008 but this DR is explicitly concerned with the unreasonably large values and accuracy of the surface pressure ancillary product. It may also be related to DR4865 (OMPS Psurf > 1.1). The CrIMSS EDR ATBD describes the desired science-code calculation; however, this computation was moved to a higher level within the operational code. We are not certain exactly what was implemented at the system level in the IDPS. We need assistance from Raytheon to understand where (and how) this calculation is done and how to fix it. We have prepared a discussion of what we have learned with our off-line calculation and suggest that the CrIMSS team holds a TIM with Raytheon to determine how to proceed to solve this issue.

• Investigation:– Note: CrIS SDR GEO are not terrain corrected, but CrIMSS GEO are. Was the offline calculation using the CrIMSS

EDR GEO or CrIS SDR GEO? Nonetheless, the differences are big and • Recommendation:

– Awaiting IDPS investigation results – Has it been officially assigned to IDPS?– NGAS can investigate if IDPS can’t get it resolved quickly

13

Bug in atmospheric altitude calculation in subroutine “calcAltitude” inside the “ProEdrCrimssAtmProf.cpp” file

outVec.push_back(0.0);

for(Int32 j = temp.size()-1; j > 0; j--) { // Calculate the virtual temperatures vt1 = temp.at(j) * (1.0 + (DRY_MOIST_RATIO * moisture.at(j)) / CrIMSS_KILO); vt2 = temp.at(j-1) * (1.0 + (DRY_MOIST_RATIO * moisture.at(j-1)) / CrIMSS_KILO); virTemp = (vt1 + vt2) / 2; gort = gor / virTemp; if (gort > 0.0) { dzp = log(pressLvls.at(j) / pressLvls.at(j-1)) / gort; } alt = outVec.front() + dzp; outVec.insert(outVec.begin(), alt); }}

•Will impact the 31-layer AVPP EDR•Will have small impact on 42-layer AVTP and 22-layer AVMP EDRs

LaRC recommendations:•Comment out the line “outVec.push_back(0.0);”•Pass the surface elevation height for the CrIS FOR into the subroutine “calcAltitude”

Initializing altitude levels to zero. Need to start from surface height

14

Planned LUTs updates• ATMS bias LUT (CrIMSS-MW-BT-BIAS-CORR-LUT)• CrIS bias LUT (CrIMSS-IR-RTM-BIAS-LUT)

– Better clear sky identification• Climatology LUT (CrIMSS-CLIM-LUT)

– Atmospheric profile LUT– MW surface emissivity LUT– IR surface emissivity LUT

• CrIMSS-IR-ATM-NOISE-LUT– Accounts for trace gas interferences, RTM error, emissivity errors

• CrIMSS-IR-NOISE-LUT– Check impact of apoziation

• CrIMSS-MW-ATM-NOISE-LUT– RTM error, Zeeman splitting, impact of TDR, surface emissivity…

• CrIMSS-MW-NOISE-AMPL-LUT – Use real remapping NAF

• CrIMSS-MW-NOISE-LUT – Use post-launch ATMS noise values

• NewCrIMSS-CHAN-SEL-LUT– Avoid trace gas interference

DR 7069 (NGAS slide)

• Description: – MW-Only Convergence QF and 2nd Stage MW Chi

Square are incorrect when missing MW retrieval

• Investigation:– LaRC or STAR are working on this?

• Recommendation:– Candidate for MX8.0?

16

QC flag fix (1 of 2)• The discrepancies are introduced in the instances of “EXIT irLoop”

– e.g. when it is overcast (no MW+IR retrieval is performed and MW retrieval is used as final EDR)

• LaRC recommended solution (copied from Mike’s word document): In calcCrimssProfiles.f, change the following:! Initialize variables used during iteration chisq0 = CHISQ_INIT_VALUE chisq_ir = CHISQ_INIT_VALUE chisq_mw1(1:maxIrIter) = CHISQ_INIT_VALUE chisq_mw2 = CHISQ_INIT_VALUE rms_ir = CHISQ_INIT_VALUE ! Added for output to

Becomes:! Initialize variables used during iteration chisq0 = CHISQ_INIT_VALUE chisq_ir = CHISQ_INIT_VALUE chisq_mw1(1:maxIrIter) = chisqmw1 chisq_mw2 = CHISQ_INIT_VALUE rms_ir = CHISQ_INIT_VALUE ! Added for output to

In calcCrimssProfiles.f, change the call to the subroutine as follows: CALL qc(chisq,chisqmw,chisqairs,nsurf,xgesg,xgesmwsav,plandavg, qcflag, iflagqc)

Becomes: CALL qc(chisq_ir(iter),chisq_mw2(iter), chisqairs,nsurf,xgesg,xgesmwsav,plandavg,qcflag,iflagqc)

QC flag fix (2 of 2)In calcCrimssProfiles, comment out the following line:! In Retr space, map error cov to IR matrix ! needed for calc_cc_rad_cor routine hcir = 0. DO i=1,npar hcir(i,i)= 1 / amax1(sxir(i,i), 1.e-9) ENDDO ENDIF !End of graceful deg. block used if MW is not available crimssRetrLvlDataPtr%chisqMw(currRet) = chisq_mw1(iter) ! This line will be removed ! Save surface pressure for output crimssRetrLvlDataPtr%sfcPressure(currRet) = xbakg(ipsfcg) ! Save land fraction for output crimssRetrLvlDataPtr%landFraction(currRet) = plandavg

Becomes:! In Retr space, map error cov to IR matrix ! needed for calc_cc_rad_cor routine hcir = 0. DO i=1,npar hcir(i,i)= 1 / amax1(sxir(i,i), 1.e-9) ENDDO ENDIF !End of graceful deg. block used if MW is not available ! Save surface pressure for output crimssRetrLvlDataPtr%sfcPressure(currRet) = xbakg(ipsfcg) ! Save land fraction for output crimssRetrLvlDataPtr%landFraction(currRet) = plandavg

• Increase the number of hinge points for IR surface emissivity retrieval– Current IR emissivity hinge

points cannot represent land emissivity variations

– Will degrade land performance– Right now we are relaxing the

noise to handle this (not optimal)

• Expect to improve ozone retrieval and better EDR performance

• Not too much code changes• Effort in updating the

climatology covariance matrix and indexes for emissivity and state vectors

19

Increase surface emissivity hinge points

• Better surface emisssivity first guess for the cloud clearing algorithm– Having a better first guess will improve the clear CrIS radiance estimate when performing

cloud clearing– Read in emissivity map from a static database– LaRC group (Dan Zhou) and UW have generated global emissivity map from IASI

hyperspectral data and MODIS data– At least stratify the IR emissivity climatology background to more classes if not using the

map

• Improve scene classification module to identify clear/overcast scene better– Use CrIS surface channel for clear ID (George Aumann) …– Use MW retrieved cloud amount– Tuning number of cloud formation determination parameters

20

Better surface emissivity first guess ?

• Improve scene classification module to identify clear/overcast scene better– Use CrIS surface channel for clear ID (George Aumann) …– Use MW retrieved cloud amount– Tuning number of cloud formation determination parameters

21

Improve Scene classification module?