Yongtao Hu 1 , M. Talat Odman 1 , Michael E. Chang 2 and Armistead G. Russell 1

Georgia Institute of Technology

Comprehensive evaluation on air quality forecasting ability of Hi-Res

in southeastern United States

Yongtao Hu1, M. Talat Odman1, Michael E. Chang2 and Armistead G. Russell1 1School of Civil & Environmental Engineering,2Brook Byers Institute of Sustainable Systems


8th Annual CMAS Conference, October 19th, 2009


Outline

The Hi-Res air quality forecasting system.

2006-2009 O3 and PM2.5 performance for Atlanta metro.

Spatial variation of forecast performance.

Linking forecast performance to weather conditions.

Linking forecast performance to emissions conditions.


Hi-Res: forecasting ozone and PM2.5 at a 4-km resolution in metro Atlanta

area

CMAQ

SMOKE

Emission Inventory

WRF

NAM 84-hr Forecast

Forecast Product

MeteorologyEmissions

Air Quality

CMAQ

SMOKE

Emission Inventory

WRF

NAM 84-hr Forecast

Forecast Product CMAQCMAQ

SMOKE

Emission Inventory

SMOKESMOKE

Emission InventoryEmission Inventory

WRF

NAM 84-hr Forecast

WRFWRF

NAM 84-hr Forecast NAM 84-hr Forecast

Forecast Product Forecast Product

MeteorologyEmissions

Air Quality

Hi-Res Air Quality Forecasting SystemServing Metro-Atlanta Area since 2006

Hi-Res Modeling Domains

36-km (72x72)

4-km (99x78)

12-km (72x72)

Current 4-km (123x123)

36-km (72x72)

4-km (99x78)

12-km (72x72)

Current 4-km (123x123)

12Z 27Z

ramp up 36-12- & 4-km forecasts

77Z

00Z 15Z


66Z

8pm 0am0am0am0amForecast Day 2

R&R&R&S F&R R

Forecast Day 1Forecast Day 2Forecast Day 2Forecast Day 2

S F

Cycle 1

Cycle 2

OperationForecast Day 1Forecast Day 1Forecast Day 1

12Z 27Z


77Z12Z 27Z


77Z

00Z 15Z


66Z

8pm 0am0am0am0amForecast Day 2

R&R&R&S F&R R

Forecast Day 1Forecast Day 2Forecast Day 2Forecast Day 2

S F

Cycle 1

Cycle 2

OperationForecast Day 1Forecast Day 1Forecast Day 1

Hi-Res Cycle


Hi-Res Forecast Products“Single Value” Report: tomorrow’s AQI, ozone and PM2.5 by metro area in Georgia

Air Quality Forecasts: AQI, ozone and PM2.5, 48-hrs spatial plots and station profiles

Meteorological Forecasts: precipitation, temperature and winds, 48-hrs spatial plots and station profiles

Performance Evaluation: time series comparison and scatter plots for the previous daySnapshots from Hi-Res homepage:

http://forecast.ce.gatech.edu


Evolving history of Hi-Res during 2006-2009

Updated to new release of WRF each year before ozone season.

•WRF2.1, 2.2, 3.0 and 3.1

Projected NEI to current year in the very beginning of each year.

Updated forecast products website each year before ozone season.

Switched from single-cycle forecasting to two-cycles in 2008.

Enlarged 4-km domain to cover the entire Georgia in 2009.

Introduced Georgia Tech’s new SOA module in 2009.

Data assimilation in ozone forecasting is in experiment.


Purposes of Forecasting Performance Evaluation

To hopefully have a good performance show and hence to give a good reason for being further funded.

•It’s in fact very important…

To explore reasons for why bad forecasting so that we can improve.

•Science? Can we blame for “Smog Alert” that reduced emissions?

Finally for users of our forecasts: to build “quantitative” confidence in the forecasts.

•If today is a sunny Monday how I am going to trust their “Smog Alert”?

“Comprehensive” evaluation, preliminary results presented here.


Performance Metrics

N

kok

ok

mk

c

cc

N 1

1MNB

N

kok

ok

mk

c

cc

N 1

1MNE

False Alarms

Hits

Correct Nonevents

MissedExceedenc

es

For

ecas

t

Observation

NAAQS

NA

QQ

S


Overall 2006-2009 Performance: Atlanta Metro

Ozone PM2.5

MNB 17%

MNE 24%MNB -25%

MNE 37%

0

75

150

0 75 150

Obs.

4-km

94 98

34637

0.0

35.0

70.0

0 35 70

Obs.

4-km

0 0

517

47


Ozone Performance


Forecast vs. Observed O3

0.0

20.0

40.0

60.0

80.0

100.0

120.0

May-01 Jun-01 Jul-01 Aug-01 Sep-01

O3 (

pp

b)

Obs. 4-km

0

20

40

60

80

100

120


O3

(pp

b)

Obs 4-km

0

20

40

60

80

100

120


O3

(pp

b)

Obs 4-km

0

20

40

60

80

100

120


O3

(pp

b)

Obs 4-km

2006

20092008

2007


2006 O3 Performance: Hi-Res vs. EPD’s

0.0

85.0

170.0

0.0 85.0 170.0

Obs.

EP

D

9 17

10

117

0.0

85.0

170.0

0.0 85.0 170.0

Obs.

4-km

27 15

995

Our 4-km Forecast EPD Ensemble Forecast

MNB 11%

MNE 29%

MNB 6.2%

MNE 15%




MNB 8.5%

MNE 19%

MNB 9.0%

MNE 18%

0

85

170

0 85 170

Obs.

4-km

10 11

10611

0

85

170

0 85 170

Obs.E

PD

10 16

11210




MNB 17%

MNE 23%

MNB 11%

MNE 19%

0

75

150

0 75 150

Obs.

4-km

22 19

1039

0

75

150

0 75 150

Obs.E

PD

20 23

1065




MNB 28%

MNE 30%MNB 13%

MNE 21%

0

75

150

0 75 150

Obs.

4-km

24 7

1084

0

75

150

0 75 150

Obs.E

PD

7 6

1255


Particulate Matter Performance


Summer


Forecast vs. Observed PM2.5

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0


PM

2.5

(m g

/m3)

Obs. 4-km

0

5

10

15

20

25

30

35

40

45


O3

(pp

b)

Obs 4-km

0

5

10

15

20

25

30

35

40

45


O3

(pp

b)

Obs 4-km

0

5

10

15

20

25

30

35

40

45


PM

2.5

(ug

/m3)

Obs 4-km

2006 2007

2008 2009


2007 PM2.5 Performance: 4-km vs. EPD’s


MNB -37%

MNE 44%

MNB 8.6%

MNE 28%

0

35

70

0 35 70

Obs.

4-km

0 0

118

210

35

70

0 35 70

Obs.E

PD

9 12

11810




MNB -38%

MNE 42%

MNB -0.2%

MNE 22%

0

35

70

0 35 70

Obs.

4-km

0 0

143

6

0

35

70

0 35 70

Obs.E

PD

2 2

1414




MNB 8%

MNE 25%

MNB 11%

MNE 24%

0

35

70

0 35 70

Obs.

4-km

0 0

129

3

0

35

70

0 35 70

Obs.E

PD

0 0

129

3


Winter


Forecasted vs. Observed PM2.5

0

5

10

15

20

25

30

35

40

45

Oct-07 Nov-07 Dec-07 Jan-08 Feb-08 Mar-08

O3

(pp

b)

Obs 4-km

0

5

10

15

20

25

30

35

40

45

Oct-08 Nov-08 Dec-08 Jan-09 Feb-09 Mar-09

O3

(pp

b)

Obs 4-km

2007

2008


Spatial Variation of Performance: Ozone

30%

32%

26%Douglasville

29%

26%27%

28%37%

36%

26%

“Single value” forecast for Atlanta metro has a MNE as 24%


Spatial Variation of Performance: PM2.5

41%Douglasville

43%

40%

40%41%

39%

42%

“Single value” forecast for Atlanta metro has a MNE as 37%


Mostly Sunny

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0.00

%

10.0

0%

20.0

0%

30.0

0%

40.0

0%

50.0

0%

60.0

0%

70.0

0%

80.0

0%

90.0

0%

100.

00%

110.

00%

120.

00%

130.

00%

140.

00%

8-hr O3 Forecast Error

Rain

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0.00

%

10.0

0%

20.0

0%

30.0

0%

40.0

0%

50.0

0%

60.0

0%

70.0

0%

80.0

0%

90.0

0%

100.

00%

110.

00%

120.

00%

130.

00%

140.

00%


Cloudy

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

120%

130%

140%


Partly Cloudy/Sunny

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0.00

%

10.0

0%

20.0

0%

30.0

0%

40.0

0%

50.0

0%

60.0

0%

70.0

0%

80.0

0%

90.0

0%

100.

00%

110.

00%

120.

00%

130.

00%

140.

00%


24%

24%24%

24%

78%64%

46%42%

This means a 78% chance that a 8-hr O3 forecasts error in a sunny day is less than 24%

Linking Performance to Weather Conditions: (1) Ozone


Mostly Sunny

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

120%

130%

140%

24-hr PM2.5 Forecast Error

Rain

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

120%

130%

140%


Linking Performance to Weather Conditions: (2) PM2.5

Cloudy

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

120%

130%

140%


Partly Cloudy/Sunny

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

120%

130%

140%


37% 37%

37% 37%

49% 54%

67%53%


Linking Performance to Emissions Conditions: (1) Ozone

Monday and Friday

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

120%

130%

140%


Tuesday,Wednesday and Thursday

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

120%

130%

140%


Weekends and Holidays

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

120%

130%

140%


24% 24%

24%

67%67%

67%

10% 10%

10%

30%35%

37%


Linking Performance to Emissions Conditions: (2) PM2.5

Monday and Friday

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

120%

130%

140%


Tuesday, Wednesday and Thursday

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

120%

130%

140%


Weekends and Holidays

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

120%

130%

140%


37%

37%

37%

53% 68%

39%


Summary

• 2006-2009 Ozone forecasts are good.– Overall bias is +17% and error is 24%

• 2006-2009 PM2.5 forecasts are not very accurate. – May-September bias is -25% and error is 37%

• The new SOA module helped a much better 2009 PM2.5 performance– May-September bias is 8% and error is 25%

• “Single Value” forecasts for Atlanta metro is slightly in better performance than specific station forecasts. – Larger spatial variance for ozone performance, PM2.5

performance is more uniform spatially.

• Less cloud coverage, better ozone performance, but worse PM2.5 performance

• Worse PM2.5 performance in weekends and holidays– But not seen for ozone performance.


Acknowledgements

We thank Georgia EPD for funding the Hi-Res forecasts, Dr. Jaemeen Baek of our group for the new SOA module,

Dr. Carlos Cardelino of Georgia Tech for team forecasts.

Documents

Yongtao Hu 1 , M. Talat Odman 1 , Michael E. Chang 2 and Armistead G. Russell 1