WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
WFM 6311: Climate Change Risk Management
Akm Saiful Islam
Lecture-5b: Quality Controlling Observational Data
December, 2009
Institute of Water and Flood Management (IWFM)Bangladesh University of Engineering and Technology (BUET)
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Quality Controlling Observational Data
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Task 1
After selecting climate station, download daily CSV files for past 30 years to folder titled with climate station name
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Quality Controlling Observational Data
1. organizing and viewing data
2. data checks (range, visual, missing data)
3. data summaries (monthly, annual)
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
1. Organizing and Viewing DataCheck folder for 30 years of data available (remember 2009 is not complete and cannot be included)
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Opening data file in MS Excel
Double click on file in folder and file will open in MS Excel
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Save as Station Name 1979 to 2008
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Highlight Rows 1 to 23, then delete
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Scroll to bottom of dataset
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Open next year of data (1980)
Select rows 25 to 390
Copy rows
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Paste at end of Climate station dataset
Then repeat for all years
* remember start with earliest year
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Delete unnecessary columns
Select columns B, C, D, E, G, I, K, L, M, N, O
Then delete
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Delete unnecessary columns 2
Select columns E, F,G,H, J, K, L, M, N, O, P, Q
Then delete
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
This leaves Dataset with Tmax, Tmin, Tmean and Ptotal
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Format Column A for Date
Select Column A
Right click
Select Format Cells
Select date
Select 14-Mar-01 style
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Eliminate all rows dated February 29(Leap years)
In this case, 1980, 1984, 1988, 1992, 1996, 2000, 2004, 2008
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Insert Worksheet to separate data into 20-year intervals
Name
Worksheets
In this case
1979 to 1998
1999 to 2008
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Cut and paste between worksheets to separate into 20-year datasets
Full 20-year dataset should contain 7301 rows
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
2. Data Checks(range, visual, missing)
Calculate maximum value in column
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Calculate minimum value in column
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Calculate Mean
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Calculate Standard Deviation
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Max/Min/MeanCheck Max/
Min/Mean values for consistency
i.e. Tmean falls between Tmax and Tmin
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
View Data
Select column
Select Chart Wizard Icon
Select Line Chart
Select Chart sub-type
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
View Data for Tmax, Tmin, Tmean
Max Temp (��C)
-30
-20
-10
0
10
20
30
40
50
1 545 1089 1633 2177 2721 3265 3809 4353 4897 5441 5985 6529 7073
Min Temp (��C)
-40
-30
-20
-10
0
10
20
30
1 545 1089 1633 2177 2721 3265 3809 4353 4897 5441 5985 6529 7073
Mean Temp (��C)
-30
-20
-10
0
10
20
30
40
1 545 1089 1633 2177 2721 3265 3809 4353 4897 5441 5985 6529 7073
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Missing Data
Count empty cells
3 and 5 rule
If 3 consecutive days data missing, then cannot use that month’s data
If 5 days of data missing, then cannot use that month’s data
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
3. Data Summaries
Go to CCCSN website(www.cccsn.ca/training/) Click on “monthly, seasonal, annual” template
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
Copy columns G to W
WFM 6311: Climate Risk Management © Dr. Akm Saiful IslamPaste into 20-year climate worksheet at cell G1
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
WFM 6311: Climate Risk Management © Dr. Akm Saiful Islam
View summaries
Select columns to view and chart