A Hybrid Cloud Regime Methodology Used to Evaluate Southern Ocean Cloud andShortwave Radiation Errors in ACCESS

SHANNON MASON AND JENNIFER K. FLETCHER

School of Earth, Atmosphere and Environment, Monash University, Clayton, Victoria, Australia

JOHN M. HAYNES

Cooperative Institute for Research in the Atmosphere, Colorado State University, Fort Collins, Colorado

CHARMAINE FRANKLIN

Oceans and Atmosphere Flagship, Commonwealth Scientific and Industrial Research Organization, Aspendale, Victoria, Australia

ALAIN PROTAT

Centre for Australian Weather and Climate Research, Melbourne, Victoria, Australia

CHRISTIAN JAKOB

School of Earth, Atmosphere and Environment, Monash University, Clayton, and Australian Research Council

Centre of Excellence for Climate System Science, Melbourne, Victoria, Australia

(Manuscript received 11 December 2014, in final form 19 April 2015)

ABSTRACT

A deficit of shortwave cloud forcing over the Southern Ocean is persistent in many global climate models.

Cloud regimes have been widely used in model evaluation studies to make a process-oriented diagnosis of

cloud parameterization errors, but cloud regimes have some limitations in resolving both observed and

simulated cloud behavior. A hybrid methodology is developed for identifying cloud regimes from observed

and simulated cloud simultaneously.

Through this methodology, 11 hybrid cloud regimes are identified in the ACCESS1.3 model for the high-

latitude Southern Ocean. The hybrid cloud regimes resolve the features of observed cloud and characterize

cloud errors in themodel. The simulated properties of the hybrid cloud regimes, and their occurrence over the

Southern Ocean and in the context of extratropical cyclones, are evaluated, and their contributions to the

shortwave radiation errors are quantified.

Three errors are identified: an overall deficit of cloud fraction, a tendency toward optically thin low andmidtopped

cloud, andan absenceof a shallow frontal-type cloudat high latitudes and in thewarm fronts of extratropical cyclones.

To demonstrate the utility of the hybrid cloud regimes for the evaluation of changes to the model, the

effects of selected changes to the model microphysics are investigated.

1. Introduction

The representation of clouds in global climate models

(GCMs) is critical to modeling the earth’s radiative en-

ergy budget, atmospheric circulation, and hydrological

cycle, and many processes at smaller scales. Model eval-

uation studies consistently identify significant cloud errors

(e.g., Gates et al. 1999; Zhang et al. 2005; Trenberth and

Fasullo 2010) and—while GCMs are improving by some

measures (Klein et al. 2013)—subsequent cloud feed-

backs continue to be the greatest source of uncertainty

in estimates of climate sensitivity (e.g., Cess et al. 1990,

1996; Colman 2003; Dufresne and Bony 2008). Many of

the processes regulating cloud formation, composition,

and behavior—and interactions with aerosols, radiation,

Corresponding author address: Shannon Mason, School of Earth,

Atmosphere and Environment, 9 Rainforest Walk, Monash Univer-

sity, Wellington Road, Clayton VIC 3800, Australia.

E-mail: shannon.mason@monash.edu

1 AUGUST 2015 MASON ET AL . 6001

DOI: 10.1175/JCLI-D-14-00846.1

� 2015 American Meteorological Society

and dynamics—occur at scales below the resolution of

GCMs and must be parameterized. Errors related to pa-

rameterized cloud can compensate to match the bulk ob-

servations against which GCMs are tuned; for example,

the recurring ‘‘too few, too bright’’ low cloud errors in

many phase 3 of the Coupled Model Intercomparison

Project (CMIP3) models nevertheless produced near-

realistic radiative fluxes (Klein et al. 2013). To identify

these compensating errors and to inform the improve-

ment of parameterizations, there is a need for ‘‘process

oriented’’ approaches to model evaluation (Stephens

2005; Jakob 2010).

To better understand cloud processes in observa-

tions, and to evaluate them in GCMs, we identify ‘‘cloud

regimes’’—classes of cloud with common physical char-

acteristics and atmospheric contexts—and quantify both

the physical and microphysical properties of clouds and

the atmospheric processes to which they correspond.

Cloud regimes can be identified from dynamical or

thermodynamical parameters (e.g., Bony and Dufresne

2005), or directly from observed cloud characteristics by

using a clustering algorithm to identify repeating pat-

terns of cloud properties (Jakob and Tselioudis 2003;

Jakob et al. 2005). The latter cloud regimes, also called

‘‘weather states,’’ have proved useful in associating ob-

served cloud properties with dynamical and thermody-

namical conditions in the tropics (e.g., Rossow et al.

2005; Tan et al. 2013), extratropics (e.g., Gordon and

Norris 2010; Haynes et al. 2011; Oreopoulos and Rossow

2011), and globally (e.g., Tselioudis et al. 2013;Oreopoulos

et al. 2014). A challenge when using cloud regimes for

model evaluation is to identify them in such a way that

the representation of clouds in one or more GCMs can

be compared against each other and satellite observa-

tions. There have been two approaches to identifying

cloud regimes for model evaluation: In the first ap-

proach (Williams and Tselioudis 2007, hereafterWT07),

cloud regimes are identified from the simulated cloud

properties of each GCM using the same methodology as

for satellite observations. Thismethod has the advantage

of using simulated cloud directly, so that the cloud re-

gimes accurately represent the coherent structures of

cloud properties in each model. A disadvantage is that

eachGCMengenders a new set of cloud regimes thatmay

be very different from those observed; without a common

set of cloud regimes, evaluation between model and

observations is problematic. In the WT07 approach, if

simulated cloud regimes are significantly different from

observations, they may be subjectively grouped into

‘‘principal’’ cloud regimes for evaluation. Alternatively

cloud regimes can be identified from satellite observa-

tions only and then simulated clouds are assigned to cloud

regimes based on average cloud properties (Williams and

Webb 2009, hereafter WW09). The WW09 method has

the advantage of using a consistent set of observed cloud

regimes for evaluation and model intercomparison, and

this approach has been widely used in subsequent studies

(e.g., Tsushima et al. 2013; Bodas-Salcedo et al. 2012,

2014). A disadvantage of the WW09 methodology is

that the observed cloud regimes are not necessarily

representative of the coherent structures of cloud

properties in the models, so that the links between

cloud properties and processes in the GCM are un-

certain. In this paper we aim to extend these ap-

proaches by developing a hybrid methodology that

retains the structures of both observed and simulated

clouds. Hybrid cloud regimes are identified from ob-

served and simulated cloud simultaneously, ensuring

the retention of observed cloud regimes to which the

model must be compared, while including the cloud

structures peculiar to the model—and hence the errors

we aim to explore.

We apply the hybrid cloud regime methodology to a

significant cloud evaluation problem for many state-of-

the-art models: the shortwave (SW) radiation biases in

the high-latitude Southern Ocean (508–658S) during the

austral summer [December–February (DJF)]. An ex-

cess of absorbed SW radiation in this region—associated

with a deficit of cloud or cloud reflectivity—was identi-

fied in CMIP3 (Trenberth and Fasullo 2010), and per-

sists in the CMIP5models (Li et al. 2013). Evaluations of

the Met Office (UKMO) model and other CMIP5

models using the WW09 methodology (Bodas-Salcedo

et al. 2012, 2014) have attributed the radiation biases to

low and midtopped cloud regimes, especially in the

postfrontal and cold-air part of extratropical cyclones.

Observational studies have shown that the high-latitude

Southern Ocean is dominated by near-ubiquitous low

cloud, much of which is assigned by passive satellite ob-

servations to midtopped cloud regimes (Haynes et al.

2011; Bodas-Salcedo et al. 2014). While WW09 identify a

single midtopped cloud regime in the Southern Ocean,

observational studies distinguish between twomidtopped

cloud regimes with distinct dynamical contexts and radia-

tive properties (Haynes et al. 2011), and further evaluation

of these midtopped cloud regimes at high latitudes has

shown that the optically thicker midtopped cloud regime

includes instances of both stratiform cloud under strongly

subsiding conditions and shallow frontal-type clouds,

which were associated with conditions resembling the

warm conveyor belt in extratropical cyclones (Mason et al.

2014). Resolving these distinct cloud processes associated

with midtopped cloud in observations and GCMs is a pri-

ority for an extended cloud regime methodology.

The GCM used in this study is the Australian Com-

munity Climate and Earth-System Simulator, version

6002 JOURNAL OF CL IMATE VOLUME 28

1.3 (ACCESS1.3; Bi et al. 2013). ACCESS1.3 exhibits

SW radiation errors in the high-latitude SouthernOcean

during DJF typical of the persistent biases in CMIP3 and

CMIP5 models. A first-order evaluation of ACCESS1.3

indicates SWcloud radiative effect (CRESW; the difference

between outgoing fluxes at top of the atmosphere under

clear-sky and cloudy-sky conditions) errors of 48Wm22

with respect to ISCCP flux data (ISCCP-FD) observa-

tions and 38Wm22 with respect to CERES synoptic 18(SYN1deg) observations over the high-latitude Southern

Ocean (Fig. 1); in the same region the bulk cloud errors

include a 20%underestimate of total cloud cover (TCC)

and a deficit of optically thick low and midtopped cloud

(Fig. 2; see also Franklin et al. 2013a).

The purpose of this study is to develop an extended

cloud regime methodology and to demonstrate its appli-

cation to the evaluation of Southern Ocean cloud and

radiation errors in ACCESS1.3. The satellite observa-

tions and reanalysis data used and the configuration of

theGCMare described in section 2. Themethodology for

identifying hybrid cloud regimes is described in section 3,

followed by the properties and statistics of the hybrid

cloud regimes for ACCESS1.3, their contribution to the

CRESW error, and their distribution in the context of a

composite extratropical cyclone. To demonstrate the

utility of the hybrid cloud regimes for making a quanti-

tative and process-oriented assessment of the effects of

changes to the model, we test some selected changes to

FIG. 1.Maps of the CRESW bias inACCESS1.3 with respect to (top left) ISCCP-FD observations and (bottom left)

CERES SYN1deg over the Southern Hemisphere extratropics, and (right) the zonal means of the same. The high-

latitude Southern Ocean region taken as the area of interest for this study (508–658S) is indicated with thick lines.

1 AUGUST 2015 MASON ET AL . 6003

themodelmicrophysics in section 4. The applications and

limitations of the methodology for model evaluation are

discussed in section 5, with some concluding remarks.

2. Data

a. Passive satellite observations

The International Satellite Cloud Climatology Project

(ISCCP; Rossow and Schiffer 1999) combines passive

observations from geostationary and polar-orbiting satel-

lites to provide a continuous global dataset for the period

July 1983–2009. Observations of cloud-top pressure (CTP)

and cloud optical thickness (t) are made at the scale of 1–

5km.The ISCCPD1dataset consists of joint histograms of

CTP-t observations within a 280km3 280km equal-area

grid at 3-h intervals in all day-lit areas.Upwelling top of the

atmosphere (TOA) SW radiative flux observations are

obtained from the ISCCP radiative flux profile dataset

(ISCCP-FD; Zhang et al. 2005). Term CRESW is the dif-

ference between the upwellingTOAradiative fluxes under

clear-sky and cloudy-sky conditions. The sign convention

for TOA fluxes is such that the CRESW is negative (the

cloud acts to reflect more SW radiation) and CRELW is

positive (cloud inhibits LW radiation to space).

To quantify the observational uncertainty, we compare

CRESW from ISCCP-FD against that from Clouds and

the Earth’s Radiant Energy System (CERES) gridded

data (SYN1deg 3-h Ed3A; Wielicki et al. 1996), derived

from the sun-synchronous Aqua and Terra satellites.

The spatial patterns of the SW errors in ACCESS1.3

(Fig. 1) are qualitatively similar using the two observational

datasets; the zonal means differ by about 10–15Wm22

equatorward of 608S but are very similar at high latitudes.

Two summers (2006–08; DJF) of daily averages of

ISCCP D1 and TOA flux data were interpolated onto a

regular 2.58 grid: the CTP-t histograms were interpolated

according to a nearest-neighbor interpolation scheme

and radiative fluxes using linear interpolation. In keeping

with previous studies (e.g., Haynes et al. 2011; Mason

et al. 2014), daily averages of 3-h ISCCP observations

are used.

b. Reanalysis

The European Centre for Medium-Range Weather

Forecasts (ECMWF) interim reanalysis (ERA-Interim;

Dee et al. 2011) is available at 1.58 spatial and 6-h temporal

resolution. Two summers (2006–08; DJF) of ERA-Interim

data were reinterpolated onto a regular 2.58 grid using a

linear interpolation scheme. The first and second de-

rivatives of the ERA-Interim mean sea level pressure

(MSLP) were used to identify cyclone centers as described

in Field and Wood (2007). Cyclone composites are con-

structed by reinterpolating contemporary data onto a

regular 4000km 3 4000km grid centered at the MSLP

minimum of each identified extratropical cyclone.

c. Global climate model

ACCESS1.3 (Bi et al. 2013) is a coupled climate model

developed by the Centre for Australian Weather and

Climate Research (CAWCR). Its atmosphere model is

based on the UKMO Unified Model (UM) Global At-

mosphere model, version 1.0 (GA1.0; Hewitt et al. 2011).

To facilitate the consistent comparison of simulated cloud

with observations, the ISCCP satellite simulator (Klein

and Jakob 1999; Webb et al. 2001), part of the Cloud

Feedback Model Intercomparison Project (CFMIP) Ob-

servation Simulator Package (COSP; Bodas-Salcedo et al.

2008), is integrated into ACCESS1.3 (Franklin et al.

2013a). The ISCCP simulator output differs from ISCCP

observations in that subvisible cloud (t, 0:3) are in-

cluded; however, when comparing models to observa-

tions, these thin clouds are omitted, as it is assumed that

they would not be detected by the ISCCP cloud detection

algorithms (Klein and Jakob 1999; Webb et al. 2001).

ACCESS1.3 was run in atmosphere-only mode with

prescribed sea surface temperatures at N96 resolution for

two years (2006–08). The output fields were daily MSLP,

TOA SW radiative fluxes for full-sky and clear-sky con-

ditions, and CTP-t joint histograms from the ISCCP

simulator. These data were reinterpolated onto a 2.58 gridfor consistency with satellite observations and reanalysis

FIG. 2. The mean cloud bias (ACCESS1:32 ISCCP) over the

region of interest for two austral summers (DJF), presented as the

difference between CTP-t joint histograms. The part of the histo-

gram representing subvisible optical thicknesses (t, 0:3), which is

simulated in ACCESS1.3 but not observed in ISCCP, is hatched.

At top-right the TCC error is indicated for the full joint histogram

and for the visible (42 elements) joint histogram.

6004 JOURNAL OF CL IMATE VOLUME 28

data. MSLP and radiative fluxes were interpolated using a

linear interpolation scheme, and ISCCP simulator data

were interpolated using a nearest-neighbor interpolation

scheme.

3. Evaluation of ACCESS1.3

a. Hybrid cloud regime methodology

In this section we extend the existing methodologies

for assigning observed and simulated cloud properties to

cloud regimes for model evaluation. Previous approaches

have involved either clustering on the simulated cloud

properties from a model (WT07)—so that the resultant

cloud regimes accurately represent the model cloud be-

havior, but they are not necessarily comparable with

observed cloud regimes—or assigning simulated clouds

directly to predefined observed cloud regimes (WW09),

maintaining a consistent set of cloud regimes against

which GCMs can be evaluated, at the risk of not neces-

sarily resolving the cloud structures peculiar to each

model. Based on the strengths and weaknesses of the

existing approaches, we aim to combine the advantages

of both so that the extended cloud regimemethodology is

capable of resolving both the behavior of the GCM (i.e.,

the repeating structures of simulated cloud properties)

and the cloud regimes identified in observational studies.

To achieve this, we cluster observed and simulated

CTP-t histograms simultaneously. The resulting clus-

ters, which we call hybrid cloud regimes, represent a

blend of model and observation: each cloud regime may

consist of amixture of observed and simulated clouds, or

they may be made up of either mostly observed or

mostly simulated clouds. It is worth considering what

information we would expect the hybrid cloud regime

approach to resolve in some idealized scenarios. If the

GCM reproduces the observed cloud very accurately,

then we could expect the resultant hybrid cloud regimes

to be identical to the observed cloud regimes in terms of

both physical characteristics (as represented by the av-

erage CTP-t histogram of the cluster members) and

geographic distribution. Conversely, if the GCM was

unable to reproduce any cloud resembling the obser-

vations, then we would expect to identify a superset of

independent cloud regimes populated either by ob-

served or simulated clouds alone—that is, there would

be no cloud regimes jointly populated by both. In

practice we expect the results for a state-of-the-art GCM

to lie somewhere between these two extremes: it is

conceivable to identify certain observed cloud regimes

that the model fails to reproduce and some simulated

cloud regimes that do not occur in nature; we would also

expect to identify jointly populated cloud regimes that

the GCM can simulate in principle but which occur with

the wrong frequency of occurrence or geographical

distribution, or slight differences in cloud properties,

and finally some cloud regimes that are well simulated in

all respects. In the former cases, the cluster centroids

would resemble those obtained from clustering observa-

tions and simulations independently; in the latter cases, we

would find that the cluster centroid of the resulting hybrid

cloud regimes are modified somewhat from the observed

cloud regimes, reflecting the features of the simulated

members. The advantages of having this comprehensive

information and especially of using it to analyze model

errors in other fields, such as cloud radiative effects, will

become apparent when we apply this technique to the

high-latitude Southern Ocean clouds below.

Using the method for identifying cloud regimes first

described in Jakob and Tselioudis (2003), we apply the

k-means clustering algorithm (Anderberg 1973) to the

42-element state vectors of the CTP-t histograms from

both the ISCCP observations and from the ISCCP sim-

ulator running in ACCESS1.3 (omitting the simulated

t, 0:3 classes for consistency with observations), using

2 years of austral summer (DJF) data over the region of

interest (508–658S). The pool of state vectors fromwhich

the hybrid cloud regimes are identified consists of equal

parts observed and simulated cloud properties, which

are not differentiated by the clustering algorithm. Con-

sistent with Haynes et al. (2011) andMason et al. (2014),

clear conditions are not removed from the data before

clustering, and instances of clear skies will be included in

the hybrid cloud regime with the lowest total cloud

cover. Because of the ubiquity of cloud in the Southern

Ocean, completely clear skies at these scales are rare in

both observations (0.3%) and the model (0.06%).

The clustering algorithm is somewhat subjective in

that the number of clusters (k) must be specified.

Rossow et al. (2005) propose four criteria by which k can

be objectively chosen: 1) the stability of the resulting

cluster centroids across random initializations of the al-

gorithm, and for random subsets of the data; 2) avoiding

similarity between cluster centroids; 3) avoiding similar-

ity between the frequency of occurrence patterns of the

clusters in space and time; and 4) ensuring the Euclidean

distances between cluster centroids exceed the distances

between cluster members. As Rossow et al. (2005) note,

in practice criteria 1 and 4 are automatically met: the al-

gorithm is restarted with randomly seeded centroids a

number of times and the most stable solution is selected,

and the algorithm iterates until the distance between clus-

ter centroids is greater than the spread of cluster members.

Criteria 2 and 3 are intended to avoid redundant clus-

ters. However, for the purposes of a hybrid cloud regime

analysis—simultaneously clustering on two independent

populations—this is not necessarily desirable: for

1 AUGUST 2015 MASON ET AL . 6005

example, it is possible for a mostly simulated hybrid

cloud regime to occur in a similar context—and therefore

exhibit a similar spatial distribution—to a mostly observed

cloud regime. As these cases provide information about

the model errors, we would wish to resolve them.

Therefore, for the hybrid cloud regime analysis, we

increase k until the set of clusters includes the observed

cloud regimes identified previously. We note that the

hybrid cloud regimes identified here are not identical to

those identified for the broader Southern Ocean (308–658S; Haynes et al. 2011) and the refined analysis of

midtopped cloud subregimes conducted for the high-

latitude Southern Ocean (Mason et al. 2014), as the

cloud regimes identified in the former study include

lower-latitude clouds, and the cloud subregimes identified

in the latter were derived by clustering within previously

identified cloud regimes. Nevertheless, all of the features

identified in the observational studies for the region of

interest are represented in the hybrid cloud regimes.

For the area of interest in this study, two seasons

(6 months) of data provide a large enough dataset that

the hybrid cloud regimes are reliably identified. A test of

robustness was made by clustering on randomly selected

subsets of the data; the resulting cluster centroids were

found to be substantially similar for even one month of

data. We note that the relative frequencies of occur-

rence of the cloud regimes are more sensitive to inter-

annual variability; we present these values here for

evaluation of the GCM over the period in question, and

not as a climatology of the identified cloud types.

Once the hybrid cloud regimes have been identified,

each CTP-t histogram (observed or simulated) is as-

signed to one of the cloud regimes. The assignation may

be performed concurrent to the cloud regime identifi-

cation, or subsequently. Assignation is determined by

the least Euclidean distance between each CTP-t joint

histogram and the centroids of the cloud regimes. An

alternative approach used in WW09 and subsequent

studies calculates the least Euclidean distance between

the total cloud cover (TCC; the sum of the joint histo-

gram), mean cloud-top pressure (CTP), and mean cloud

albedo (a; derived from t using the ISCCP lookup table

reproduced in WW09). The WW09 methodology is in-

tended for multimodel intercomparison, wherein the

mean cloud properties are a simpler requirement for

participating modeling centers and using the full CTP-t

joint histograms requires a greater volume of data out-

put and computation. While the WW09 methodology

produces coherent results in many studies, we note that

the differences in cloud regime frequencies of occur-

rence when using the two assignation methods are not

negligible, especially for cloud types with complex dis-

tributions of cloud-top properties. A comparison of the

two methods using the observed cloud regimes of

Haynes et al. (2011) and 5 years of ISCCP-D1 data be-

tween 308 and 658S resulted in significant differences in

the frequency of occurrence (.10%) of some cloud re-

gimes (not shown). In the present application to a single

GCM, it is practicable to use the full CTP-t joint his-

tograms in order to retain the structures of cloud-top

properties, which are indicative of subgrid-scale vari-

ability in ISCCP observations (see discussion in Mace

and Wrenn 2013; Mason et al. 2014).

b. Identification and properties of hybrid cloudregimes

Eleven hybrid cloud regimes (H1–H11, ranked from

high to low CTP and from low to high t) are found to be

sufficient to represent the observed cloud regimes within

the region of interest, as well as the emergent features of

mostly simulated cloud regimes. The CTP-t joint his-

tograms represent the average of all observed and sim-

ulated members of each hybrid cloud regime (Fig. 3). It

is helpful to make a coarse classification by CTP level,

grouping the 11 hybrid cloud regimes into boundary

layer, low, midtopped, and frontal cloud regimes. The

three lowest hybrid cloud regimes (H1–H3) are pre-

dominantly boundary layer clouds representing shallow

cumulus (H1) through to cumulus–stratocumulus tran-

sition clouds (H3). The low hybrid cloud regimes (H4

andH5) are associated with a range of marine stratiform

clouds. The hybrid cloud regimes identified as mid-

topped (H6–H8) are consistent with Mason et al. (2014)

and are dominated by low cloud (see also Haynes et al.

2011; Bodas-Salcedo et al. 2014) under a range of mostly

subsiding conditions, as distinct from the shallow frontal

cloud regime (H9;Mason et al. 2014). The frontal hybrid

cloud regimes (H9–H11) also include prefrontal cirrus

(H10) and deep frontal cloud (H11); we note that, while

H9 occurs at a lower CTP level than the higher frontal

cloud regimes H10 and H11, it is displayed between the

latter in the figures to save space.

By assigning each daily CTP-t histogram to a hybrid

cloud regime, we derive the relative frequency of oc-

currence (RFO) of the hybrid cloud regimes over the

region of interest during DJF in both the observations

and ACCESS1.3, as an overall RFO (Table 1) and as

geographical distributions (Fig. 4). The overall RFO of

each cloud regime gives an initial indication of the

hybrid cloud regimes that are accurately represented

in ACCESS1.3 and those that are over and under-

produced in the model. Shallow cumulus and very low

cloud fraction scenes (H1) are strongly overproduced

in the model, as are low optical thickness low and

midlevels clouds (H4 and H6); with the exception of

deep frontal and cirrus clouds, the model severely

6006 JOURNAL OF CL IMATE VOLUME 28

underrepresents optically thick clouds at all levels (H3,

H5, H8, and H9).

Cloud fractions are underestimated in ACCESS1.3 by

10%–20% for almost all cloud regimes, consistent with

the overall deficit of TCC in the model. The tendency of

the model to produce low cloud fractions is most ap-

parent in the overproduction of the lowest cloud frac-

tion, lowest optical thickness cloud regime (H1)

throughout the area of interest (RFO of 42% compared

with 9% observed). The CTP-t histogram of H1 shows

FIG. 3. The CTP-t joint histograms representing the cluster centroids of the hybrid cloud regimes. The hybrid cloud regimes are

arranged according to the dominant features of the joint histograms with the optically thinnest and lowest cloud regime in the bottom-left

corner, and the optically thickest and highest cloud regime in the top-right corner, such that clouds of similar CTPs are comparably along

the horizontal axis and similar optical thicknesses along the vertical axis. The mean observed and simulated RFO over the regions of

interest are indicated in the top-right corner of each histogram. Where the cloud regime is underrepresented (underrepresented) in

ACCESS1.3 bymore than 50%with respect to the observed value, the histogram is bordered in blue (red). The TCC for each cloud regime

(the sum of each joint histogram) is indicated at top right of each histogram.

1 AUGUST 2015 MASON ET AL . 6007

that this cloud regime includes low amounts of optically

thin cloud from the surface to midlevels; we also note

that the simulated spatial distribution of H1 is in areas

associated not only with the observed boundary layer

cloud regimes but also with the occurrence of the mid-

topped cloud regime H7 in the high-latitude Atlantic

and Indian Oceans.

ACCESS1.3 has a systematic bias toward optically

thin (low t) low andmidtopped cloud regimes. The low-t

errors appear to lead to H1 being simulated where H2 is

observed (in the eastern edges of the Pacific and Indian

Ocean basins), and the simulated distribution of H2

downstream of the Drake Passage resembles that ob-

served ofH3.A tendency to create low-t counterparts of

observed cloud regimes gives rise to the low and mid-

topped hybrid cloud regimes H4 and H6, respectively.

H4 (predominantly simulated) and H5 (predominantly

observed) differ in mean cloud-top properties but cor-

respond very closely in terms of overall RFO (15%) and

distribution in the high-latitude Atlantic and central

Pacific oceans. Similarly, the optically thin midtopped

cloud regime H6 is simulated in the high-latitude At-

lantic and Pacific Oceans where H7 is observed. Corre-

sponding to the overprediction of these low-t cloud

regimes in ACCESS1.3, the optically thickest low and

midtopped hybrid cloud regimes are very rarely simulated.

The warm frontal cloud regime H9 is almost absent in

ACCESS1.3, compared with an observed RFO of 16%,

and has no clear compensating cloud regimes as identi-

fied above. As distinct from a systematic deficiency in

cloud properties, this cloud error may be related to the

representation of a dynamical process in the model. The

higher frontal cloud regimes H10 andH11 are simulated

with similar RFO to observations, but with a less co-

hesive distribution around the midlatitude storm track.

With an evaluation of the occurrence of the hybrid cloud

regimes in the model and observations, the major cloud

errors in the GCM are made explicit in a process-oriented

way. The tendencies toward low TCC and low-t cloud

are manifest in the predominantly simulated low and

midtopped hybrid cloud regimes (H4 and H6) and the

overprediction of sparse cloud regime (H1). While the

prefrontal cirrus (H10) and frontal (H11) hybrid cloud

regimes are relatively wellrepresented in the GCM, the

shallow frontal hybrid cloud regime (H9) is almost absent.

We next turn our attention to quantifying how these cloud

errors contribute to the overall SW radiation bias.

c. Contributions to SW radiation bias

Eleven hybrid cloud regimes have been identified

from passive satellite observations and simulated cloud

properties in ACCESS1.3. The most significant and re-

curring deviations from observed cloud properties give

rise to hybrid cloud regimes that are not frequently

found in the observations: these predominantly simu-

lated hybrid cloud regimes provide an initial indication

of themajor cloud errors in theGCM. By associating the

hybrid cloud regimes with radiation errors, we can

quantify the relative contributions of these major cloud

errors to the total SW cloud radiative effect bias in the

high-latitude Southern Ocean in ACCESS1.3.

The cloud radiative effect (CRE) is the difference

between clear-sky and cloudy-sky TOA radiative fluxes.

Since outgoing fluxes at TOA are defined as positive and

cloud typically has the effect of increasing reflected SW

radiation, values of CRESW are negative. The hybrid

cloud regimes with the lowest observed CRESW are

generally those with the optically thickest clouds and the

highest total cloud cover (Table 1); however, the spatial

distribution of the cloud regime is also important. We

note that the differences between CRESW with respect

to ISCCP-FD and CERES observations are greatest

for the optically thick cloud regimes (e.g., H3, H5, H8,

H9, andH11), where the observational uncertainly is as

much as 15Wm22. We therefore include both obser-

vational datasets in what follows, but unless otherwise

TABLE 1. The observed (obs) and simulated (sim) properties of the hybrid cloud regimes.

TCC (%) RFO (%) CRESW (Wm22)

CTP class Cloud regime Obs Sim Obs Sim Obs (ISCCP-FD) Obs (CERES) Sim

Boundary layer H1 63.0 42.8 8.9 41.8 276.8 277.1 288.0

H2 74.6 71.6 7.1 11.5 286.0 281.0 2104.2

H3 91.9 79.7 9.1 0.1 2149.6 2134.3 2124.4

Low H4 85.5 72.6 2.1 15.4 2101.8 299.1 2103.4

H5 93.9 76.2 15.1 0.4 2154.8 2140.3 2120.9

H6 89.7 66.4 0.2 8.5 296.3 299.8 295.7

Midtopped H7 89.5 73.6 14.2 6.6 2116.0 2113.6 299.6

H8 95.1 74.6 10.5 0.1 2159.8 2148.6 2111.5

H9 96.4 76.5 16.9 0.5 2171.8 2158.8 295.4

Frontal H10 93.3 73.6 8.9 10.7 2140.1 2133.3 287.4

H11 97.2 75.0 6.9 4.3 2194.0 2178.2 291.2

6008 JOURNAL OF CL IMATE VOLUME 28

FIG. 4. Maps of the observed and simulated RFO of the hybrid cloud regimes over the region of interest. As in

Fig. 3, the hybrid cloud regimes are arranged according to the dominant features of the CTP-t histogram, with the

optically thinnest and lowest regime in the bottom-left map and the optically thickest and highest cloud regime in

the top-right map. Note that the simulated RFO of H1 is on a different color scale to resolve the significant over-

production of this hybrid cloud regime in ACCESS1.3.

1 AUGUST 2015 MASON ET AL . 6009

stated, the DCRESW is presented with respect to

ISCCP-FD.

The mean SW cloud radiative effect bias (DCRESW)

associated with each hybrid cloud regime is calculated by

subtracting observed values of CRESW from the simu-

lated values for the instances where each hybrid cloud

regime is identified. Accordingly, positive DCRESW in-

dicate insufficent—or insufficiently reflective—cloud in

themodel, and negative biases indicate too-cloudy or too-

reflective cloud errors.

The total Southern Ocean DJF DCRESW can be de-

composed (followingWT07) into parts corresponding to

errors in the cloud regime frequency of occurrence or

spatial distribution (RFO errors), errors in cloud regime

radiative properties (CRE errors), and a cross term of

covariant errors. The total bias is the sum of decom-

posed errors for all cloud regimes, such that

DCRESW 5 �n

r51

CRESWr � DRFOr|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}

RFO errors

1 �n

r51

CRESWr � DRFOr|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}

CRE errors

1 �n

r51

CRESWr � DRFOr|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}

cross term

. (1)

The decomposed DCRESW associated with each hybrid

cloud regime (Fig. 5) is dominated by RFO-related er-

rors; this is partially a consequence of the hybrid cloud

regime methodology, in which simulated clouds that are

sufficiently different in their cloud-top properties from

observed clouds form new, predominantly simulated,

clusters. This is especially true for the optically thin low

and midtopped hybrid cloud regimes that are almost

never observed (H4 and H6). In comparing the magni-

tude of DCRESW for ISCCP-FD and CERES observa-

tions, we note that the optically thickest cloud regimes

exhibit lower errors with respect toCERES observations;

as noted earlier, this corresponds to a total error of

38Wm22. To avoid redundancy in the following discus-

sion, we refer to SW errors with respect to ISCCP-FD

observations unless otherwise stated.

At each CTP level, the optically thinner cloud regimes

make a small or negative contribution to the DCRESW,

while the optically thick hybrid cloud regimes are asso-

ciated with larger positive biases. This indicates that the

low-t, low-TCC hybrid cloud regimes—which occur

with similar distributions to their high-t counterparts—

partially compensate for the latter. In the case of the low

cloud regimes, the total error associated with the low-

RFO and low CRESW of H5 (23Wm22) is partially

compensated by the high RFO of H4 (213Wm22); the

netDCRESW related to low cloud is 10Wm22. Similarly,

FIG. 5. The CRESW bias in ACCESS1.3 with respect to ISCCP-FD radiative flux observa-

tions. The biases are decomposed into parts related to errors in RFO and CRESW, and a cross

term. Thick (thin) bars indicate the total CRESW bias associated with each cloud regime with

respect to ISCCP-FD (CERES) observations, and the total mean bias over the region of in-

terest is indicated with a thick black line. Pale bars indicate the net contribution to the CRESW

error across each CTP-level class.

6010 JOURNAL OF CL IMATE VOLUME 28

the overprediction of the optically thinnest midtopped

hybrid cloud regimeH6partially compensates (28Wm22)

for the lowRFOofH7; themajority of the error associated

with midtopped cloud (18Wm22) is related to the under-

prediction and low CRESW of the optically thicker H8.

The strongest compensating (negative) contribution

to DCRESW is associated with the low-TCC, low-t hy-

brid cloud regime H1. Despite being associated with the

lowest observed cloud fractions and cloud brightnesses,

H1 contributes 230Wm22 to the net DCRESW. This is

due to H1 accounting for more than 40% of the high-

latitude Southern Ocean cloud in ACCESS1.3. The

dominance of a low-TCC and low-t cloud regime is

symptomatic of the two key systematic biases identified

in Franklin et al. (2013a).

The greatest positive contributor to the netDCRESW is

H9, the warm frontal cloud regime observed at high lat-

itudes but hardly represented in ACCESS1.3. The deficit

in the occurrence of this cloud regime is not compensated

for by any optically thinner cloud regimes. The frontal

cloud regimes H10 and H11 contribute around 15Wm22

to the overall bias.Unlike the cloud regimes at other CTP

levels, the largest portion of these biases are due to CRE-

related errors; the RFO-related errors may be related to

the tendency in ACCESS1.3 toward a less coherent band

of frontal cloud at lower latitudes (Fig. 4).

d. Dynamical contexts of the hybrid cloud regimes

We have used hybrid cloud regimes to identify the

major shortcomings in the simulation of clouds in the

high-latitude Southern Ocean in ACCESS1.3 and have

quantified their contributions to the SW radiation er-

rors. It remains to investigate if the hybrid cloud regimes

are associated with consistent dynamical and thermody-

namical processes. In observational studies it is common

to characterize cloud regimes by their contemporary

meteorology derived from reanalyses (e.g., Gordon and

Norris 2010; Haynes et al. 2011; Mason et al. 2014). This

approach should be well suited to GCMs, wherein these

fields are directly available; however, direct comparisons

of dynamical fields inGCMs and reanalysis are frustrated

by possible errors in model dynamics. An alternative

approach is to consider cloud regimes in the context of a

composite extratropical cyclone, the structure of which is

both well understood in terms of observed dynamical and

thermodynamical structure and well resolved by climate

models (Catto et al. 2010). An evaluation of Southern

Hemisphere extratropical cyclones in an earlier version

of the ACCESS model, modified to use the same cloud

scheme as ACCESS1.3 (Govekar et al. 2014), found that

the circulation and dynamical variables were significantly

weaker than in reanalyses and showed that the deficits of

low cloud in this context are consistent with the broader

evaluation of clouds in ACCESS1.3 (Franklin et al.

2013a) and in other models. Extratropical cyclones are

the dominant synoptic-scale feature in the high-latitude

SouthernOcean in terms of both cloud and precipitation

(Bodas-Salcedo et al. 2014; Papritz et al. 2014), and

cloud regimes have been used effectively to evaluate

cloud in this context (e.g., Bodas-Salcedo et al. 2012,

2014). However, we note that not all dynamical contexts

relevant to the high-latitude Southern Ocean are nec-

essarily represented within the composite extratropical

cyclone. The clouds associated with other features in the

high-latitude Southern Ocean, such as anticyclones and

mesoscale cyclones, are of considerable interest for

model evaluation but are not considered here.

We identify extratropical cyclones in observations and

simulations as described in Field andWood (2007) using

MSLP from ERA-Interim to identify cyclone centers

contemporary to the satellite observations. Cloud re-

gime occurrence and TOA radiative flux fields from

observations and ACCESS1.3 are reinterpolated on to a

2000km 3 2000km grid centered at each MSLP mini-

mum, and candidates are filtered to select only cyclones

with centers from 508 to 658S latitude.

The observedRFOs of the hybrid cloud regimes in the

context of the composite cyclone (Fig. 6) agree well with

other composite cyclone studies (e.g., Bodas-Salcedo

et al. 2012, 2014), with some key differences. The most

physically important difference is that the midtopped

cloud regimes identified in this study are located in

separate and coherent parts of the extratropical cyclone.

This reinforces the distinction between optically thin

and optically thickmidtopped, and shallow frontal cloud

regimes made in Mason et al. (2014). The profiles of

dynamical and thermodynamical properties from re-

analysis (not shown) are consistent with those presented

in previous studies (e.g., Gordon and Norris 2010; Haynes

et al. 2011; Mason et al. 2014). The fronts associated with

the composite extratropical cyclone are characterized by

the occurrence of the cirrus (H10) and deep frontal (H11)

cloud regimes. H10 is observed farther from the cyclone

center and appears to be associated with both prefrontal

cirrus and other high and thin cloud in other contexts.

H11 is found along the warm and cold fronts. H9 occurs

predominantly near the cyclone center and into the cold

sector, and resembles the warm conveyor belt (WCB)

flow that overshoots the warm front; we note that a

similar midtopped cloud subregime identified in Mason

et al. (2014) was associated with conditions resembling

that of theWCB. The warm sector is also associated with

the shallow cloud regime H3. The cold sector of the ex-

tratropical cyclone consists of easterly flow ahead of the

warm front turning equatorward behind the storm center,

where it meets the descending dry sector. The cold sector

1 AUGUST 2015 MASON ET AL . 6011

FIG. 6. Observed (above) and simulated (below) relative RFOof each hybrid cloud regime in the context of the composite extratropical

cyclone. The cloud regimes are organized according to optical thickness (horizontal axis) and CTP (vertical axis). Note that the simulated

RFO of H1 is on a different color scale to resolve the significant overproduction of this hybrid cloud regime in ACCESS1.3. A cartoon

illustrating the orientation of warm and cold fronts and warm and cold sectors in Southern Hemisphere extratropical cyclones is shown at

top left.

6012 JOURNAL OF CL IMATE VOLUME 28

is dominated by low and midtopped stratiform cloud

regimes H5 and H8, which form under subsiding con-

ditions ahead of the warm front. These stratiform clouds

transition to H7 in the postfrontal region of cold-air

advection, and finally in the driest section the shallow

cumulus (H2) dominates.

The ACCESS1.3 cloud errors in the context of the

extratropical cyclone are consistent with those identified

above. The frontal cloud regime H11 is found closest to

the storm center: the shallow frontal cloud regime H9 is

not present near the warm front, while the optically

thinner H10 is found throughout the storm center and

inner frontal region. The compensating relationships

between predominantly simulated and predominantly

observed hybrid cloud regimes are evident, indicating

the consistent low-t and low TCC biases. Where H3 is

observed in the warm sector, its low-t counterpart H2 is

simulated. The cold sector is dominated by the low-t and

low-TCC biases, indicated by the overproduction of H1,

H4, and H6.

The occurrence of the hybrid cloud regimes in the

context of the extratropical cyclone illustrate consistent

relationships between cloud properties and dynamical

and thermodynamical conditions in both the observa-

tions and the model, reinforcing the corresponding

spatial distributions of the mostly observed and mostly

simulated hybrid cloud regimes over the high-latitude

Southern Ocean. In both the composite cyclone and

Southern Ocean maps, the significant overproduction of

the low-TCC cloud regime H1 and the compensation of

low-t cloud regimes H2, H4, and H6 for their higher-t

counterparts are evident. These overpredicted optically

thin low and midtopped cloud regimes make negative

contributions to the SW radiation errors but only par-

tially compensate for the lack of brighter clouds. This is

consistent with the evaluation of GA2.0 and GA3.0 in

Bodas-Salcedo et al. (2012), where the systematic bias

toward optically thinner cloud led to the overassignation

of shallow cumulus and cumulus–stratocumulus transi-

tion cloud regimes, especially in the cold-air section of

extratropical cyclones. In the present study the use of

hybrid cloud regimes permits many of these clouds to be

assigned to distinct optically thinner stratocumulus and

midtopped cloud regimes; nevertheless, the significant

overproduction of H1 in this part of the extratropical

cyclone is indicative of the magnitude of the cloud

fraction and optical thickness errors in the model.

In contrast to the cloud regimes used in Bodas-Salcedo

et al. (2012), wherein a single midtopped cloud regime

was identified more than 30% of the time in both the

cold-air sector and the warm front part of the composite

extratropical cyclone, the distinction between shallow

frontal (H9), optically thin (H7), and stratiform (H8)

midtopped hybrid cloud regimes are confirmed by their

distributions through the extratropical cyclone. The

single largest contributor to the CRESW error is the

near-absence of H9 in ACCESS1.3.

4. Evaluation of parameterization changes

We have used the hybrid cloud regimes to associate

cloud and radiation errors in ACCESS1.3 in a process-

oriented way. The identification of hybrid cloud regimes

is a diagnostic of the simulated cloud properties in aGCM,

making this approach suited to quantifying the sensitivity

of the errors to changes made to the model. In this section

we illustrate the use of the hybrid cloud regimes to eval-

uate the sensitivity of the radiation errors to targeted

changes to cloud parameterizations in ACCESS1.3 and to

quantify their effects on the hybrid cloud regimes.

ACCESS1.3 is characterized by systematic deficits of

both TCC and t affecting low and midtopped cloud in

the high-latitude Southern Ocean. The low-t bias sug-

gests that errors in cloud properties relating to micro-

physical parameterizations may be affecting the cloud

radiative effect. Relevant cloud properties include cloud

thermodynamic phase, droplet size and concentration,

and total cloud amount or cloud lifetime, which are af-

fected by precipitation rates and mixing with dry air.

To target the optical thickness biases, we make three

changes to the representation of clouds intended to re-

duce the Southern Ocean cloud and radiative biases in

ACCESS1.3. A new autoconversion scheme (Franklin

2008) is implemented; it was shown by Franklin et al.

(2013b) to increase the occurrence of optically thicker

low clouds and to reduce the overestimate of drizzle in

tropical boundary layer clouds. The fall speeds of the ice

aggregate category are reduced by one-third; Franklin

et al. (2013a) demonstrated that by reducing these fall

speed, the occurrence of optically thicker low- and

midlevel clouds in ACCESS1.3 was increased over the

Southern Ocean. A change is also made to the erosion

time-scale parameter that controls the rate at which the

liquid cloud fraction is reduced by the mixing of cloudy

air with drier environmental air. This parameter takes

the value of 24 3 1025 s21 in the control version of

ACCESS1.3 and is reduced by half in themodified cloud

parameterizations experiment. While this change di-

rectly affects the cloud fraction, it also indirectly affects

the microphysical processes by changing the in-cloud

water contents that are used in the microphysical pa-

rameterizations, such as the autoconversion scheme.We

note that the autoconversion, ice fall speed, and erosion

rate are not the only possible microphysics changes that

could be made in order to increase cloud brightness.

Changes to ice particle size, ice deposition rate, and

1 AUGUST 2015 MASON ET AL . 6013

heterogeneous nucleation temperature were also con-

sidered; the effects of the selected changes compare

most favorably in a bulk evaluation of cloud properties

againstCloudSat andCloud–Aerosol Lidar and Infrared

Pathfinder Satellite Observations (CALIPSO) observa-

tions (not shown). Testing the three selected changes in-

dependently shows that the largest change in both the

ISCCP diagnostics and CRESW comes from the different

autoconversion scheme (not shown); the other two cloud

microphysics changes also positively affect the radiative

properties of the clouds over the Southern Ocean. The

combination of the three changes produces the lowest

Southern Ocean CRESW bias in ACCESS1.3 and is there-

fore used in this experiment. The positive effect of reducing

drizzle rates and increasing cloud optical thickness by

changing the autoconversion scheme and ice fall speed is

consistent with the improved Southern Ocean CRESW

biases betweenGA2.0 andGA3.0 shown in Bodas-Salcedo

et al. (2012) and attributed to the introduction to the latter

model of a prognostic rain formulation and amore accurate

rain fall speed parameterization in GA3.0 (documented in

Walters et al. 2011).

While our interest is in the sensitivity of the Southern

Ocean cloud and radiation errors to these microphysics

changes, their global impacts have also been investigated

individually and in combination. Franklin et al. (2013b)

showed that the Franklin (2008) autoconversion scheme

led to more stratocumulus and less drizzle in the tropics,

and an increase in stratocumulus and fair-weather cu-

mulus cloud-top height attributed to a stronger cloud

radiative effect driving enhanced entrainment at cloud

top. Franklin et al. (2013a) showed that a reduction in ice

fall speeds resulted in increasedmidlevel cloud fraction in

the tropical warmpool and optically thicker high cloud.A

reduction in the erosion rate leads to increased low- and

midlevel liquid cloud fraction. The combined changes

have a significant impact on the global DJF CRESW error

with respect to ISCCP-FDobservations, which is reduced

from 5.7Wm22 for the control run to 22.6Wm22 with

the modified microphysics.

ACCESS1.3 is run in atmosphere-only mode with the

modified cloud microphysics in the same way as for the

initial model evaluation. Two years of DJF data, including

CTP-t histograms from the ISCCP simulator, MSLP, and

TOA radiative fluxes, are processed as described earlier.

Instead of generating a new set of hybrid cloud regimes,

the CTP-t histograms are assigned to the cloud regimes

derived from the standard ACCESS1.3 model; the differ-

ences in the RFO and CRESW can be compared directly,

or summarized according to their contributions to the

DCRESW.

The decomposed CRESW bias summarizes the effects

of the modified microphysics on the radiation errors in

ACCESS1.3 (Fig. 7). The total effect over the area of

interest is a 10Wm22 reduction in the magnitude of the

overall CRESW bias: approximately 20%–25% of the

FIG. 7. As in Fig. 5, but comparing the control simulation (solid) against the modified mi-

crophysics case (hatched). Thick (thin) bars indicate the total CRESW bias associated with each

cloud regime with respect to ISCCP-FD (CERES) observations. Pale and very pale bars in-

dicate the combined contribution to the CRESW error across each CTP-level class in the

original and modified cloud microphysics parameterizations.

6014 JOURNAL OF CL IMATE VOLUME 28

total error with respect to ISCCP-FD and CERES ob-

servations. The majority of this improvement is due to a

systematic increase of t. The CRESW-related component

of the error for each cloud regime decreases by as much

as 2.0Wm22, including an increase in the magnitude of

the negative errors associated with the optically thin hy-

brid cloud regimes (H1, H2, and H4), which act to com-

pensate for the absent optically thicker cloud regimes.

Where the changes in the optical thickness of a cloud

regime are large enough, instances of cloud previously

belonging to one hybrid cloud regime may be assigned to

an optically thicker hybrid cloud regime. These changes

in cloud regime assignations lead to changes in the RFO-

related error of both the prior and subsequent cloud

regime: this is most apparent in the compensating RFO-

related errors associatedwith the optically thinmidtopped

hybrid cloud regimes H6 and H7—each is reduced in

magnitude by around 5Wm22, while the net bias associ-

ated with midtopped cloud overall is largely unchanged.

The RFO of H1 decreases in a similar manner, corre-

sponding to a 3Wm22 improvement in the RFO-related

bias—a net increase in the overall CRESW error, since H1

strongly compensates for the lack of optically thicker

cloud regimes. Reductions of the errors associated with

the frontal cloud regimes H10 and H11 are partly due to

increased t and partly due to increasedRFOof both cloud

regimes. This shift in cloud properties leads to reduced

RFO of the optically thinnest cloud regimes (H1 andH6)

and highermean t within cloud regimes, with the greatest

net effect being a reduction of errors associated with low

and frontal clouds of around 4Wm22. In several cases the

increase in t increased the magnitude of the compensat-

ing CRESW bias (H2 and H4), or it had little effect (H8

and H9).

The hybrid cloud regime approach allows us to resolve

the contexts in which the changes to the microphysics

had the most—and least—effect on the radiation errors.

This evaluation illustrates the benefits of the method-

ology: major model errors can be explicitly identified,

rather than assigned to an adjacent observed cloud re-

gime. We can quantify how the model errors compen-

sate for, or contribute to, the total radiation error, and

we can evaluate changes to the model to determine

whether an improvement to the total radiation error is

the result of an increased compensating bias or from a

shift toward the observed cloud state.

We reiterate that this experiment is intended to illus-

trate the use of the hybrid cloud regimes to evaluate

model changes in a process-oriented way. The micro-

physics changes targeted the optical thickness biases in

low and midtopped clouds, with the overall result of a

20%–25% improvement in the total Southern Ocean

CRESW error, due to a brightening of low andmidtopped

clouds and an increased occurrence of high and frontal

clouds. The largest single contributor to the radiation

error, the shallow frontal cloud regime, which appears to

be poorly simulated in ACCESS1.3, was not improved

by these changes. It is likely that there are many other

interrelated and unrelated contributors to the overall

SW radiation errors in GCMs. For example, Bodas-

Salcedo et al. (2012) evaluated modifications to the di-

agnosis of the shear-dominated boundary layer in

GA3.0 (ACCESS1.3 is most similar to GA1.0), dem-

onstrating a significant reduction in the CRESW bias

related to enhanced production of stratocumulus cloud

in the cold-air part of the composite extratropical cy-

clone, and concluded that further work was needed to

increase the optical thickness of low and midtopped

cloud regimes in the model.

5. Discussion and conclusions

We have presented a hybrid methodology for identi-

fying cloud regimes from satellite observations and

model-simulated cloud properties simultaneously. This

approach expands on previous methodologies for iden-

tifying cloud regimes for model evaluation, with the

advantage that the cloud regimes include a fixed reference

to the observed cloud properties against which the models

are evaluated, while also permitting cloud regimes that are

peculiar to the model. The emergent hybrid cloud regimes

include pairs of cloud regimes with similar spatial distri-

butions and dynamical contexts, where one hybrid cloud

regime is mostly simulated and the other is mostly ob-

served; the differences between these pairs of hybrid cloud

regimes relate to the major cloud errors in the GCM.

Based on twoDJFs of simulated and observed cloud data,

we identify 11 hybrid cloud regimes with which to evaluate

high-latitude Southern Ocean clouds in ACCESS1.3. We

use the cloud regimes to associate errors in cloud

properties with SW radiation errors and to describe the

dynamical context of the cloud regimes as inferred from

composite extratropical cyclones.

Consistent with Franklin et al. (2013a), total cloud frac-

tion inACCESS1.3 is underpredicted, which contributes to

the weak CRESW of most cloud regimes. The cloud regime

associatedwith the lowest cloud fraction is overproduced in

the model, especially in the cold and dry sectors of extra-

tropical cyclones and throughout the high-latitude South-

ern Ocean; while the CRESW of this cloud regime is

relatively weak, its strong overprediction has a significant

compensating effect (230Wm22) in the net SW bias.

Low t biases in ACCESS1.3 were found across almost

all cloud types. In some cases this is manifest in mostly

simulated cloud regimes that are produced in place of

optically thicker counterparts in observations, especially

1 AUGUST 2015 MASON ET AL . 6015

in the cold and dry sectors of extratropical cyclones.

These emergent cloud regimes partially compensate for

optically thicker cloud regimes not well represented in

the model. A combination of three microphysics changes

targeting low cloud optical thickness were shown to

mitigate the total SW radiation errors by 20%–25%. The

result was a systematic increase in cloud brightness, which

had the greatest net improvement on the CRESW of the

low and frontal cloud regimes. Changes to the cloud mi-

crophysics to increase cloud optical thickness may con-

stitute part of an improvement to the Southern Ocean

cloud and radiation biases identified in ACCESS1.3, but

in this case it was not sufficient to address some of the

largest deficits in the optical thickness of stratiform

clouds. This is consistent with the model evaluation of

Bodas-Salcedo et al. (2012), who showed that improving

the diagnosis of the shear-dominated boundary layer in a

related GCM improved the simulation of stratocumulus

cloud in the cold-air part of the composite extratropical

cyclone but identified a need to further increase the op-

tical thickness of low and midtopped cloud.

The largest contributor to the SW radiation bias over

the Southern Ocean is the shallow frontal cloud regime

observed at high latitudes and in the warm fronts of

extratropical cyclones. These clouds were very rarely

identified in the model, and a significant compensating

relationship with an optically thinner cloud regime was

not evident. The other frontal cloud regimes are also

generally too optically thin: the cirrus cloud regime is

simulated too frequently, especially close to the center

of extratropical cyclones; and the frontal cloud regime is

simulated infrequently and without the distinct mid-

latitude storm-track distribution found in observations.

While the structure and frequency of Southern Ocean

storms were sufficiently well represented in ACCESS1.3

to carry out a comparison of composite extratropical

cyclones, we note that the significant errors in frontal

cloud structure, especially relating to shallow frontal

cloud at high latitudes, are consistent with the dynami-

cally weak midlatitude storms identified in Govekar

et al. (2014) for a related version of the ACCESSmodel.

The insignificant impact on the shallow frontal cloud

regime of the changes to the erosion rate and ice fall

speeds—which could be expected to increase midlevel

cloud fraction—suggests further work is required to

evaluate the thermodynamics and dynamics of fronts.

We note that considering the Southern Ocean cloud

regimes only in the context of extratropical cyclones is

not necessarily representative. For example, while the

shallow frontal cloud regime makes the largest net

contribution to the SW bias, the warm front does not

correspond to the region of greatest SW bias in the

context of the composite extratropical cyclone—the

cold-air advection in the cold and dry sectors (e.g.,

Bodas-Salcedo et al. 2012, 2014) where errors in total

cloud cover and optical thickness dominate, and where

modifications to the diagnosis of the shear-dominated

boundary layer have had some success in mitigating the

total error (Bodas-Salcedo et al. 2012). These two distinct

dynamical contexts—cold-air advection and shallow

frontal conditions—are each associated with midtopped

cloud and are assigned to the same cloud regime in studies

using WW09 cloud regime identification. We have dem-

onstrated that the hybrid cloud regimes in this study are

capable of distinguishing between midtopped cloud in

these contexts, but further work is required to explore the

representation of these distinct cloud processes: more

detailed considerations of cold-air outbreaks and high-

latitude storms may be an effective approach.

The approach of identifying pairs of mostly simulated

and mostly observed hybrid cloud regimes has proved

useful: the pairs were shown to be simulated with similar

spatial distributions to their observed counterparts, and

these relationships were also found in the dynamical

context of a composite extratropical cyclone. However,

more work is required to quantify the strength of the

associations between these cloud regimes with dynami-

cal contexts in the model and observations, perhaps by

using the model in hindcast mode.

The hybrid cloud regimes have been identified for a

singleGCM, and this approach is not immediately suited

to the evaluation of multiple models. However, the hy-

brid cloud regimes provide a fixed reference to cloud

errors in the model, making this a promising tool for

quantifying the effects of changes to the model on the

properties and statistics in a process-oriented way.

While the Southern Ocean SW radiation errors in

ACCESS1.3 are representative of radiation biases inmany

state-of-the-art GCMs, the nature and causes of these

biases are almost certainly not the same in each model.

Acknowledgments.This researchwas supportedbyARC

Discovery and Linkage Grant Schemes (DP130100869 and

LP0883961), and by the ARC Centre of Excellence for

Climate System Science (CE110001028). We are grateful

to Alejandro Bodas-Salcedo and Keith Williams at the

UKMO for their generous cooperation and feedback.

REFERENCES

Anderberg, M. R., 1973: Cluster Analysis for Applications. Prob-

ability and Mathematical Statistics Series, Vol. 19, Academic

Press, 359 pp.

Bi, D., and Coauthors, 2013: The ACCESS coupled model:

Description, control climate and evaluation. Aust. Meteor.

Oceanogr. J., 63, 41–64.

Bodas-Salcedo, A., M. J. Webb, M. E. Brooks, M. A. Ringer, K. D.

Williams, S. F. Milton, and D. R. Wilson, 2008: Evaluating

6016 JOURNAL OF CL IMATE VOLUME 28

cloud systems in the Met Office global forecast model using

simulated CloudSat radar reflectivities. J. Geophys. Res., 113,

D00A13, doi:10.1029/2007JD009620.

——, K. D. Williams, P. R. Field, and A. P. Lock, 2012: The surface

downwelling solar radiation surplus over the SouthernOcean in

the Met Office model: The role of midlatitude cyclone clouds.

J. Climate, 25, 7467–7486, doi:10.1175/JCLI-D-11-00702.1.

——, and Coauthors, 2014: Origins of the solar radiation biases

over the Southern Ocean in CFMIP2 models. J. Climate, 27,

41–56, doi:10.1175/JCLI-D-13-00169.1.

Bony, S., and J.-L. Dufresne, 2005: Marine boundary layer clouds

at the heart of tropical cloud feedback uncertainties in climate

models. Geophys. Res. Lett., 32, L20806, doi:10.1029/

2005GL023851.

Catto, J. L., L. C. Shaffrey, andK. I.Hodges, 2010: Can climatemodels

capture the structure of extratropical cyclones? J. Climate, 23,

1621–1635, doi:10.1175/2009JCLI3318.1.

Cess, R. D., and Coauthors, 1990: Intercomparison and in-

terpretation of climate feedback processes in 19 atmospheric

general circulation models. J. Geophys. Res., 95, 16 601–

16 615, doi:10.1029/JD095iD10p16601.

——, and Coauthors, 1996: Cloud feedback in atmospheric general

circulation models: An update. J. Geophys. Res., 101, 12 791–12 794, doi:10.1029/96JD00822.

Colman, R., 2003: A comparison of climate feedbacks in general

circulation models. Climate Dyn., 20, 865–873, doi:10.1007/s00382-003-0310-z.

Dee, D. P., and Coauthors, 2011: The ERA-Interim reanalysis:

Configuration and performance of the data assimilation sys-

tem. Quart. J. Roy. Meteor. Soc., 137, 553–597, doi:10.1002/qj.828.

Dufresne, J.-L., and S. Bony, 2008: An assessment of the primary

sources of spread of global warming estimates from coupled

atmosphere–oceanmodels. J. Climate, 21, 5135–5144, doi:10.1175/2008JCLI2239.1.

Field, P. R., and R. Wood, 2007: Precipitation and cloud structure

in midlatitude cyclones. J. Climate, 20, 233–254, doi:10.1175/JCLI3998.1.

Franklin, C. N., 2008: A warm rain microphysics parameterization

that includes the effect of turbulence. J. Atmos. Sci., 65, 1795–

1816, doi:10.1175/2007JAS2556.1.

——, Z. Sun, D. Bi, M. Dix, H. Yan, and A. Bodas-Salcedo, 2013a:

Evaluation of clouds in ACCESS using the satellite simulator

package COSP: Global, seasonal, and regional cloud proper-

ties. J. Geophys. Res. Atmos., 118, 732–748, doi:10.1029/

2012JD018469.

——,——,——,——,——, and——, 2013b: Evaluation of clouds

in ACCESS using the satellite simulator package COSP:

Regime-sorted tropical cloud properties. J. Geophys. Res.

Atmos., 118, 6663–6679, doi:10.1002/jgrd.50496.

Gates, W. L., and Coauthors, 1999: An overview of the results

of the Atmospheric Model Intercomparison Project

(AMIP I). Bull. Amer. Meteor. Soc., 80, 29–55, doi:10.1175/

1520-0477(1999)080,0029:AOOTRO.2.0.CO;2.

Gordon, N., and J. Norris, 2010: Cluster analysis of midlatitude

oceanic cloud regimes: Mean properties and temperature

sensitivity. Atmos. Chem. Phys., 10, 6435–6459, doi:10.5194/

acp-10-6435-2010.

Govekar, P. D., C. Jakob, and J. Catto, 2014: The relationship

between clouds and dynamics in Southern Hemisphere ex-

tratropical cyclones in the real world and a climate model.

J. Geophys. Res. Atmos., 119, 6609–6628, doi:10.1002/

2013JD020699.

Haynes, J. M., C. Jakob, W. B. Rossow, G. Tselioudis, and

J. Brown, 2011:Major characteristics of SouthernOcean cloud

regimes and their effects on the energy budget. J. Climate, 24,

5061–5080, doi:10.1175/2011JCLI4052.1.

Hewitt, H. T., D. Copsey, I. D. Culverwell, C. M. Harris, R. S. R.

Hill, A. B.Keen,A. J.McLaren, and E. C.Hunke, 2011: Design

and implementation of the infrastructure of HadGEM3: The

next-generation Met Office climate modelling system. Geosci.

Model Dev., 4, 223–253, doi:10.5194/gmd-4-223-2011.

Jakob, C., 2010: Accelerating progress in global atmosphericmodel

development through improved parameterizations: Chal-

lenges, opportunities, and strategies.Bull. Amer. Meteor. Soc.,

91, 869–875, doi:10.1175/2009BAMS2898.1.

——, and G. Tselioudis, 2003: Objective identification of cloud

regimes in the tropical western Pacific.Geophys. Res. Lett., 30,2082, doi:10.1029/2003GL018367.

——, ——, and T. Hume, 2005: The radiative, cloud, and thermo-

dynamic properties of the major tropical western Pacific cloud

regimes. J. Climate, 18, 1203–1215, doi:10.1175/JCLI3326.1.Klein, S. A., and C. Jakob, 1999: Validation and sensitivities of

frontal clouds simulated by the ECMWF model. Mon. Wea.

Rev., 127, 2514–2531, doi:10.1175/1520-0493(1999)127,2514:

VASOFC.2.0.CO;2.

——, Y. Zhang, M. D. Zelinka, R. Pincus, J. Boyle, and P. J.

Gleckler, 2013: Are climate model simulations of clouds im-

proving?An evaluation using the ISCCP simulator. J. Geophys.

Res. Atmos., 118, 1329–1342, doi:10.1002/jgrd.50141.

Li, J.-L. F., D. E.Waliser, G. Stephens, S. Lee, T. L’Ecuyer, S. Kato,

N. Loeb, and H.-Y. Ma, 2013: Characterizing and un-

derstanding radiation budget biases in CMIP3/CMIP5 GCMs,

contemporary GCM, and reanalysis. J. Geophys. Res. Atmos.,

118, 8166–8184, doi:10.1002/jgrd.50378.

Mace, G. G., and F. J. Wrenn, 2013: Evaluation of the hydrometeor

layers in theeast andwestPacificwithin ISCCPcloud-toppressure–

optical depth bins using merged CloudSat and CALIPSO data.

J. Climate, 26, 9429–9444, doi:10.1175/JCLI-D-12-00207.1.

Mason, S., C. Jakob, A. Protat, and J. Delanoë, 2014: Character-izing observed midtopped cloud regimes associated with

Southern Ocean shortwave radiation biases. J. Climate, 27,

6189–6203, doi:10.1175/JCLI-D-14-00139.1.

Oreopoulos, L., and W. B. Rossow, 2011: The cloud radiative effects

of International Satellite Cloud Climatology Project weather

states. J.Geophys. Res., 116,D12202, doi:10.1029/2010JD015472.

——, N. Cho, D. Lee, S. Kato, and G. J. Huffman, 2014: An exami-

nation of the nature of globalMODIS cloud regimes. J.Geophys.

Res. Atmos., 119, 8362–8383, doi:10.1002/2013JD021409.

Papritz, L., S. Pfahl, I. Rudeva, I. Simmonds, H. Sodemann, and

H. Wernli, 2014: The role of extratropical cyclones and fronts

for Southern Ocean freshwater fluxes. J. Climate, 27, 6205–6224, doi:10.1175/JCLI-D-13-00409.1.

Rossow, W., and R. Schiffer, 1999: Advances in understanding

clouds from ISCCP. Bull. Amer. Meteor. Soc., 80, 2261–2288,doi:10.1175/1520-0477(1999)080,2261:AIUCFI.2.0.CO;2.

——, G. Tselioudis, A. Polak, and C. Jakob, 2005: Tropical climate

described as a distribution of weather states indicated by dis-

tinct mesoscale cloud property mixtures. Geophys. Res. Lett.,

32, L21812, doi:10.1029/2005GL024584.

Stephens, G. L., 2005: Cloud feedbacks in the climate system: A critical

review. J. Climate, 18, 237–273, doi:10.1175/JCLI-3243.1.

Tan, J., C. Jakob, and T. P. Lane, 2013: On the identification of

the large-scale properties of tropical convection using

cloud regimes. J. Climate, 26, 6618–6632, doi:10.1175/

JCLI-D-12-00624.1.

1 AUGUST 2015 MASON ET AL . 6017

Trenberth, K., and J. T. Fasullo, 2010: Simulation of present-day

and twenty-first-century energy budgets of the southern

oceans. J. Climate, 23, 440–454, doi:10.1175/2009JCLI3152.1.

Tselioudis, G., W. Rossow, Y. Zhang, and D. Konsta, 2013: Global

weather states and their properties from passive and active

satellite cloud retrievals. J. Climate, 26, 7734–7746, doi:10.1175/

JCLI-D-13-00024.1.

Tsushima, Y., M. A. Ringer, M. J. Webb, and K. D.Williams, 2013:

Quantitative evaluation of the seasonal variations in climate

model cloud regimes. Climate Dyn., 41, 2679–2696,

doi:10.1007/s00382-012-1609-4.

Walters, D. N., and Coauthors, 2011: The Met Office Unified

Model Global Atmosphere 3.0/3.1 and JULES Global Land

3.0/3.1 configurations. Geosci. Model Dev. Discuss., 4, 1213–

1271, doi:10.5194/gmdd-4-1213-2011.

Webb, M. J., C. Senior, S. Bony, and J.-J. Morcrette, 2001: Com-

bining ERBE and ISCCP data to assess clouds in the Hadley

Centre, ECMWF and LMD atmospheric climate models.

Climate Dyn., 17, 905–922, doi:10.1007/s003820100157.

Wielicki, B. A., B. R. Barkstrom, and E. F. Harrison, 1996: Clouds and

the Earth’s Radiant Energy System (CERES): An Earth Ob-

servingSystemExperiment.Bull.Amer.Meteor. Soc., 77, 853–868,

doi:10.1175/1520-0477(1996)077,0853:CATERE.2.0.CO;2.

Williams, K. D., and G. Tselioudis, 2007: GCM intercomparison of

global cloud regimes: Present-day evaluation and climate

change response. Climate Dyn., 29, 231–250, doi:10.1007/

s00382-007-0232-2.

——, and M. J. Webb, 2009: A quantitative performance assess-

ment of cloud regimes in climate models. Climate Dyn., 33,141–157, doi:10.1007/s00382-008-0443-1.

Zhang, M. H., and Coauthors, 2005: Comparing clouds and their

seasonal variations in 10 atmospheric general circulation

models with satellite measurements. J. Geophys. Res., 110,

D15S02, doi:10.1029/2004JD005021.

6018 JOURNAL OF CL IMATE VOLUME 28