RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Global Carbon Project Workshop 7-10 Dec 2015‘Tools and Indicators for Measuring Urban Resilience’
Monitoring and evaluating progress towards becoming a more adaptive and resilient region: lessons from Melbourne
Dr Susie Moloney, Senior Research Fellow, Climate Change and Resilience ProgramCentre for Urban Research, RMIT University, Melbourne.
Outline
1. Definitions and Context: Adaptive Capacity Building and Urban Resilience
2. Research: low carbon transitioning? (governance, social change and urban planning); adaptation planning, capacity building across local governments
3. Project: How well are we adapting? How can we monitor and evaluate progress across the western region of Melbourne
4. Designing a Framework for Monitoring and Evaluating Climate Change Adaptation – Approaches and Indicators (Lessons from the literature)
5. Piloting: Themes and Indicators
6. Lessons/questions for measuring and assessing resilience
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
1. Definitions and Context: Adaptive Capacity Building and Urban Resilience Some useful definitions (O’Connell et al 2015:6)
• Adaptability (adaptive capacity): The capacity of actors in a system to respond to shocks and to trends and (if known) the proximity of the state of the system to a threshold and so to influence resilience
• Adaptation: a process of responsive change that improves the ability of a system to achieve desired goals, including by reducing vulnerability to disturbance or threats (including climate change)
• Adaptive governance: institutional and political frameworks designed to adapt to changing relationships in society and between society and eco-systems
• General resilience: capacity of all parts of the system to cope with all kinds of shocks and disturbances, and so be able to avoid crossing thresholds, known or unknown, to alternate regimes or systems. It is sometimes referred to as ‘coping capacity’ and in this report is used synonymously with adaptive capacity.
• Transformation: is physical or qualitative changes in form, structure, function or meaning.
• Transition: the course of the trajectory from one regime of a system to another or from one kind of system to another (ie. transformational change).
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
What does it mean to build urban climate change resilience
What is urban climate change resilience? (ACCCRN and Rockerfeller2013:
• It is the capacity of cities (individuals, communities, institutions, businesses and systems) to survive, adapt, and thrive in the face of climate related stresses and shocks and even transform when conditions require it
What is required to build urban climate change resilience?
• effective leadership,
• strong partnerships,
• inclusive processes and
• an ability to translate technical data and information into practical action
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
ISET Climate Resilience Framework and characteristics of resilience (Tyler and Moench 2012)
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Systems:• flexibility and diversity; • redundancy and
modularity;• safe failure
Institutions:• rights and entitlements, • decision-making
structures, • access to information• processes that support
learning and change
Agents: • ability to learn, • resourcefulness• responsiveness
2. Research: adaptive capacity building across local governments in Melbourne in the context of climate change
• The necessity to build adaptive capacity at the local scale has led to alternative forms of networked and informal governance
• ‘Climate Change Alliances’ (4 metro/peri-urban, 6 rural)
• Important role in building adaptive capacity of councils
and communities
• How? (Moloney and Funfgeld 2015):– Facilitating institutional interactions horizonally across
councils and vertically between federal, state and local levels
– Regional scale risk assessments, mitigation and adaptation strategies
– Creating opportunities for reflexive learning between members and wider stakeholders, across scales through knowledge exchange, learning and advocacy
– Characteristically adaptive and integrative forms of governance
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
3. Project Context “How Well Are We Adapting?” (2014-2017): Developing an Monitoring and Evaluation Framework for Climate Change Adaptation (Western Alliance for Greenhouse Action and Victorian Gov’t)
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Greater Melbourne: 7000+ km24.4mill (8 mill by 2050)
Western Region (8 LGAs): 4700 km2, 838,500 pop’n
• WAGA Climate Change Adaptation Strategy and Action Plan (2012):– Mainstream adaptation across councils– Embed adaptation planning processes– Review progress of adaptation work carried out by WAGA councils and
regionally
• 88 risks identified (17 severe) : covering council service areas– assets and infrastructure; transport; open space and recreation; natural
environment; emergency management; health and community; planning and building; business continuity
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
A Framework to Monitor Evaluate and Report (ME&R) on Climate Change Adaptation
Purpose?
This framework will:
• Help us track how councils are managing or responding to climate change
• Monitor the impacts of climate on council operations
• Communicate with the community about climate vulnerability and council action.
• Assess the effectiveness of actions and inform future actions* (i.e. if there are maladaptive actions occurring) (*Eventual goal).
It is designed to:
• Evaluate and report on actions that help manage climate risks.
• Focus on learning rather than measuring success or failure.
• Focus on areas of commonality across all WAGA councils.
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Reporting to Who and Why?Two key audiences, council staff and executives and the local community.
Council objectives
• To inform future decision making and adaptation activities
• To improve learning and build capacity
• For comparative purposes
Community objectives
• Improve awareness:– What is happening– What is council’s responsibility
• Assure action is happening
• Climate change education
We want to monitor:
• The delivery of actions with concrete outcomes
• Building the capacity of the council and community to respond to and manage climate impacts.
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
4. Designing a Framework - Approaches and Indicators:Some key lessons from the literature
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
https://gallery.mailchimp.com/b38874b25e686137780eb836e/files/M_E_Lit_Review.pdf
AdaptME toolkit (UKCIP 2011)
Are we doing things right?Are we doing the right things?
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Why are we doing this?Many possible reasons…..• To evaluate effectiveness
• To assess efficiency
• To understand equity
• To provide accountability
• To assess outcomes
• To improve learning
• To improve future activities or interventions
• To compare with other similar activities or interventions.
(AdaptME Toolkit, 2011)
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Understand the different types of evaluation/assessment
• Formative: informs decisions to improve, assess program development
• Summative: assesses overall effectiveness at the end
Source: Learning to ADAPT (Villaneuva 2011, p. 20)
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Approach: Input-output-outcome
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
1. Inputs 2. Activities 3. Outputs 4. Outcomes 5. Impacts What resources go into a program
What activities the program undertakes
What is produced through those activities
The short and medium term effects of an interventions outputs; change in development conditions.
Actual or intended changes that result from the program over the longer term.
Approach: Process based evaluation
• Aims to determine how a programme is being implemented.
• Seeks to define the key stages in a process that would lead to the ‘best choice of end point’ without specifying that point at the outset.
• Aims to measure and reflect on building adaptive capacity but does not define what type of outcomes will emerge.
• Allows for flexibility according to changing circumstances and encourages a learning-by-doing approach. (E.g. build climate change adaptation response into policies, establish an adaptation committee etc.)
• Process evaluation uses quantitative and qualitative data collection and analysis methods.
• The limitations of the process evaluation is that it does not evaluate outcomes.
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Process-based indicators Outcome-based indicators• Allow stakeholders/sectoral experts
to choose the most appropriate adaptation action to meet an outcome
• Flexible approach – can adjust to new information as it becomes available
• Most government policy objectives/targets are outcome-based• May be possible to link adaptation objectives with objectives in other policy areas• Likely to be sector-specific
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Process-based indicators Outcome-based indicators• Defining a process does not guarantee successful adaptation• A different approach from most other government targets, so often unfamiliar to practitioners• May make it difficult to integrate adaptation objectives with objectives in other policy areas• Not necessarily sector-specific
• Defining an outcome does not guarantee successful adaptation• Risk of being overly prescriptive of adaptation options (specifying suboptimal options)• May be inflexible and make it difficult to introduce new information (though great scope for flexibility in implementing specific actions to achieve outcome)
Disadvantages
Advantages
Developing adaptation related indicators
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Source: Harley et al. (2009)
M & E Framework aims to measure performance
• Against objectives– Comparing outputs and outcomes against what a program intended
• Against ‘good adaptation’– Defined by guiding principles such as sustainability, collaborative, open,
effective, efficient, equitable– ADAPT Principles: Adaptive learning; Dynamic monitoring, Active,
Participatory, Thorough (Villeneuva 2011)
• Against a baseline– Conditions change, before and after interventions not always adequate – Shifting baselines….
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Issues to consider in dealing with shifting or dynamic baselines?• Moving targets and scenarios
• Adaptive capacity (AC) and vulnerability (V) are dynamic variables
• 3 issues to consider (Villaneuva 2012:18)1. Distinguish between generic and specific indicators (ie. generic
indicators capture underlying causes of vulnerability while the latter target the specific measures undertaken to reduce vulnerability)
2. Evaluation processes snapshot V and AC at the end of a program but this needs to be followed up by constant monitoring over long-term
3. M & E needs to capture the existence of V and AC and those processes that may effect the distribution of V or how capacity leads to action
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Baselines for 5 areas (World Bank 2010)• Climate data: climate parameters such as temperature, rainfall, humidity, and local
habitat parameters such as soil conditions and water salinity
• Socio-economic data: Indicators of economic and social well-being in a community, including income, food security, health, and the impact of climate change on these factors
• Data on institutional and policy processes: Capacity and existence of appropriate institutions (official or unofficial) and the legal framework (e.g. existence and implementation of climate change policies.)
• Ecosystem services: The extent to which ecosystem services are affected by the impacts of climate change
• Coping Strategies: What strategies the local population has so far used to cope with climate change variability.
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
WAGA(Draft)FrameworkNov (2015)
5. Piloting the Framework: Themes and Indicators (examples)
Regional Baselines: Indicators
1. Regional Climate variables (temperature, rainfall, water supply etc)
2. Regional Vulnerability and Resilience:– Regional heat vulnerability/canopy cover mapping– Demographic vulnerability to heatwave impact by age– Number of distributed heat related deaths and illness– Demographic vulnerability to CC (socio-economic disadvantage; recent
migrant distribution)– Understanding community cohesiveness and support structures (surveys)
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Community Wellbeing and Emergency Management: Indicators
1. Service Vulnerability and Resilience:– Knowledge of vulnerable people exposed to extreme weather events– Continuity of critical home and community care services– Residents seeking refuge in official council run emergency relief centres
during severe weather events
2. Institutional capacity:– Staff capacity to address climate change in decision making (interviews
and surveys)– Emergency management framework recognises and responds to
changing risk levels with climate change
3. Resourcing and Budgeting:– Tracking long term trends in investment towards preparing
for/responding to extreme weather events
4. Community Participation:– Commuity preparedness to respond to extreme weather events– Community lifestyle and household resilience to a changing climate
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Open Space and Water Security: Indicators
1. Service Vulnerability/Resilience: – Type and Volume of water supply used; – variation of gap in water supply compared to demand over time; – water efficiency
2. Institutional Capacity:
3. Resourcing: – changes in long term trend in the cost and frequency of extreme weather
event cleanups– Understanding the impact of climate change on council’s operational
budgets
4. Participation and Awareness:– Understanding community satisfaction with open space over time
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Data Collection – Indicator templatePiloting with four councils – training and implementation early 2016
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Finally - Challenges and limitations in M & E
• Identifying success (defining)
• Timeframes go beyond the duration of an intervention
• Measuring the impact of an intervention
• Lack of concrete definitions
• Measuring against a ‘shifting baseline’
• Avoiding maladaptation
• Dealing with uncertainty
• Diversity in adaptation scales and sectors
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
How to address these challenges and what to consider
• Engage broad range of stakeholders to help define what success might be, who benefits, who does not; flexibility of an intervention might be a measure of success
• Systematic M & E, iterative processes, tracking progress helps deal with long time lines
• Contribution rather than attribution of interventions
• Share learning – across organisations and beyond
• Learnings from M & E timed to inform key decisions
• Where, when and to whom do the key learning messages need to be articulated?
6. Questions for consideration: • Clarify purpose? What are we measuring and assessing resilience for
(resilience of what for what and by whom and for whom?)
• Develop a tool as a learning framework? - emphasise adaptive learning to shape and inform the course of adaptation over time, review adaptive management and DRR interventions
• Consider these principles (or something similar) to guide the choice and type of indicators?
RMIT University©2015 GCP Workshop Measuring Urban Resilience Tokyo 2015
Principles Indicators
Adaptive Reflect possibility of changing conditionsDynamic Capture the way processes are changingActive Capture actions rather that statesParticipatory Are developed by and with those affected by
interventionsThorough Include maladaptation indications and capture
how, or not, the intervention addresses the underlying causes of vulnerability