TECHNICAL Research Foundation Collaborative · PDF fileWater Research Foundation Collaborative Utility ... will evaluate the process and future opportunities ... opportunity to discuss

T E C H N I C A L M E M O R A N D U M

1

Water Research Foundation Collaborative Utility Benchmarking in North America Project

Benchmarking Workshop Results Report PREPARED FOR: Water Research Foundation

PREPARED BY: CH2M

DATE: February 4, 2016

Overview This document summarizes the Benchmarking Workshop conducted as part of Water Research Foundation’s (WRF) Tailored Collaboration Project “Collaborative (formally Enhancing) Utility Benchmarking in North America”. This project aims to further develop, refine, and implement a unified industry benchmarking framework, and collection of industry tools and process for North America that is connected to WSAA’s 2016 Asset Management Customer Value (AMCV) project. This WRF project is intended to enhance the value offered by WSAA’s AMCV, AWWA Utility Benchmarking Survey, Effective Utility Management (EUM), and other available tools, such as the ISO 55000 framework. This research will evaluate the process and future opportunities for benchmarking within the US water sector. During the course of the executing of the Collaborative Utility Benchmarking in North America project, an evaluation, business case, and recommendation will be made for potential tool integration and enhancements, association and utility collaboration, and other benchmarking opportunities.

The specific objectives for the Benchmarking Workshop were the following:

present the benchmarking tool and process envisioned for implementation in North America, and facilitate input, discussion and agreement on the final tool and approach;

discuss and solicit input on the questions that will be important to ask the participating utilities at the end of the benchmarking exercise to determine the costs, benefits, and value to participating utilities; and

solicit understanding, input and discussion on the business case development.

The Benchmarking Workshop agenda is provided in Appendix A. The Benchmarking Workshop Day 1 Presentation is provided in Appendix B. The Benchmarking Workshop Day 2 Presentation is provided in Appendix C.

Benchmarking Workshop Attendees Role Name

WRF Project Manager Linda Reekie

PAC Heather Pennington

OVERVIEW

2

PAC Kurt Vause

PAC Kevin Campanella

Steering Group Greg Ryan

Steering Group Mike Sweeney

Steering Group Leisa Thompson

Steering Group Frank Roth

Steering Group Sarah Neiderer

Steering Group Jeff Leighton

Steering Group Matt Ries

Steering Group Ken Mercer

Steering Group Stephanie Passarelli

Participant Dave Plank

Participant Zsolt Silberer

Consultant Scott Haskins

Consultant Priscilla Bloomfield

Consultant Terry Brueck

Agenda Discussion, Outcomes, and Action Items – Day 1

AGENDA TOPIC DISCUSSION/OUTCOME ACTION ITEM

INTRODUCTIONS/ OBJECTIVES/ EXPECTATIONS

There was general discussion that this project is an opportunity for the associations with different benchmarking/ self‐improvement tools/frameworks to survey utility participants and evaluate the outcome of the project to determine what opportunities could exist in the future. This workshop was a way to get the Steering Team, PAC, and research team on the same page to better understand the tools and how they fit together. It was also an opportunity to discuss what they could look like going forward and identify the key success criteria for the project. Participant expectations for the workshop were discussed.

None

CONTEXT – WRF TC PROJECT

There was discussion about the scope, schedule, roles and responsibilities (Steering Group, PAC, etc.), and engagement/communication for the WRF TC project. Components of the deliverables were confirmed: Benchmarking Workshop Results Report, Benchmarking Evaluation Report (includes business case evaluation), North American Leading Practice Report, North American Industry Report. The WRF TC project builds upon previous work done by WSAA, AWWA, and others. Various associations will be included and informed as the project progresses. It was confirmed that the project will make recommendations for the future and not to produce a final tool.

Contact associations

Provide workshop presentations to participants

OVERVIEW

3


AMCV PROJECT AMCV is WSAA’s rebranded Aquamark tool, focused on customer value which addresses the requirements of ISO 55000, with a focus on utility management. There is a cost reduction for this round in 2016 of approximately 40%, 25% of the tool has been updated with the addition of new measures in relation to leadership and culture, ensuring a customer focus in all relevant questions, addition of 40 measures to ensure alignment with ISO 55000, rationalization of Function 7 ‐ support systems, reducing it from 18 to 6 Functions along with consolidation of the measures in Function 7 from 18 to 9, peer review of measure weightings to ensure they reflect contemporary practice, complete overhaul of the software platform, ISO aligned, with integration of AWWA and EUM elements for the North American version. There is a major focus on leading practices and knowledge sharing. The group discussed the AMCV project’s deliverables.

None

AWWA TEC AWWA’s Benchmarking Committee and Asset Management Committee submitted a TEC proposal. The budget may be increased to $30,000, and the scope elements are to be determined. The project will be released as an RFP. The focus of the project is around developing a BCE for AWWA about making their benchmarking program stronger and aligns with the WRF TC project’s evaluation and BCE component.

None

ASSOCIATED INDUSTRY FRAMEWORKS AND FEATURES ‐ AMCV

Information on the 2016 AMCV project can be accessed via http://amcv.wsaa.asn.au/AMCV. Improvements include simpler data entry, more interactive engagement online with discussion boards, dialogues, etc., 2 assessment options – self or facilitated, alignment with ISO, possibly with EUM, AWWA, etc. There are various potential options for the future. A live online demonstration was provided. There was dialogue about the tool’s functionality and various project aspects.

Provide WSAA ISO decoder to participants

Confirm that the 2012 measure comments (Aquamark) can be imported into the 2016 version (AMCV)

Include ISO and EUM mapping in the utility and industry reports if feasible

MAPPING There was a discussion of mapping ISO, EUM, AWWA metrics, and others to the AMCV framework and adding new measures to fill gaps in the AMCV tool. Metrics will be shown at the Subprocess level in a manner similar to a measure. A definition and calculation will be provided. Users will enter a current and target value. This data would be provided by the utility separately from the AWWA 2016 Utility Benchmarking Survey, which will continue as a separate exercise, as in previous years. There was a discussion

Complete mapping EUM to AMCV

Map AWWA metrics to AMCV if cost

OVERVIEW

4


about the potential mapping of AWWA Utility Management Standards to AMCV. AWWA Utility Management Standards could be a resource for a utility to help close their gaps.

CONSENSUS: Pursue mapping other frameworks to AMCV for possible inclusion. This was with the caveat that such mapping must add value to the industry and be cost effective.

effective to do so (see discussion below)

Secure copy of AWWA Utility Management Standards

Investigate level of effort to map/include AWWA Utility Management Standards in AMCV

Investigate level of effort to map EUM to ISO directly

ASSOCIATED INDUSTRY FRAMEWORKS AND FEATURES ‐ EUM

A steering group is currently working to update EUM based on changes in the industry in the last 10 years. The 10 attributes and 5 keys to success are expected to stay the same with some changes to their descriptions. The changes are centered around automated and smart systems and data integration, climate variability and extremes, customer expectations and public awareness, employee recruitment and retention, resource recovery, regulatory requirements and operating conditions, and stormwater and watershed management/one water. The updates are expected to be finalized by spring 2016.

Continue to stay abreast of EUM updates

ASSOCIATED INDUSTRY FRAMEWORKS AND FEATURES – AWWA SURVEY

AWWA’s Utility Benchmarking Survey is now conducted annually. Improvements for 2016 include ease of use (pre‐population of data, definitions, etc.), data quality, confidence levels (guess, audited number, etc.), new indicators and adjustments to existing metrics, and feedback from users and utilities. It has also been mapped to EUM. Utilities answer questions in an Excel file and get the metrics as the output, which AWWA compiles and reports, both specific to the utility and generalized for the industry.

CONSENSUS: Pursue incorporating AWWA metrics into AMCV.

Pursue coordination between AWWA and WSAA on inclusion of AWWA metrics in AMCV

MCES CASE STUDY MCES piloted the proposed 2016 AMCV project approach and tool integration and presented the results. The scope and process was discussed, along with innovations. Key themes were breaking down silos, engagement, and translating a vision and benchmarking results into prioritized actions. There were many questions and answers. There were suggestions as to how to further improve the process, especially for repeat participants.

None

OVERVIEW

5


ASSOCIATED INDUSTRY FRAMEWORKS AND FEATURES – NACWA

NACWA conducts a comprehensive annual survey that is mostly financial and gathers data (versus metrics). There is some overlap with the AWWA Utility Benchmarking Survey. In discussions, NACWA has been supportive of this WRF TC project. It is anticipated that NACWA would like to contribute to the project with measures around the utility of the future for potential addition to AMCV.

CONSENSUS: Collaborate with NACWA if possible.

Follow up with NACWA on involvement/ contribution

ASSOCIATED INDUSTRY FRAMEWORKS AND FEATURES – MATT RIES PHD

Matt Ries is writing his PhD thesis on sustainability indicators for

urban water utilities with a focus on defining the drivers and

attributes. The US does not have much data so Matt interviewed

12 utility leaders and conducted an online survey with water

professionals. The results indicated 8 top practices (Education and

communication, Community ROI, Bond rating/financial

management, Resource recovery, Green infrastructure, Asset

management, etc.) and 6 attributes (leadership, political will,

training, board support, etc.), as well as drivers (public demand,

political will, tools, vision, regulations) and barriers (resources, lack

of incentive, lack of definition). Based on this work, Matt

developed a set of sustainability measures. Measures that are not

currently addressed may be incorporated into AMCV.

CONSENSUS: Pursue incorporating Matt’s PhD work into AMCV.

Pursue potential integration of sustainability measures into AMCV

UTILITY PROFILE AND UTILITY DRIVERS

AMCV – The utility profile contains basic utility information such as size and sector. The drivers include Regulatory, Financial, Sustainability, Customer expectations and demand, Knowledge management (staff), Asset lifecycle management, Industry movements, Completion, and Technology advancements. The top 5 are selected by each utility.

AWWA – Drivers are a new part of the Utility Benchmarking Survey and are being developed. Draft list includes Regulatory, Water supply, Rates, Aging infrastructure, Expected efficiency, Growth, Security, Strategic planning, Economic climate, Continual improvement, and Workforce/succession planning/training. Utilities rate each driver as high, medium, and low.

In the past, knowing the drivers has been helpful information to compare and contrast utilities and in formulating improvement initiatives. This also aligns with Importance and Urgency.

CONSENSUS: There should be alignment between AMCV and the Utility Benchmarking Survey drivers if possible.

WSAA and AWWA to work to get alignment between drivers and add an open ended category

METRICS There was a discussion of the possible inclusion of metrics into AMCV. The group decided to include AWWA metrics as well as other metrics (to be identified). Jeff Leighton offered to support identifying additional metrics. [Subsequent to the workshop, Jeff provided some sources of potential additional metrics.] The 2016 AMCV project could provide offline support for developing customized metrics for a utility, which would not be included as a standard approach in AMCV.

Identify additional metrics for possible inclusion in AMCV

OVERVIEW

6


CONSENSUS: AWWA metrics and potentially additional metrics should be included in AMCV.

IMPORTANCE AND URGENCY

There was a discussion around the inclusion of an Importance and Urgency rating system in AMCV to help prioritize gaps, which would be applied at the Sub‐process level. The group discussed the definitions and differences between Importance and Urgency. In general, the group felt that Importance and Urgency were an important enhancement for AMCV. These elements will be incorporated into the 2016 AMCV deliverables.

CONSENSUS: Include Importance and Urgency in AMCV.

Include Importance and Urgency in AMCV

EVALUATION There was a discussion around the evaluation component of the WRF TC project that includes a survey for the 2016 AMCV participants and a business case evaluation (BCE). A list of questions and considerations for the survey was developed. These include how the project was done, how much time did the project take, what is the value proposition and benefits, engagement, and survey timing. This component of the project will be discussed in more detail on Day 2.

None

ADDITIONAL ITEMS There was a discussion around the clarity of the roles of PAC, the Steering Group, and the associations.

The 2106 AMCV project will be called the 2016 North American AMCV.

Provide 2016 AMCV marketing materials

PARKING LOT Define the criteria for success for the project

Discuss what happens after the project ends in a year

Discuss WRF TC project name ideas

Address items on Day 2 if necessary

OVERVIEW

7

Agenda Discussion, Outcomes, and Action Items – Day 2


DAY 1 UPDATES WSAA and AWWA came to agreement on the list of drivers to be used in the 2016 AMCV and the Utility Benchmarking Survey. The same definitions will be used as well.

List is:

Regulation and standards

Financial

Sustainability

Customer expectations

Resources and service demand

Workforce evolution

Asset lifecycle management

Security

Efficiency and continual improvement

Technology

Other (fill in)

None

EVALUTION The participants divided into 2 facilitated break out groups to further develop the evaluation questions for the survey with equal representation from utilities, associations, and PAC members. One group focused on the project experience, and the other group focused on future and tool improvements. The groups met for over an hour and reported results back to the entire group. Overall there was general alignment on the survey questions between the two groups.

The break out groups work products are provided in Appendix D.

Further develop the survey questions

BUSINESS CASE EVALUATION

The group discussed important features of the BCE, and the various stakeholders described their interests as discussed below.

Develop the BCE

WSAA WSAA is interested in whether the process has value to the users, should we continue on this path, is it the right thing to develop a consolidated tool, if so what are the enhancements, what are the options for the future, are there different options, one tool or many, what is the market for the tool(s), can we get an idea of the frequency and approach, what is appropriate pricing, risks and benefits, SWOT by associations, and what are the association roles (part of this WRF TC project or separate).

None

AWWA AWWA is interested in whether to integrate or coordinate their survey going forward, is the survey meeting their needs, best way to deliver the survey, targeting leading edge utilities versus the whole water industry, reaching the utilities that

None

OVERVIEW

8


they want to reach in the ways they want, linking the results from this process across the entire spectrum of utilities, how to build on existing content and partner with other organizations, best way forward to meet industry needs, and how to translate the aggregate results into something meaningful to apply to affect regulatory changes or legislative interests. The Water Utility Council is interested in knowing whether there is opportunity for consolidation and what is needed for the entire sector, how to translate aggregate results into something meaningful (regulatory changes or legislative initiative), how to take the industry report and manifest into meaningful actions, and exploring opportunities for consolidation/integration (metrics, TEC surveys (2 per year), partnered surveys (1‐2 per year), non‐partnered surveys, rates survey, state of the water industry, compensation survey).

WEF WEF has no benchmarking system but they have recognition programs (utility of the future, stormwater, etc.) where benchmarking practices can be helpful. Their interest is in sharing information, best practices, and innovative approaches.

None

PAC The PAC is focused on utility improvement. They would like to have broad participation by utilities in the 2016 AMCV and beyond. They want the BCE to provide the value proposition for the utilities to participate in benchmarking.

Provide the value proposition for the utilities to participate in benchmarking in the report

UTILITIES The utilities are focused on continuous improvement, gap analysis as what to do differently to expand the program, monitoring, exploring the capacity for benchmarking at a greater frequency, demonstrating the value through the initial study, addressing survey fatigue, considering the do nothing option, quantifying benefits, opportunity cost and avoided cost, incorporating TBLI (Infrastructure), exploring ways to increase participation, informing the sector affects the benefits, cost and broad appeal, using case studies in the BCE, why, why now, value, return on investment, what is the internal cost, what can it augment, time commitment for internal resources and what falls off the plate, what is needed to accomplish, how does it support the organizations that also represents the utility, AMCV and other tools, description of tools and where they fit, values they add, costs (internal and external), ROI, etc., what are the boundaries around the tools, competition for use of tools, could the business case compare and contrast in terms of the cost and resource, what is the business case moving forward, do we use all or some, combined or separate.

A potential future project could be mapping of each of the currently available tools, the gaps they fill and when they

None

OVERVIEW

9


should be applied and used, and identifying different programs and the value they add and the cost. This could include how these tools might be better rationalized, variation of utilities, regions, scale and scope, and compare and contrast all the tools and resources required and costs (internal and external).

SUCCESS CRITERIA The group defined the success criteria for the WRF TC project. These are listed below.

Comprehensive BCE with elements from the discussion with recommendation for a path forward. Would be good if utilities understood the options – what is the depth and breadth of the approach.

High response rate to the evaluation and clear response to the evaluation. I.e. we understand what people want – needs to be comprehensive. Are we going to set goals for success rate for feedback on the survey? Need to encourage people to complete the survey, but not set a metric.

Ability to have the AWWA Utility Management Standards mapped and implemented – we have committed to look at level of effort only at this point.

Evaluation report will culminate effort and suggest next steps from the perspective of what was learned. The BCE could take that into account. I.e. the evaluation will inform the BCE.

Catalyst for change within the utility with utility report and possibly the industry as a whole with industry report. Ability to pull the data together and determine what is happening across the sector could be a catalyst for change if there are common areas.

Lessons learned to build on for next steps. The number of survey questions are appropriate for this first stage to get as much feedback as possible. Can we do this in a way that doesn’t result in survey fatigue? Need several options. Success = being able to effectively gather data from the utilities without causing survey fatigue.

Collect feedback in a variety of ways. Expect survey results from at least the project coordinator and encourage survey feedback from others as well. If survey has the right questions, the feedback at the end could be really helpful in the evaluation.

Include Canadians as well.

Workshop report that documents the outcomes and discussions.

None

NEXT STEPS The decision was made to hold the next meeting in conjunction with the 2016 Utility Management Conference in San Diego. The meeting is tentatively planned for Tuesday, February 23 from 3 pm to 5 pm EST. Call in information will be provided to those who are not attending the conference. Location to be determined.

Set up the meeting for February 23

OVERVIEW

10


Another group meeting may be held in conjunction with AWWA’s ACE in June in Chicago.

PROJECT NAME The group decided to name the WRF TC project “Collaborative Utility Benchmarking in North America”. Formerly the title was “Enhancing Utility Benchmarking in North America”.

None

WRAP UP The Benchmarking Workshop Results Report will be provided by December 31, 2015.

The group participated in a discussion of items to consider in moving forward with the WRF TC project. Items included communication, schedule and milestones, incorporating feedback, and focus on value propositions.

Develop the Benchmarking Workshop Results Report


11

Appendix A: Benchmarking Workshop Agenda Appendix A includes the Benchmarking Workshop agenda for both days.


12

Day 1 Agenda Time Topic Presenter

6:00 – 8:00 am Breakfast provided at hotel

8:10 am Shuttle from hotel to AWWA, meet in hotel lobby

8:30 am Introductions/objectives/expectations

Objectives:

present the benchmarking tool and process envisioned for implementation in North America, and facilitate input, discussion and agreement on the final tool and approach

discuss and solicit input on the questions that will be important to ask the participating utilities at the end of the benchmarking exercise to determine the costs, benefits, and value to participating utilities

solicit understanding, input and discussion on the business case development

Linda Reekie

8:45 am Context – overall scope, objectives, schedule and deliverables for the project:

WaterRF Tailored Collaboration aspects

Benchmarking process and approach

AWWA TEC (If funded)

Scott Haskins Greg Ryan Ken Mercer

9:30 am Associated industry frameworks and features

Asset Management Customer Value (AMCV) ‐ highlighting changes from 2012, detailed explanation of tool

ISO 55000

Greg Ryan Greg Ryan

10:15 am Break

10:30 am Associated industry frameworks and features (continued)

EUM

AWWA Survey

NACWA Survey

PhD Dissertation

Matt Ries Terry Brueck Stephanie Passarelli Scott Haskins Matt Ries

11:15 am Presentation and discussion of MCES case study Leisa Thompson

12:00 pm Lunch provided by AWWA

Afternoon sessions on details of the process. Throughout the process we are seeking to achieve:

1) Understanding of the benchmarking tool and the benchmarking process as presented below

2) Confirm you are comfortable with the approach

3) Fatal flaws

OVERVIEW

13

1:00 pm Presentation of utility profile and utility drivers methodology alignment and how this will be used as part of the project

Greg Ryan/ Terry Brueck

1:30 pm Presentation and discussion of the approach used to map AMCV and EUM, metrics, ISO 55001

Validation of concept

Greg Ryan

2:15 pm Metrics – presentation of the approach and process


Scott Haskins

3:15 pm Break

3:30 pm Importance and urgency– presentation of the approach and process


Scott Haskins

4:00 pm Evaluation – major components for Experience and Future Opportunities

Validation of Process and Tools

Scott Haskins

4:30 pm Closeout

How did the day go?

Day 2 plans

Linda Reekie

5:00 pm Finish Day 1

5:15 pm Shuttle from AWWA to hotel, meet in AWWA lobby

6:30 pm Dinner – Bonefish Grille provided by WRF

OVERVIEW

14

Day 2 Agenda

Time Topic Presenter

6:00 – 8:00 am Breakfast provided at hotel

8:10 am Shuttle from hotel to AWWA, meet in hotel lobby

8:30 am Evaluation – break out exercise incorporating the outcomes from Day 1.

Form two break out groups, to describe how the final survey will be designed to ensure a robust evaluation of the project outcomes in terms of:

Participants’ experience of the benchmarking process and tools

Future opportunities arising from the process

All

Facilitated by Scott Haskins, Priscilla Bloomfield

9:30 am Group 1 report out and feedback All

10:00 am Group 2 report out and feedback All

10:30 am Break

10:45 am Business Case Evaluation (BCE) – what are the key elements of the future business case from associations’ and utilities’ perspective? Presentations to frame the discussion

AWWA

WSAA

Utilities

Discussion and input into the final structure of the BCE

Ken Mercer

Greg Ry an

Utilities

Facilitated by Scott Haskins

11:45 am Feedback/next steps Scott Haskins/Linda Reekie

12:00 pm Lunch provided by AWWA

12:30 pm Close

1:00 pm Shuttle to airport, meet in AWWA lobby


15

Appendix B: Benchmarking Workshop Day 1 Presentation Appendix B includes the Benchmarking Workshop Day 1 Presentation.

© 2015 Water Research Foundation. ALL RIGHTS RESERVED.© 2015 Water Research Foundation. ALL RIGHTS RESERVED. No part of this presentation may be copied, reproduced, or otherwise utilized without permission.

Tailored Collaboration Project – Utility Excellence

• December 7, 2016

© 2015 Water Research Foundation. ALL RIGHTS RESERVED.© 2015 Water Research Foundation. ALL RIGHTS RESERVED.

Introductions, objectives, expectations


Overall context

Scott Haskins

© 2015 Water Research Foundation. ALL RIGHTS RESERVED.

Why are we here?

• Building on the 4 yearly WSAA benchmarking

cycle

• Marketing analysis

• Opportunity but with some complexity

• TC project to address complexity


TC Project Scope# Task Completed?

1 Form a North American Steering Group that is made up of six utilities/leaders and industry association representatives from AWWA, WSAA, WEF, and possibly others.

Y

2 Evaluate potential tools and recommend a process that can be applied to the execution of the 2016 Utility Benchmarking project.

Y

3 Develop a base case and options for an implementation approach, suggested tool and process methodology, and availability of materials that can be reviewed and lead to recommendations.

Y

4 Conduct a Benchmarking Workshop with the Steering Group that incorporates items 1, 2, and 3 above

In progress

5 Document the workshop and outcomes in a Benchmarking Workshop Results Report

N

6 During course of the execution of the 2016 Utility Benchmarking project, develop an evaluation, business case, and recommendation

N

7 Document the evaluations result in a Benchmarking Evaluation Report N

8 Provide Leading Practice Report from 2016 Utility Benchmarking project N


TC Project Organization ChartWRF

Linda Reekie

Steering Group• WSAA – Greg Ryan • AWWA – Ken Mercer• WEF – Matt Ries

• 5 Utilities

CH2MScott Haskins, PI

Priscilla Bloomfield, PM & Lead Analyst

Steering Group Utilities: Portland Water Bureau, DC Water and Sewer, Toho Water, Metropolitan Council Environmental Services (Minneapolis-St Paul), Albuquerque Bernalillo County Water Utility Authority

EMA (sub)Terry Brueck

Project Advisory Committee

CH2M Resources


Functional Structure

WRF TC WSAA AMCV AWWA Metrics Benchmarking

Project Advisory Committee

AMCV Steering Group

AWWA Steering Committee

TC Outcomes- Enhanced

process & tool proposition

- Business Case

Utility & Industry Reports

Utility & Industry ReportsLeading Practice

Workshop


ScheduleMonth TC Project N American AMCV Benchmarking

Project

Dec WorkshopWorkshop Summary Report

Utility sign up period

Jan Preparation

Feb Quarterly SG Meeting Kickoff and Training

Mar -May

Training Facilitated assessments/selfassessments

June Quarterly SG Meeting Facilitated assessments/audits

July Facilitated assessments/audits

Aug Facilitated assessments/audits

Sept Quarterly SG MeetingSurvey

Draft utility and industry reportLeading Practice Workshop

Oct SurveyDraft Evaluation Report/BCELeading Practice and Industry Reports

Final utility and industry reports

Nov Draft Evaluation Report/BCE Review

Dec Quarterly SG MeetingFinal Evaluation Report/BCE


Deliverables

• Benchmarking Workshop Recommendations Report

• Draft and Final Benchmarking Evaluation Report

• North American Leading Practice Report from 2016 Utility Benchmarking project

• North American Industry Report


Tools and frameworks considered

• Asset Management Customer Value

• Effective Utility Management

• AWWA Survey

• NACWA Survey

• Matt Ries PhD research

• ISO 55000


Benchmarking Process and Approach

Greg Ryan


Asset Management Customer Value Project (AMCV)Enabling Business and

Customer Value through Asset Management

Previously

Enabling Business and Customer Value through Utility Management

Previously

The next generation of management

‘An international utility knowledge base that drives world class

utility management and delivers enhanced customer value’


Evolution since 2004

The next generation of utility managementThe next generation of management

Success, learning & improvement

2004Audit focus

2008Review & improvement

2012Utility engagement & development

2016International excellence and tailored learning

The next generation of managementThe next generation of management


AMCV 2016 Snapshot

CH2M appointed for North America with AECOM, Third Horizon for elsewhere

Costs reduced

from 2012 (Assisted vs

self-assessed options)

The assessment

tool has been

updated (25%)

• Customer focused• ISO55001

consistency/maturity

• Incorporate AWWA metrics and EUM linkage

The next generation of utility managementThe next generation of managementThe next generation of managementThe next generation of management


Initial Indication of Participant Numbers


• USA/Canada

• 11 confirmed utilities (up to 30 total utilities)

• Australia (21 confirmed and up to 40 total water utilities)

– Goal 5-10 non water utilities – gas, oil, rail, ports

• Europe: 4 confirmed, up to 10 water utilities (incl. interest from Suez &

Veolia)

• Japan – initial interest from 5 water utilities

• Manilla/ Philippines: 2-3 water utilities

• Middle East: 1-4 utilities

• South Africa: 1-2 utilities



Key aspects of the AMCV


The AMCV framework covers seven key lifecycle and support functions driving business outcomes from an asset management perspective



The AMCV Process

Select assessment option

Internal Review

Verification

Identify leading practices

Ongoing improvement

Incorporated into facilitated

approach or separate 2d

reviewConsultant coordinated

What you want reported – AMCV, EUM, AWWA, ISO

Facilitated or self assessed

Training

(Online, recorded or face

to face)


Project outcomes


Utility Report Industry

Report

Leading

Practice

Report

TC Project

Report

Utility engagement, understanding, sharing and learning



Success Criteria


• Learning opportunities with international water and other

businesses

• Evaluation of 2016 project for North American utilities

• First steps in developing an integrated benchmarking

approach for the US

• Evaluation of that approach

• Development of a business case for future integration with

North American tools



AWWA TEC

Ken Mercer


Associated industry frameworks and features


AMCV and ISO55000

Greg Ryan


AMCV Enhancements (summary)

Two options for internal

data gathering-consultant assisted or

utility unassisted

Leaner more engaging training,

with on-line support

Enhanced engagement

and interaction

Simpler data entry

• Online• Peer-to-peer



Additional AMCV Enhancements


• Alignment to ISO55000

• Approaches to align with EUM and AWWA Metrics

• Specific engagement platform

• On-line interaction and questions

• Changed database

• Flexibility to map against ISO55000, EUM, AWWA Metrics

• Easier data entry and review



AMCV Specific Enhancements


• Beyond 2016

• Ability for ongoing interaction (AMCV Life)

• Share achievements

• Lessons Learned

• AMCV Learn for small to medium utilities

• AMCV Lite

• Leading practices but without verification

• AMCV Leap – only undertake 2-3 Functions



Approach to Mapping AMCV, EUM, AWWA metrics, ISO

Greg Ryan


Alignment of Aquamark to ISO55001 – App B


How the AMCV accommodates different approaches

2016 Software

AMCV 2016

Function 1

ISO 55000 Survey/EUM Attribute/AWWA Metric

Function 2

Process 1.1

Sub-Process 1.1.1

Measure 1.1.1.1

Process 1.2

Sub-Process 1.1.2

Measure 1.1.1.2

Measure 1.1.1.2

Category 1 Category 2

Sub-category 1 Sub-category 2


ISO 55001 linkagesISO 55001 Requirements• Defined by the 71 ‘shall’ statements• Aggregated according to topic where relevant for interpretation

Aquamark linkages • At Aquamark measure level• ‘One –to-many’ relationship between each ISO ‘shall’ statements

and one or more Aquamark measures, grouped by subject or relevance

• Categories of ‘compliance’, ‘evidence’ and ‘beyond’ relating to each Aquamark measure

• 44 NEW measures required to cover gaps between Aquamark and ISO

Scoring• ISO compliance alignment with Aquamark maturity scale• Consider impact of Process Effectiveness component


Illustration of links between AMCV and ISO55000

• Development of the pilot model for converting 2012 AMCV (Aquamark) scores to ISO55001 compliance scores, and providing a tool for scoring gaps and partial ISO gaps in Aquamark

New – Additional AMCV measuresTweeks – Modification to existing AMCV measures


ISO 55000 Maturity assessment


AMCV Metrics Data Entry


BREAK


Effective Utility Management

Matt Ries


Effective Utility Management• 10 attributes

— Product quality— Employee & leadership development— Financial viability— Operational resiliency— Water resource adequacy— Customer satisfaction— Operational optimization— Infrastructure stability— Community sustainability— Stakeholder understanding

• 5 keys to success— Leadership, Strategic business planning, Organizational

approaches, Measurement, Continual improvement management framework


• Workgroup – association representatives— Shellie Chard-McClary, Oklahoma DEQ— Andrew Clarkson, American Water— Lisa Daniels, Pennsylvania Bureau of Water— Ken Fischer, Southwest Water Company— Dan Hartman, West Palm Bay, FL— George Martin, Greenwood, SC— Diane Taniguchi-Denis, Clean Water Services, OR— Tyler Richards, Gwinnett Co., GA— Steve Schneider, St. Paul, MN— Tom Sigmund, NEW Water (Green Bay, WI)— John Sullivan, Boston Water & Sewer Commission— Tim Wilson, Marshalltown, IA

• Plus association workgroup representatives

• Addition of ACWA and ASDWA in 2015



Effective Utility Management: Key Operating Context Changes

• Automated and “Smart” Systems and Data Integration

• Climate Variability and Extremes• Customer Expectations and Public Awareness• Employee Recruitment and Retention• Resource Recovery• Regulatory Requirements and Operating

Conditions• Stormwater and Watershed Management



• Anticipated that same 10 attributes will be retained—But updates to each attribute—Emphasis on expanding the business model

(to recovery of resources, energy & nutrients)

• 5 keys to success will remain the same—Possibly add a key focused on knowledge

management


AWWA Survey

Terry BrueckStephanie Passarelli


• AWWA began developing a utility benchmarking program in 1995

• Well-defined and time-tested performance indicators specific to the water sector

• Surveys conducted in 2004, 2006, 2007, 2009, 2010, 2011, 2012, 2013

• Today’s Utility Benchmarking Program— Annual Utility Benchmarking Survey— Participant report— Public release of aggregate results

Evolution


Benchmarking Performance Indicators


Performance Indicators Mapped to 10 Attributes of Effective Utilities


Utility Benchmarking Survey

• Electronic Based Survey— Questions— Output Metrics

• Performance Indicator Categories— Organizational

Development— Business Operations— Customer Service— Water Operations— Wastewater

Operations


Utility Benchmarking Survey

• Performance indicators generalized/ normalized in order to provide greatest general applicability

• Report central tendency measures (quartiles)

• Provides system for decision-makers to understand/quantify past and current performance


Benchmarking Survey Improvements

• Ease of use• Data quality

(confidence levels)• New indicators


NACWA Survey

Scott Haskins


NACWA Survey

• Conducts annual survey

— Largely financial information

— Metrics appear to be covered by AWWA metrics survey

• Exec Director and Staff positive about project

• Board meeting to confirm extent of involvement

• Likely outcome – to incorporate NACWA ‘major

industry themes’ into practice measures


PhD Dissertation

Matt Reis


A Framework to Assess Key Attributes Driving Sustainability for U.S. Urban Water Utilities

Matt Ries, P.E.PhD Candidate, Civil & Environmental Engineering

University of South FloridaSenior Research Fellow,

USF Patel College of Global Sustainability

© 2015 Water Research Foundation. ALL RIGHTS RESERVED.50

Map: Athanasius Kircher’s Mundus Subterraneus, 1664

Background


Background: Internal/External Commonalities

51

External Drivers

Climate change

Regulations Aging infrastructure

Economic pressures

Internal Attributes


Sustainability & Performance Indicators

Performance Indicators

52

Sustainability Indicators

Inter-generational, or Predictive

Indicators

Risk assessment &

mitigation

Long-term strategic/

financial plan

Long-term water supply

adequacy

Bond rating or credit strength

Avg. wait time per call

Revenue/ expenditure

ratio

Regulatory compliance

Service disruptions


U.S. Water Sector: data availability

53

European Benchmarking Cooperation

(2005)

American Water Works Association

(2005)

World Bank (1995)


Methodology

54

External Advisory Committee

Water Professionals

1

2

© 2015 Water Research Foundation. ALL RIGHTS RESERVED.55

Sustainability question number Key Question

1 What do you think about using the “triple bottom line-plus” framework,with the plus being infrastructure, as a water utility sustainabilityframework?

2 What do you believe are the most important economically-sustainablepractices for U.S. urban water utilities?

3 What do you believe are the most important environmentally sustainablepractices for U.S. urban water utilities?

4 What do you believe are the most important socially sustainable practicesfor U.S. urban water utilities?

5 What do you believe are the most important infrastructure-relatedsustainability practices for U.S. urban water utilities?

6 What do you see as the most significant barriers to more widespreadadoption of sustainability indicators?

7 Do you currently, or do you plan to publicly reporting your utility’ssustainability performance, either through Global Reporting Initiative (GRI)formats or others?

Method #1: Semi-Structured Interviews

recording → transcription → discourse analysis


Question 1Indicate if you primarily interact with (choose one):• Water utilities• Wastewater utilities• Both or combined water/wastewater utilities

Question 2Provide up to 20 brief responses for the following. “LIST EXAMPLES OF SUSTAINABLE PRACTICES FOR U.S. URBAN WATER UTILITIES.” Do not research the answers. Rather, simply provide answers in the order they come to mind.

56

Method #2: Freelisting Survey

Researchware.com & Analytictech.com


0

1

2

3

4

5

6

7

8

No.

of

Res

pons

es

Sustainability Results: Semi-Structured Interviews

57

Rank Practice Responses1 Education & communication 8

2 (T) Community ROI 62 (T) Bond rating/financial management 62 (T) Resource recovery 62 (T) Green infrastructure 62 (T) Asset management 67 (T) Meet or exceed permit 57 (T) Environmental justice 57 (T) Water conservation 57 (T) Habitat/Watershed protection 511 Affordability 412 Long-term resource plan 313 Maintenance plan/MMS 214 Sourcewater protection 215 Multi-function infrastructure 216 Good neighbor 217 Ability to adapt/flexibility 218 Recycling/minimize materials 219 Providing access to water 120 LCC approach 121 Growth rate (city) 122 Availability of water resources 123 Commercial/residential distribution 124 Energy costs 125 Fit-for-purpose water 126 Water losses 127 Climate 128 Stormwater 129 Community giving 130 Understanding service level 131 Everyone pays 132 Fixed cost rate model 133 Service outages 134 Response time (customer calls) 135 Value engineering 136 Resiliency 137 Minimize maintenance 138 Envision rating system 139 Spills/overflows 140 Pipe leaks 1


Sustainability Results: Semi-Structured Interviews

58

Rank Practice No. of Responses

1 Education & communication 82 Community ROI 6

3Bond rating/financial management 6

4 Resource recovery 65 Green infrastructure 66 Asset management 67 Meet or exceed permit 58 Environmental justice 59 Water conservation 510 Habitat/Watershed protection 511 Affordability 412 Long-term resource plan 3


Sustainability Results: Freelisting Survey

59

0%

10%

20%

30%

40%

50%

60%

70%

Res

ourc

e re

cove

ryW

ater

con

serv

atio

nAss

et m

anag

emen

tEn

ergy

eff

./ E

sta

r /

E co

ns.

Bond

rat

ing/

fina

ncia

l m

anag

emen

tG

reen

inf

ra/p

erm

eabl

e pv

mt

Ren

ewab

les

Empl

oyee

ski

lls

eval

/pla

n/H

RLo

ng-t

erm

res

ourc

e pl

anEd

ucat

ion

& c

omm

unic

atio

nClim

ate

Hab

itat

/wat

ersh

ed p

rote

ctio

nEn

visi

on/L

EED

rat

ing

syst

emRec

ycling

/min

. m

ater

ials

Ris

k an

aly.

/vul

nera

bility

ass

ess.

Gre

en c

hem

istr

yC

onti

nuou

s im

prov

emen

tAva

ilabi

lity

of w

ater

res

ourc

esPe

rfor

man

ce m

easu

res/

KPIs

Sour

cew

ater

pro

tect

ion

Tre

atm

ent

wet

land

s/na

tura

l sy

s.Su

stai

nabi

lity

ana

lysi

sLe

ader

ship

AM

IM

eet

or e

xcee

d pe

rmit

Wat

er a

udit

s /

wat

er l

osse

sM

itig

atio

n/ad

apt

& f

lood

bar

rier

sD

ecen

tral

izat

ion/

Dis

t. S

yste

ms

Envi

ronm

enta

l m

itig

atio

nH

ealt

h &

saf

ety

Min

imiz

e m

aint

enan

ceReg

ulat

ory

know

ledg

eAud

its

Stra

tete

gic

busi

ness

pla

nSt

orm

wat

erD

ocum

ent

cont

rols

Inte

grat

ed w

ater

man

agem

ent

Benc

hmar

king

Org

aniz

atio

nal re

spon

sibi

lity

plan

Trai

ning

Reg

iona

l pa

rtne

rshi

psAut

omat

ion

Emer

genc

y re

spon

se p

lan

ISO

…N

on-c

orro

sive

col

l. S

yste

mVFD

sW

ater

qua

lity

/qua

ntit

y da

taEn

viro

nmen

tal

stew

ards

whi

pW

ater

sup

ply

dive

rsif

icat

ion

LCC

app

roac

hFl

exib

le m

anag

emen

tSO

PsSu

pply

cha

in m

anag

emen

tRes

ilien

cyBr

acki

sh g

roun

dwat

er u

sage

Red

uced

I/I

Cor

rect

ive/

prev

enta

tive

act

ion

plan

sFi

t-fo

r-pu

rpos

e w

ater

Reg

ulat

ory

supp

ort

- su

st.

mea

s.Su

cces

sion

pla

nnin

gPe

ak s

havi

ngW

ater

mar

kets

(pr

ivat

e ex

chan

ges)

Abi

lity

to

adap

t/fl

exib

ilit

yAff

orda

bility

Cro

ss-f

unca

tion

al t

eam

sM

obile

tech

nolo

gyC

ompo

stin

gEM

SBu

sine

ss-m

inde

d C

IPSo

urce

con

trol

/pre

trea

tmen

tSm

art

irri

gati

onSo

urce

sep

arat

ion

Smar

t ci

ties

Mgm

t re

view

of

org.

im

prov

emen

tAna

mm

oxG

oal-

sett

ing

& p

lann

ing

Und

erst

andi

ng s

ervi

ce lev

elFO

G r

ecyc

ling

Com

post

ing

toile

tsEU

MC

ultu

ral pr

eser

vati

onTa

nkle

ss w

ater

hea

ters

Inno

vati

ve f

inan

cing

GH

G m

easu

rem

ent

GRI

Cul

tura

l/or

gani

zati

onal

align

men

tSu

stai

nabi

lity

mgm

t sy

stem

sTr

ansb

ound

ary

wat

er law

sQ

ualit

y of

lif

eBe

havi

oria

l ec

onom

ics

(billin

g)

Resp

onse

rat

e


Sustainability Results: Interview/Survey Comparison

60

Interview results Survey resultsRank # (of 40) Practice Practice

Rank # (of 90)

1 Education & communication Education & communication 10

2 (T) Community ROI N/A

2 (T)Bond rating/financial management Bond rating/financial management 5

2 (T) Resource recovery Resource recovery 1

2 (T) Green infrastructureGreen infrastructure/perm. pavement 6

2 (T) Asset management Asset management 3

7 (T) Meet or exceed permit 25

7 (T) Environmental justice N/A

7 (T) Water conservation Water conservation 2

7 (T) Habitat/Watershed protection Habitat/Watershed protection 12

11 Affordability 64

12 Long-term resource plan Long-term resource plan 6


Preliminary Results: Semi-Structured Interviews

Barriers to adoption of

sustainability measures?

61

• Resources (time, staff, $)• Lack of incentive (monopoly)• Lack of definition of sustainability

Actions to drive adoption of

sustainability measures?

• Public demand• Political will• Tools, Vision, Regulations


Attributes Results: Semi-Structured Interviews

62

Rank AttributeNo. of Responses

1 1 Leadership 7

2 2 (T)Board support/political will

5

3 2 (T)Link job to sustainability

5

4 4 Training 4

5 5Strategic planning / deployment

3

6 6 (T) Flexible staff 2

7 6 (T) Incentives 2

8 6 (T) Innovative culture 2

9 6 (T) Vision 2

10 10 (T) Break down silos 1

11 10 (T)Communicating sust. (internal)

1

12 10 (T)Continuous improvement

1

13 10 (T) EMS/ISO 1

14 10 (T) External focus 1

15 10 (T)Financial management

1

16 10 (T) Measurement 1

17 10 (T)Mindset beyond permit

1

18 10 (T) Reliable service 1

19 10 (T) Staff 1

0

1

2

3

4

5

6

7

8

No. of Responses


0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Publ

ic/s

take

hold

er o

utre

ach

& e

ngag

em't

fina

ncia

l m

anag

emen

t/st

ewar

dshi

pSt

aff

trai

ning

& d

evel

opm

ent

Lead

ersh

ipCoo

pera

tion

wit

h ot

her

orgs

/uti

liti

esClim

ate

adap

tati

on/m

itig

atio

n /

goal

sSu

st.

Mgm

t. P

rog.

/Goa

ls-c

omm

itm

ent

Cul

ture

- o

pen

to n

ew ide

asIn

nova

tion

- c

ultu

re CI

infr

astr

uctu

re p

lann

ing

& m

aint

enan

ceSy

stem

s th

inki

ngLe

ader

ship

dev

elop

men

tRat

es s

uppo

rt u

pdgr

ades

(fu

ll c

ost

$?)

EUM

Ener

gy e

ffic

ienc

yPo

l. s

uppo

rt/c

oaliti

ons

w/

pub.

off

icia

lsRes

ourc

e re

cove

ryw

ater

res

ourc

es p

lann

ing/

adeq

uacy

Wat

er r

euse

Envi

ronm

enta

l aw

aren

ess/

stew

ards

hip

Obj

ecti

ves

/ ta

rget

sCul

ture

- a

lign

edReg

ulat

ory

com

plia

nce

TBL

Ass

et M

anag

emen

tCI -

KPI

sSo

urce

wte

r/w

ater

shed

pro

tect

ion

Staf

fing

eff

icie

ncy

Indu

stry

aw

aren

ess

Com

mun

ity

RO

I/Q

OL

Aud

its

Inte

grag

ed p

lann

ing

Tec

hnol

ogy

(CM

MS,

SCAD

A)

/ In

telleg

ent

WS

P3 /

inn

ovat

ive

fina

ncin

gRes

earc

hO

pera

tion

al r

esili

ence

Flex

ibilit

ySa

fety

pro

gram

Com

mit

men

t to

pub

lic h

ealt

hCI -

fin

anci

al r

epor

ting

Cul

ture

- t

eam

wor

kCul

ture

- r

isk

taki

ngLi

nk lan

d us

e /

wat

er m

anag

emen

tCom

mun

ity

sust

aina

bility

Reg

ulat

ory

unde

rsta

ndin

g (T

MD

L)CI -

Lon

g an

d sh

ort

term

cap

. pl

anni

ngU

se o

f be

st e

ffec

tive

pra

ctic

esD

ata

/ to

ols

CI -

sta

ffin

g pl

anni

ngCos

t av

oida

nce

Tech

nolo

gy t

o re

duce

cos

tsCul

ture

- lis

ten

to a

ll e

mpl

oyee

sU

tilit

y in

tegr

atio

nEP

A's

willing

ness

to

try

new

sol

utio

nsSt

aff

- s

elf-

mot

ivat

edIn

fras

truc

ture

sta

bility

E-W

Nex

usCus

tom

er f

eedb

ack

Priv

ate

sect

or e

xper

ienc

eIn

cent

ives

/ p

roce

ss im

prov

emen

tG

row

th m

anag

emen

tCre

ativ

ity

Long

ter

m p

lann

ing

Wat

er c

onse

rvat

ion

inno

vati

onTra

nspa

renc

ySu

cces

sion

pla

nnin

gD

esal

adv

ance

sPi

lot

proj

ects

Con

sent

dec

rees

Org

aniz

atio

n ap

proa

ches

LCC

Org

aniz

atio

nal m

anag

emen

tD

ynam

ic s

imul

atio

n m

odel

ing

Inte

rnal

ide

as (

not

all co

ntra

ctor

s)St

rate

gica

lly

focu

sed

Polic

ies/

proc

edur

esAff

orda

bility

Wat

er m

arke

tsG

reen

inf

rast

ruct

ure

Cri

sis

Benc

hmar

king

New

sta

ffO

pera

tion

al e

ffic

ienc

yO

utsi

de ind

ustr

y aw

aren

ewss

Off

stre

am s

tora

geSt

orm

wat

er m

anag

emen

tCus

tom

er s

ervi

ceRed

uce

foss

il fu

els

Rec

ycle

d m

ater

ials

Org

aniz

atio

nal vi

sion

Pollut

ion

prev

enti

onCus

tom

er-o

rien

ted

Cul

ture

- e

mpo

wer

men

tH

appi

ness

Ris

k as

sess

men

tRed

uce

chem

ical

sLe

ak m

anag

emen

tO

ptim

ism

Lean

man

ufac

turi

ng

Resp

onse

Rat

e

Attributes Results: Freelisting Survey

63


Attributes Results: Interview/Survey Comparison

64

Interview results Survey results

Rank (of 19) Attribute Attribute

Rank (of 98)

1 Leadership Leadership 4

T2 Board support/political willPolitical support/coalitions with

public officials17

T2Link employees’ jobs to

sustainabilitySustainability mgmt.

programs/goals‐commitment7

4 Training Staff training & development 3

5 Strategic planning/deployment 76

T6 Staff (flexible) Culture ‐ open to new ideas 8

T6 Incentives 61

T6 Innovative culture Innovation – Culture 9

T6 Vision 91


Measurement

65

Practices• Asset management• Education & communication• Financial management• Green infrastructure• Habitat/watershed protection• Long-term resource plan• Resource recovery• Water conservation

Attributes• Board support/political will• Innovative culture• Leadership• Flexible staff• Organizational commitment• Staff training/development


Measurement

66

Practices• Asset management• Education & communication• Financial management• Green infrastructure• Habitat/watershed protection• Long-term resource plan• Resource recovery• Water conservation

Attributes• Board support/political will• Innovative culture• Leadership• Flexible staff• Organizational commitment• Staff training/development


Example

67

Interviews• Education (1, 7, 11)• Connected to community,

provide speakers bureau (2)• Surveys after service, dialogue

(6)• Public support, social

awareness (5)• Communicate (7, 11)• Cultural competency and

outreach (9)

Surveys• Education on STEM• Communication plan• Measured support of community

sustainability efforts• Stakeholder engagement• Stakeholder collaboration• Partnerships with others• Community support for

sustainability efforts


Example

68

Interviews• Education (1, 7, 11)• Connected to community,

provide speakers bureau (2)• Surveys after service,

dialogue (6)• Public support, social

awareness (5)• Communicate (7, 11)• Cultural competency and

outreach (9)

Surveys• Education on STEM• Communication plan• Measured support of

sustainability efforts• Stakeholder engagement• Stakeholder collaboration• Partnerships with others• Community support for

sustainability efforts

Education & Communication


Indicator examplePractice 2: Education & Communication

Indicator 2.1: Does your utility have a public education program about its sustainability efforts? Guidance: A public education program is externally‐focused and designed to build support for and awareness of utility operations and sustainability efforts.

1 2 3 4 5This activity is not practiced at our utility

This activity is implemented, but only occasionally or without uniformity

This activity is implemented, but there is room for substantial improvement

This activity is largely implemented, but there is room for improvement

This activity is fully implemented at our utility

Indicator 2.2: Does your utility have an effective communications plan that surveys stakeholders and engages them in dialogues?Guidance: A communications plan solicits responses from and engage stakeholders before, during, and after service events and infrastructure activities.

1 2 3 4 5This activity is not practiced at our utility

This activity is implemented, but only occasionally or without uniformity

This activity is implemented, but there is room for substantial improvement

This activity is largely implemented, but there is room for improvement

This activity is fully implemented at our utility

69


MCES Case Study

Leisa Thompson


☑ Vision


☑ Connect Silos☑ Identify Important & Urgent☑ Shared Focus


Creating Clarity and ConnectionMission, Vision, Values

•What we do•Where we want to be•How we do it

Outcomes•Desired end state•Clustered into EPA Water Sustainability Framework

Strategies •The paths to achieve outcomes

Strategic Goals

•Measures of performance•Movement from X to Y•Time based

Business Unit &

Team Goals

•Cascaded from strategic goals

•These goals support achievement of strategic goals

Individual Goals

Business AssessmentAssessed Practices & Metrics to Provide Detailed Status, Prioritization & Urgency as Input to Initiatives

EUMAquamarkQualServeISO 55000

Best Practices


Bundling industry leading tools, practices, metrics

Industry Tool Strengths Weaknesses

WSAA Very detailed practicesBroad and deep asset management focus

No metrics

EUM Framework Broad coverage of utility management (practices and metrics)

No deep coverage

AWWA Metrics Detailed metrics No practices

NACWA Survey Strong financial focus Survey, not metrics

ISO55000 Broad asset management focus

Framework and guidance only


Deliverables

• Completed assessment matrix

• Prioritized improvement recommendations

• List of prioritized candidate key performance indicators


Project elements Project kickoff

Team selection and training

Practices and metrics assessment

Findings and recommendations

Update strategic initiatives


7 Functions and teamsFunction ESET Lead Team Members

Corporate Policy and Business Planning

Leisa Thompson Leisa Thompson, Karen Neis, Bryce Pickart, Mike Mereness, Jim Schmidt, Sam Paske, Larry Rogacki, Jason Willett

Capability Forward Planning

Ned Smith George Sprouse, Kyle Colvin, Dave Simons, Judy Sventek

Acquisition Bryce Pickart Deborah Peterson, Scott Dentz, Paul Dietz, Pat Oates, Jim Schmidt

Operations Mike Mereness Craig Edlund, Dan Fox, Rene Heflin, Dan Frey, Girma Yismaw, Dave Gardner, Mary Gail Scott, Lynn Schneider

Maintenance Jim Schmidt Nick Davies, Dan White, Tim Maranda, John Peick, Dave Quast, Jim Sailer, Tim Keegan

Rehabilitation and Replacement

Sam Paske Jim Nally, Nick Davern, Adam Gordon, Jim Wawra, Dennis Lindeke

Business Support Systems

Larry Rogacki Ricky Arora, Martina Nelson, Sara Landgreen, Dawn Ellis, Terrie O’dea, Roger Knuteson, CammyJohnson, Laura Fletcher, Dan Vaaler, Judy Sventek, Matt Strickland, Matt Gsellmeier


Schedule

Kick

off

Trai

ning

, Pr

acti

ces

and

Met

rics

Ass

essm

ent

Trai

ning

, Pr

acti

ces

and

Met

rics

Ass

essm

ent

Trai

ning

, Pr

acti

ces

and

Met

rics

Ass

essm

ent,

Find

ings

and

Re

com

men

dati

ons

Find

ings

and

Re

com

men

dati

ons,

Upd

ate

Stra

tegi

c In

itia

tive

s

Week\Teams March 2 March 23 April 6 April 27 May 11ESET X X XSteering X X X X X

Function 1 X X

Function 2 X X

Function 3 X X

Function 4 X X

Function 5 X X

Function 6 X X

Function 7 X X


Process Fosters Clarity, Connection & Engagement

Create a New Habit


Importance and urgency scoring1 2 3 4 5

Importance Very low importance to the utility's desired results; does not align with business drivers, strategic vision priorities, or major industry directions

Low importance to the utility's desired results; does not align closely with business drivers, strategic vision priorities, or major industry directions

Medium importance to the utility's desired results; aligns closely with business drivers, strategic vision priorities, or major industry directions

High importance to the utility's desired results; aligns with top business drivers, strategic vision priorities, or major industry directions

Very high importance to the utility's desired results; aligns closely with top business drivers, strategic vision priorities, or major industry directions

Urgency Can be delayed with very limited negative impacts to desired results

Do when time allows with limited negative impacts to desired results

Do later with minor negative impacts to desired results

Do soon because of significant impacts/risks on desired results

Do now because of major impacts/risks on desired results


Results – function 1 subprocesses

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1.1.

1 -

Ass

et M

anag

emen

t Po

licy

1.1.

2 -

Ass

et M

anag

emen

t O

bjec

tive

s1.

1.3

- Ass

et M

anag

emen

t G

over

nanc

e…1.

1.4

- Ass

et M

anag

emen

t St

rate

gy1.

1.5

- Ass

et M

anag

emen

t Pl

an…

1.1.

6 -

Perf

orm

ance

Mon

itor

ing

and…

1.2.

1 -

Proc

edur

es,

Gui

delin

es a

nd…

1.2.

2 -

Qua

ntit

ativ

e Ana

lysi

s1.

3.1

- Ass

et-b

ased

Cos

t Att

ribu

tion

1.3.

2 -

Dev

elop

men

t of

Fut

ure…

1.3.

3 -

Busi

ness

Cas

e D

evel

opm

ent

1.3.

4 -

Ass

et L

ife

Pred

icti

on1.

3.5

- Ass

et R

epla

cem

ent

Val

uati

ons

1.3.

6 -

Val

idat

ion

and

Con

firm

atio

n of

…1.

4.1

- Ris

k Po

licy

and

Busi

ness

Con

text

1.4.

2 -

Risk

Iden

tifi

cati

on1.

4.3

- Ri

sk Q

uant

ific

atio

n1.

4.4

- Ri

sk E

valu

atio

n1.

4.5

- Ri

sk M

itig

atio

n1.

4.6

- Ri

sk M

onit

orin

g1.

5.1

- U

nder

stan

d Ass

et P

erfo

rman

ce,…

1.5.

2 -

Und

erst

andi

ng C

omm

unit

y Le

vel…

1.5.

3 -

Dis

cuss

ions

wit

h Reg

ulat

ory…

1.5.

4 -

Impa

ct S

cena

rios

of

Prop

osed

…1.

5.5

- D

efin

ed L

evel

s of

Ser

vice

1.5.

6 -

Trac

king

Fut

ure

Cha

nges

in…

1.6.

1 -

Ass

et M

anag

emen

t Sk

ills

and…

1.6.

2 -

Staf

f D

evel

opm

ent

and…

1.6.

3 -

Staf

f Su

rvey

s, F

eedb

ack

and…

1.6.

4 -

Staf

f W

orkp

lace

Com

plia

nce

1.6.

5 -

Man

agin

g O

rgan

isat

iona

l Cha

nge

1.7.

1 -

Ass

et M

anag

emen

t Pl

ans

1.8.

1 -

Gov

erna

nce

and

Polic

y1.

8.2

- M

anag

emen

t of

Inpu

ts t

o…1.

8.3

- Cap

ital

Pri

orit

isat

ion

Proc

ess

1.8.

4 -

Man

agin

g Pr

ogra

m V

olat

ilit

y or

…1.

8.5

- Pr

ogra

m D

eliv

erab

ility

1.8.

6 -

Mon

itor

ing

and

Rep

orti

ng…

1.9.

1 -

Doc

umen

t Pr

oces

ses

and

Proc

ess…

1.9.

2 -

Res

ourc

ing

and

Proc

ess

Con

trol

1.9.

3 -

Proc

ess

Aud

it R

egim

e1.

9.4

- Pr

oces

s Im

prov

emen

t…1.

9.5

- In

tern

al A

sset

Man

agem

ent…

1.10

.1 -

Iden

tifi

cati

on o

f Con

figu

rati

on…

1.10

.2 -

Con

figu

rati

on C

hang

e Con

trol

1.10

.3 -

Mon

itor

ing

of E

quip

men

t…1.

10.4

- E

quip

men

t Fo

rmal

Acc

epta

nce…

1.10

.5 -

Con

figu

rati

on M

anag

emen

t…1.

11.1

- A

sset

Man

agem

ent…

1.11

.2 -

Ass

et K

now

ledg

e M

anag

emen

t…1.

11.3

- A

sset

Man

agem

ent

Info

rmat

ion…

1.12

.1 -

Inno

vati

on P

olic

y an

d Fr

amew

ork

1.12

.2 -

Cur

renc

y an

d Acc

essi

bility

of…

1.12

.3 -

Att

enda

nce

at S

emin

ars

and…

1.12

.4 -

Fos

teri

ng o

f In

nova

tion

Idea

s1.

12.5

- F

orm

al R

ecog

niti

on o

f…1.

12.6

- C

ultu

re o

f In

nova

tion

1.13

.1 -

Out

sour

cing

Fra

mew

ork

Function 1 Subprocesses

Mat

urity


Results – function 1 subprocessesby urgency

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1.1.

1 -

Ass

et M

anag

emen

t Po

licy

1.1.

2 -

Ass

et M

anag

emen

t O

bjec

tive

s1.

1.3

- Ass

et M

anag

emen

t G

over

nanc

e…1.

1.4

- Ass

et M

anag

emen

t St

rate

gy1.

1.5

- Ass

et M

anag

emen

t Pl

an…

1.1.

6 -

Perf

orm

ance

Mon

itor

ing

and…

1.2.

1 -

Proc

edur

es,

Gui

delin

es a

nd…

1.2.

2 -

Qua

ntit

ativ

e Ana

lysi

s1.

3.1

- Ass

et-b

ased

Cos

t Att

ribu

tion

1.3.

2 -

Dev

elop

men

t of

Fut

ure…

1.3.

3 -

Busi

ness

Cas

e D

evel

opm

ent

1.3.

4 -

Ass

et L

ife

Pred

icti

on1.

3.5

- Ass

et R

epla

cem

ent

Val

uati

ons

1.3.

6 -

Val

idat

ion

and

Con

firm

atio

n of

…1.

4.1

- Ris

k Po

licy

and

Busi

ness

Con

text

1.4.

2 -

Risk

Iden

tifi

cati

on1.

4.3

- Ri

sk Q

uant

ific

atio

n1.

4.4

- Ri

sk E

valu

atio

n1.

4.5

- Ri

sk M

itig

atio

n1.

4.6

- Ri

sk M

onit

orin

g1.

5.1

- U

nder

stan

d Ass

et P

erfo

rman

ce,…

1.5.

2 -

Und

erst

andi

ng C

omm

unit

y Le

vel…

1.5.

3 -

Dis

cuss

ions

wit

h Reg

ulat

ory…

1.5.

4 -

Impa

ct S

cena

rios

of

Prop

osed

…1.

5.5

- D

efin

ed L

evel

s of

Ser

vice

1.5.

6 -

Trac

king

Fut

ure

Cha

nges

in…

1.6.

1 -

Ass

et M

anag

emen

t Sk

ills

and…

1.6.

2 -

Staf

f D

evel

opm

ent

and…

1.6.

3 -

Staf

f Su

rvey

s, F

eedb

ack

and…

1.6.

4 -

Staf

f W

orkp

lace

Com

plia

nce

1.6.

5 -

Man

agin

g O

rgan

isat

iona

l Cha

nge

1.7.

1 -

Ass

et M

anag

emen

t Pl

ans

1.8.

1 -

Gov

erna

nce

and

Polic

y1.

8.2

- M

anag

emen

t of

Inpu

ts t

o…1.

8.3

- Cap

ital

Pri

orit

isat

ion

Proc

ess

1.8.

4 -

Man

agin

g Pr

ogra

m V

olat

ilit

y or

…1.

8.5

- Pr

ogra

m D

eliv

erab

ility

1.8.

6 -

Mon

itor

ing

and

Rep

orti

ng…

1.9.

1 -

Doc

umen

t Pr

oces

ses

and

Proc

ess…

1.9.

2 -

Res

ourc

ing

and

Proc

ess

Con

trol

1.9.

3 -

Proc

ess

Aud

it R

egim

e1.

9.4

- Pr

oces

s Im

prov

emen

t…1.

9.5

- In

tern

al A

sset

Man

agem

ent…

1.10

.1 -

Iden

tifi

cati

on o

f Con

figu

rati

on…

1.10

.2 -

Con

figu

rati

on C

hang

e Con

trol

1.10

.3 -

Mon

itor

ing

of E

quip

men

t…1.

10.4

- E

quip

men

t Fo

rmal

Acc

epta

nce…

1.10

.5 -

Con

figu

rati

on M

anag

emen

t…1.

11.1

- A

sset

Man

agem

ent…

1.11

.2 -

Ass

et K

now

ledg

e M

anag

emen

t…1.

11.3

- A

sset

Man

agem

ent

Info

rmat

ion…

1.12

.1 -

Inno

vati

on P

olic

y an

d Fr

amew

ork

1.12

.2 -

Cur

renc

y an

d Acc

essi

bility

of…

1.12

.3 -

Att

enda

nce

at S

emin

ars

and…

1.12

.4 -

Fos

teri

ng o

f In

nova

tion

Idea

s1.

12.5

- F

orm

al R

ecog

niti

on o

f…1.

12.6

- C

ultu

re o

f In

nova

tion

1.13

.1 -

Out

sour

cing

Fra

mew

ork

Function 1 Subprocesses

Mat

urity

Risk Policy and Business ContextUnderstanding Community Levels of ServiceAsset Management Plans


Function 1 urgent metricsCorporate Policy and Business Planning Importance Urgency1.8 - Ratio of basic salary of men to women by employee category 4 4

1.19 - Stakeholder outreach index (%) 3 41.31 - Wastewater service affordability (% of median household income) 5 41.32 - Debt ratio (%) 5 41.45 - Technical service complaints/1000 accounts 4 4

1.46 - % of major processes and functions with established LOSs 5 5

1.47 - Number of near misses 5 5

1.48 - # of incidents that require root cause analysis and mitigation steps 5 51.49 - # of root cause analysis per number of incidents 5 5


Function 1 key take awaysStrengths Opportunities

Workforce Asset management policy,framework, and plans

Financial forecasting andbudgeting

Business case development

Managing CIP uncertainty Risk management

Asset management governance Levels of service, risk and cost

Regulatory management Configuration management

Asset management improvement plan


Results Successful and efficient process Added to past industry best practice:

– Integration of metrics with practices– Urgent and Importance scale– Using 1-5 cards– Cross functional participation– Mapped to Strategic Vision (be a leader in water

sustainability)– Included multiple tools (EUM, ISO 55000, NACWA,

QualServe)

Next Steps Identified


Next Steps

• Use results for setting priorities and sequencing of initiatives

• Asset Management Initiative• High Performance Teams Initiative• Project Management Initiative• Performance Metrics Initiative• Quarterly Reporting of Strategic Initiatives


Project statistics• 7 Functions• 46 Processes• 201 Subprocesses, Importance and Urgency by

Subprocess• 449 Measures, 4 scores per Measure• 163 metrics, Importance and Urgency by metric• ~ 65 key take aways• 60 participants• ~ 84 hours • LOTS OF ENGAGEMENT!


Thank you


BREAK


Utility Profile and Drivers Methodology

Greg RyanTerry Brueck


Profile and Drivers - Context

• Two different approaches

• How can we minimise the effort for participants

• Minimise duplication

• Reduce complexity

• More efficient process


AMCV Survey Objectives1. Collect basic comparative statistics on participants

a) Scale

b) Scope

c) Size

d) Political context (e.g. regulator driven or not and the

nature of regulation)

2. Assist in matchmaking, introducing utilities with similar

drivers

3. Compare different regions and assist in targeting priorities

for industry and utility assessments


Operational Context

• Ownership – government, private, shareholder

• Operational structure – department, corporation

• Services provided – wholesale, retail, water/ww

• Size - No. of customers/connections, revenue

• Level of regulation


Management context

• Corporate objectives

• Key challenges

• Stakeholder engagement in setting levels of

service

• Level of outsourcing and their nature


Indicative Utility Statistics• Country

• Asset replacement costs

• Population served

• Volumes of water and wastewater

• Operating cost and revenue

• FTE’s – insourced/outsourced


Drivers, goals and achievements

• Key drivers

• Objectives for next 1-4 years

• Achievements over the past 4 years

• Priority areas of focus


WSAA AMCV Drivers• Regulatory

• Financial

• Sustainability

• Customer expectations and demand

• Knowledge management – staff

• Asset lifecycle management

• Industry movements

• Competition

• Technology advancements


AWWA Survey Benchmarking

• Internal (historical assessment)— Compare current performance against

past performance baseline— Track changes year-to-year or over

longer time scales

• Look at similar utilities— Profile Data (comparable statistics)— Business Drivers (context for change)


Utility Profile Data

What specific service(s) does your utility provide?

Water - Raw Source/Supply: Wastewater Collection:Water - Treatment: Wastewater Treatment:

Water - Transmission (Wholesale): Wastewater Reclamation:Water - Distribution (Retail):

Stormwater - Collection: Other (Gas):Stormwater - Green Infrastructure: Other (Electric):

Stormwater - Treatment: Other (Solid Waste):

Other (please explain)

X Xn/r n/rn/r n/r

n/r

n/r n/r

n/r n/rn/r n/r

n/rn/r n/r


Utility Business Drivers

Regulatory:Water Supply:

Rates:Aging Infrastructure:Expected Efficiency:

Growth:Security:

Strategic Planning:Economic Climate:

Continual Improvement:Workforce (Succession planning

Training):

n/rn/rn/rn/rn/r

n/rn/rn/rn/rn/rn/r

High / Medium / Low

How do the following Business Driver(s) affect your day to day operations?

Low: Routine Operations

Medium: Emerging/ Upcomingissue

High: Immediate issue requiring


Feedback

Should we consider options to further

normalise or highlight commonalities and

incorporate into process?


Metrics - Approach

Scott Haskins


Metrics

• AWWA Survey metrics included (33+ total)

—AWWA metrics analysis results are not part of the

TC project outcomes

• Co-located with measures based on mapping

to the appropriate sub-process

• Actual and target values are entered – AMCV

does not calculate the value


Options for metrics

• AWWA

• AWWA + selected number of agreed metrics

• Canvassed from NACWA and others

• Customized metrics for each utility

• extra cost ~ $2k/utility


BREAK


Importance and Urgency

Scott Haskins


Importance and urgency

• Applied at the subprocess level in AMCV• Helps to prioritize gaps to addressScore 1 2 3 4 5

Importance Very low importance to the utility's desired results; does not align with business drivers, strategic vision priorities, or major industry directions

Low importance to the utility's desired results; does not align closely with business drivers, strategic vision priorities, or major industry directions

Medium importance to the utility's desired results; aligns closely with business drivers, strategic vision priorities, or major industry directions

High importance to the utility's desired results; aligns with top business drivers, strategic vision priorities, or major industry directions

Very high importance to the utility's desired results; aligns closely with top business drivers, strategic vision priorities, or major industry directions

Urgency Can be delayed with very limited negative impacts to desired results

Do when time allows with limited negative impacts to desired results

Do later with minor negative impacts to desired results

Do soon because of significant impacts/risks on desired results

Do now because of major impacts/risks on desired results


Evaluation – Major Components

Scott Haskins


EvaluationPurpose of the evaluation

• What are we trying to accomplish through the evaluation and business case?

• Have we got the focus of the questions right for tomorrow?


Straw proposition for process

• How do you feel the process went?

• What did you like/not like about the current process?


Straw proposition for tool

• How did you feel about the functionality of

the Framework and Tools?

—Level of collaboration?

—Tools/frameworks used?

—Level of integration?

• What do you feel are the elements of a

valuable tool?


Closeout

Linda Reekie


Day 1 closeout

• How did the day go?• Day 2 plans• Shuttle to hotel at 5:15 pm• Shuttle to dinner at 6:30 pm (meet in

lobby)

OVERVIEW

16

Appendix C: Benchmarking Workshop Day 2 Presentation Appendix C includes the Benchmarking Workshop Day 2 Presentation.

© 2015 Water Research Foundation. ALL RIGHTS RESERVED.© 2015 Water Research Foundation. ALL RIGHTS RESERVED. No part of this presentation may be copied, reproduced, or otherwise utilized without permission.


Tailored Collaboration Project – Utility Excellence

December 8, 2016


Drivers Update• Regulation and Standards• Financial• Sustainability• Customer Expectations• Resources and Service Demand• Workforce Evolution• Asset Lifecycle Management• Security• Efficiency and Continual Improvement• Technology• Other


Evaluation


Straw proposition for evaluation of the process

• How did you undertake the process?— Who led the process and what was their title?— Who did you involve to do the ratings?

• How much time did you spend undertaking the process?• How do you feel the process went?• What did you like/not like about the current process?• What did people want reported, what gave the most value

and why?• What did you like least about this process?• How are you going to share the results from this process

with stakeholders – the project and the assessment?• Comment on the ability to undertake the action plan? How

well were things implemented?


Straw proposition for evaluation of the tool

• How did you feel about the functionality of the Framework and

Tools?

— Level of collaboration?

— Tools/frameworks used?

— Level of integration?

• What do you feel are the elements of a valuable tool?

• Would you undertake this again?

• How often would you undertake?

— Industry consortia benchmarking

— Internal assessment


Straw proposition for Future Opps• What would you like to see for the future frequency of the process?

— Consortia

— Individually

• How much flexibility is desired both consortia and individual

— Links to Strategic plan

— Add or modify measure

— Add new topics

— Add additional metrics

• What do you feel is the value proposition for the utility going forward

— Consortia

— Stand alone

• What level of integration of tools is desired in the future?

— Separate

— Partial

— Full


Evaluation break out groups

• Experience

• Tools & Future opportunities


Report Out


BREAK


Business case evaluation


Business case evaluation - WSAA


Business case evaluation- AWWA


Business case evaluation -utilities


Business case evaluation straw dog

— Risks of different options considerations— Benefits of different options considerations— Costs

▪ Current▪ Preferred

— Scale and scope— Options— Roles of different organizations— Tools – appetite for integration

▪ Consortia benchmarking▪ Consistent approach▪ Level of flexibility

— Revenue— Delivery options/value proposition— Market size

▪ Consortia▪ Individual


Discussion and feedback


Benefits as seen by Project Team

Objective to inform the marketing and onboarding of participants

• Quick top 3 from each person around the room


Project Success Criteria


What does success look like for the Project?

• In 12 months time what would a successful project look like?

• What would be the evaluation criteria for success?


Next steps


Next steps

• Next Steering Committee Meeting—Utility Management Conference (Feb)


Project name

1. Enhancing water utility benchmarking in North America

2. Collaborative water utility benchmarking in North America

3. Collaborative enhanced water utility benchmarking in North America


LUNCH


Day 2 closeout

• Shuttle to airport at 12:45 pm (meet in lobby)


17

Appendix D: Benchmarking Workshop Day 2 Evolution Survey Questions Appendix D includes the Benchmarking Workshop Evaluation Survey Questions developed in the break out groups.


18

Straw Proposition of Questions for Past Experience

Can questions help compare the outcomes or satisfaction of a utility using the facilitated vs. not facilitated process?

Assess before and after attitudes about importance of various components of the survey

PROCESS

o What option did you pick? Solo or facilitated

Why did you pick the option?

Would you do it the same way again???? [does this question have value]

o How did you undertake the process?

Who led the process and what was their title?

Who did you involve to do the ratings?

Did you involve stakeholders outside of your utility? (boards, consultants, customers, advisory committees: multiple choice)

Identify the types of staff involved in the project. (front‐line staff, management)

Did you involve engineering, operations, maintenance, HR, customer service? (Include an estimate of the hours involved.) (Reference AWWA survey for job categories.)

How much time did staff spend undertaking the process (total person‐hours)?

o How did you feel about each element of the project (1‐5) (training, assessment, validation, leading practices conferences, Networking/sharing, deliverables, separate and combined)

o Did the process meet your expectations?

o What are you suggestions to improve the process?

o Was it helpful/did it add value to have the ability to map to EUM and ISO (1 – 5)

o How did you feel about having metrics and measures? (Were they relevant/meaningful to your utility?)

o Was it helpful (did it add value) to have the ability to rate importance and urgency?

ACTIONS

o What implementation plans do you have? (add time periods and the number of initiatives. ) 0‐6 mon, 7‐12 mon, etc.

o Comment on the ability to undertake the action plan in the utility report (plan as recommended)? As well as what you decided to do outside of the report

o How will you use the outcomes of the process/ tool? What else could have been provided that wasn’t? What would make the tool more complete?

VALUE

OVERVIEW

19

o What did you like/not like about the current process

Would you recommend to others (consortia, individual)?

Would you do it again, and if so, when (yearly, every 4 years, etc.) (consortia, individual)?

How much would you be willing to pay (consortia, individual)?

What would you carry forward?

Enhance?

Add?

Drop?

o Have you used other benchmarking or self‐assessment or survey tools this year?

o Did the project provide additional value over previous benchmarking experiences?

What aspects provided the additional value? (provide list of options)

COMMUNICATION

o Do you plan to share the results from this process with stakeholders (provide a list of different stakeholders for selection) – the project and the assessment?

OVERVIEW

20

Straw Proposition ‐ Future Opportunities

Free form responses with examples

1. Actions and opportunities to go forward using the results (VERY HIGH)

a. Ways to address the gaps

b. What do you feel is the value proposition for the utility going forward

i. Consortia

ii. Stand alone

c. Contacts and connections to others – leading practices…?

i. Follow‐in activities post‐benchmarking

ii. Experiences of others

iii. Pointers to other resources – e.g. AWWA, WSAA, WEF,…

d. Vehicles for Sharing with others?

i. Conferences, regional workshops, other venues

e. Analysis for patterns on gaps?

i. Ways to share with peers

ii. Creation of new content

iii. Trends over multiple benchmarking results of gap closure, support for the future

2. Cost and value proposition (VERY HIGH)

a. What was the learning/value of individual vs group process?

• Impact of the process?

— Quantification of benefits

b. Initiatives justification

c. Cost and value for money

• Cost Threshold (barrier, willingness to pay)

• Facilitated vs self‐assessment – comparison of benefit, time, etc.

• Time and effort spent vs value

3. Frequency ((MEDIUM)

What would you like to see for the future frequency of the process?

a. Consortia (group process)

b. Individually (per utility)

4. Mapping to other tools, processes and resources (HIGH)

a. How much flexibility is desired both consortia and individual

b. Links to Strategic plan

c. Add or modify measure

OVERVIEW

21

d. Add new topics

— Add additional metrics

— Mapping of benchmarking tool linkage to other tools?

— Continue or expand?

— Link to utility specifics (e.g. strategic plan, etc.)

— What level of integration of tools is desired in the future?

• Separate

• Partial

• Full

Straw Proposition for Evaluation of the Tools

1. How did you feel about the functionality of the Framework and Tools? (HIGH)

a. Tools/frameworks used?

b. Level of desired integration? More? Less? (e.g. metrics and practices, SAM Gap/SIMPLE, EUM)

c. Additions to the tool content? expectations

d. What outputs would make the tool more useful? Format, content, post processing

i. Examples of tool use and results

ii. How output was used?

iii. What was missing?

2. Would you undertake this again? (MEDIUM)

a. What would help or hinder you?

b. Tools – ease of use, reporting, etc.?

3. How often would you undertake? (MEDIUM)

a. Industry consortia benchmarking

b. Internal assessment

4. Evaluation of components of AMCV (HIGH)

a. Functions 1‐6 vs 7?

b. Importance and urgency

c. Rating scale – how to improve?

d. What do you feel are the elements of a valuable tool?

Documents

TECHNICAL Research Foundation Collaborative · PDF fileWater Research Foundation Collaborative Utility ... will evaluate the process and future opportunities ... opportunity to discuss