TCRP Report 141 (Project G-11)A Methodology for Performance Measurement andPeer Comparison in the Public Transportation IndustryAndBenchmarking Public Transportation Systems in Texas

TCRP Report 141 (Project G-11)A Methodology for Performance Measurement andPeer Comparison in the Public Transportation IndustryAndBenchmarking Public Transportation Systems in Texas2010 SCOPT/MTAP Annual Winter Meeting

Linda CherringtonTexas Transportation InstituteThe Texas A&M University System

TCRP G-11 Project PurposeTCRP G-11 Project Purpose

Develop and test a methodology for performance measurement and peer comparison for:– All fixed-route components of a public transit system– Motorbus (MB) mode specifically– Major rail modes specifically

Provide guidance on applying performance measurement and peer comparison to:– Improve public transit agency operations– Demonstrate public transit’s ability to meet local or regional

transportation goals

This presentation highlights key findings and products from the project

Research TeamResearch Team

Kittelson & Associates, Inc.

Texas Transportation Institute,Texas A&M University System

Center for Urban Transportation Research (CUTR)at University of South Florida

Nakanishi Research & Consulting

Lehman Center for Transportation Researchat Florida International University

Desired Methodology AttributesDesired Methodology Attributes

Robust

Practical

Transparent

Uniform

Innovative

Adaptable

Accessible

Updateable

Build upon TCRP G-6 work TCRP Report 88 A Guidebook for Developing a Transit Performance-Measurement System

Research StepsResearch Steps

Literature review & agency experience

Identify comparison factors, performance measures

Develop initial methodology

Small-scale test, revise methodology– Agencies chose topic and reviewed results, researchers

applied method– 10 transit agencies, 5 state DOTs, Chicago RTA

Large-scale test, revise methodology– Agencies chose topic, applied method, reviewed results– 19 transit agencies, 2 state DOTs, Chicago RTA

Definition of ”Benchmarking”Definition of ”Benchmarking”

“The continuous process of measuring products, services, and practices against the toughest competitors or those companies recognized as industry leaders.”– David Kearns, Chief Executive Officer, Xerox Corporation 

“The search for industry best practices.”– Robert  C.  Camp, Best Practice Institute.  

“A process of comparing the performance and process characteristics between two or more organizations in order to learn how to improve.”– Gregory Watson,  former Vice President of Quality, Xerox

Corp.  

Levels of BenchmarkingLevels of Benchmarking

Adapted from European EQUIP benchmarking project

Level 1: Trend analysis

Level 2: Peer comparison

Level 3: Agency contact

Level 4: Benchmarking networks

TCRP Report 141 Benchmarking MethodologyTCRP Report 141 Benchmarking Methodology

1. Understand context

2. Develop performance measures

3. Establish a peer group

4. Compare performance

5. Contact best-practices peers

6. Develop implementation strategies

7. Implement the strategy

8. Monitor results

1. Understand context1. Understand context

2. Develop performance measures2. Develop performance measures

3. Establish a peer group3. Establish a peer group

4. Compare performance4. Compare performance

5. Contact best-practices peers5. Contact best-practices peers

6. Develop implementation strategies6. Develop implementation strategies

7. Implement the strategy7. Implement the strategy

8. Monitor results8. Monitor results

Performance Measure SelectionPerformance Measure Selection

TCRP Report 141 provides guidance on National Transit Database (NTD)-derivable and other commonly used measures, linked to particular topics or applications– Outcome measures that measure results– Descriptive measures that provide clues as to why the results

turned out the way they did

TCRP Report 88 provides an expanded library of measures that can also be considered for benchmarking network applications

Peer Grouping ProcessPeer Grouping Process

Methodology seeks to find agencies with similar characteristics

Methodology produces a ”likeness score” that indicates how similar or dissimilar two agencies are, and provides guidance on how to interpret the likeness score

Ideally, use 8–10 agencies with the smallest likeness scores as the peer group– Fewer peers may be used when likeness scores are out of the

desirable range, but use at least 4 peers at a minimum

Peer Grouping FactorsPeer Grouping Factors

Service characteristics– Modes operated (NTD)– Service area type (G-11)– Percent service purchased

(NTD)– Percent service demand-

response (NTD)– Vehicle-miles operated (NTD)– Annual operating budget

(NTD)

Regional characteristics– Urban area population (Census)– Population growth (Census)– Population density (Census)– State capital (G-11)– Percent college students

(Census)– Percent low-income (Census)– Roadway delay (TTI)– Freeway lane-miles (TTI)– Distance (G-11)

Many other factors considered and tested during project

These factors provided the best differentiation between potential peers, and peer groupings that were the most acceptable to agencies participating in the research tests

Software ToolSoftware Tool

Peer-grouping methodology has been incorporated into the online Florida Transit Information System (FTIS) tool– Available now– Sponsored by the Florida DOT, but provides access to the full

NTD, plus data added by the TCRP G-11 project

Requires a free, one-time registration at www.ftis.org

Testing during the G-11 project found that users were able to learn about the methodology, learn how to use the tool, and perform their first analysis with 16 person-hours of work or less– Subsequent analyses can be performed very quickly

Software ToolSoftware Tool

Identify peer groups for specific modes or agency as a whole

Software ToolSoftware Tool

Retrieve NTD-based measures for the peer group

Software ToolSoftware Tool

Analyze data within FTIS or export to a spreadsheet

$0

$2

$4

$6

$8

$10

$12

$14

$16

Altoona Anderson BattleCreek

Racine Sheboygan Sioux City Springfield Wausau Wheeling

Fare

Reven

ue/R

even

ue H

ou

r

2003 2004 2005 2006 20072007 peer group median

Software ToolSoftware Tool

Investigate performance resultsAgency-wide

0

2

4

6

8

10

12

14

16

18

20

Altoona Anderson Battle Creek Racine Sheboygan Sioux City Springfield Wausau Wheeling

Avera

ge F

leet

Ag

e (

years

)

2003 2004 2005 2006 20072007 peer group median

Published as

TCRP Report 141

http://onlinepubs.trb.org

Research ResultsResearch Results

Benchmarking and Improving Texas Rural and

Small Urban Public Transportation

Systems

Texas Department of TransportationResearch Project 6205

Overview of Project

• Establishing peer groups (rural and state-funded

urban)▫ 38 rural transit districts

▫ 30 eligible state-funded urban transit districts*

• Examining effectiveness and efficiency by peer

group

• Identifying strategies to improve performance

(transferable best practices)* Does not include transit authorities in urban areas >200,0000

20

StateTransit Funds

35%Eligible Urban

Providers

65%Rural Providers

50%Needs

50%Performance

65%Needs

35%Performance

75% population

25% land area

100% population

Transit Funding Formula

21

Performance Measures

Urban Performance • Revenue miles/

Operating expenses• Passengers/ Revenue

miles• Local investment/

Operating expense• Passengers/ Population

for urbanized area

Rural Performance• Revenue miles/

Operating expenses• Passengers/ Revenue

miles• Locally investment/

Operating expense

Peer Grouping Environmental Data

• Population

• Service area size

• Service area density

• Percent of service area population that is age 65 or

older

• Percent of households with zero automobiles

• Percent of population below poverty level

• Percent of population ages 21 to 64 that are disabled

Urban Peer Groups (4)

Rural Peer Groups (5)

Peer Group Effectiveness and Efficiency

Determining High Performers - Urban

1 Standard Deviation Above

the Mean

Determining High Performers – Rural

1 Standard Deviation Above

the Mean

Benchmarking Strategies to Improve Performance

• Strategies to grow ridership and improve effectiveness

• Efforts to manage cost to improve efficiency

• Initiatives to maximize service and labor productivity

• Projects to improve management processes

Why Peer Comparison and Benchmarking?

Informally, “the practice of being humble enough to admit that someone else is better at something and wise enough to try to learn how to match, and even surpass, them at it.”– American Productivity & Quality Center

Questions?

Linda CherringtonL-Cherrington@tamu.edu713-686-2971 ext 15140