A Comparative Evluation of Large-Scale Personal Travel

Personal travel planning projects in England

This is a pre-publication version of the following article:

Chatterjee, K. (2009). A comparative evaluation of large-scale personal travel planning projects in England. Transport Policy, 16(6), 293-305.

1

A comparative evaluation of large-scale personal

travel planning projects in England

Kiron Chatterjee

Centre for Transport & Society

University of the West of England, Bristol, UK

WORD COUNT: 8,150 (not including abstract, acknowledgement, references, tables

and figures)




2

Abstract

Findings are presented from a study assessing the effectiveness of large-scale,

residential-based personal travel planning (PTP) projects in eight areas in England.

The project evaluation results show consistent reductions in car driver trips with an

average reduction of eleven per cent. The mode of travel which experiences the most

substantial increase is walking with modest increases reported for cycling and public

transport. Results have not been disaggregated according to the type of participation

that individuals have had in projects, therefore it is not possible to identify how

project design influences outcomes. Despite the consistency of outcomes reported

and many aspects of good practice in project evaluations, there are some concerns

about evaluation methodology, notably that the estimation of outcomes might be

systematically biased. The main concerns relate to independence of evaluators,

sample sizes and survey response biases. A priority in future project evaluations is to

use independent evaluators and to collect aggregate-level travel data with which to

corroborate survey-based results and enable monitoring of outcomes over longer

time-scales. Another priority is to increase understanding of how design elements of

PTP projects influence behavioural outcomes and to develop appropriate research

methods to investigate this. Improved evaluations will better enable the value for

money of PTP to be assessed relative to other investment options.

Keywords: voluntary travel behaviour change, personal travel planning, evaluation,

travel survey




3

Acknowledgments

This paper draws largely on the evaluation findings from a research project

conducted on behalf of the Department for Transport (DfT), UK. As such, the author

would like to thank Jacqui Wilkinson and Daniel Barrett of the DfT, who were

responsible for managing the project. The author is grateful to his colleagues Jon

Parker and, Lynsey Harris of Integrated Transport Planning, Richard Armitage of

Richard Armitage Transport Consultancy, Jo Cleary of Cleary Stevens Consulting

and Phil Goodwin of the University of the West of England, Bristol, for contributions

to the project. Thanks are also due to the large number of individuals at case study

organisations and on the project steering board and expert panel who provided input

to the research. However, the views expressed in this paper are those of the author

and do not necessarily represent the individual views of members of the research

team, the organisations they represent, or the Department for Transport. Finally, the

author would like to thank the anonymous referees for their valuable comments on

the paper.




4

1. Introduction

Personal travel planning (PTP) involves directly contacting individuals with

the offer of information, assistance, incentives and motivation to enable them to

voluntarily alter their travel choices. PTP is an example of a voluntary travel

behaviour change (VTBC) measure and has also been referred to as personalised

travel planning and individualised travel marketing. In Japan, PTP projects have been

referred to as travel feedback programs. The concept of PTP has been deployed

worldwide since the 1980s. Its proponents claim that PTP projects can reduce travel

problems (e.g. congestion) without the need for expensive infrastructure projects or

unpopular restraint measures while detractors claim that the impacts of PTP projects

are modest and short-lived.

Small-scale trials of PTP took place in the UK between 1997 and 2002 before

the Department for Transport (DfT) part-funded a set of PTP pilot projects that took

place in 2003 and 2004. Seven of the pilots targeted households in residential areas,

while six targeted employees in workplaces and two targeted pupils in schools. In

addition, Transport for London (TfL), the transport delivery authority for Greater

London, also funded four pilot projects at this time. In 2004 a DfT commissioned

review was published on the experiences and potential for VTBC measures (Cairns

et al., 2004). PTP was one of ten VTBC measures assessed. Drawing on UK and

international experience, including interim results from the DfT and TfL pilot

projects, the review found that “personalised travel planning typically reduces car

driver trips amongst targeted populations by 7-15% in urban areas”. After the DfT

pilot projects were completed, it was concluded that the residential-based projects

provided more consistent results, larger impacts and better value for money than the




5

projects based on workplaces and schools, but that it is “…difficult to evaluate how

easily the results from these pilots could be replicated in other areas across the

country” (DfT, 2005).

Cairns et al. (2004) had noted concerns about evaluation methodology and

recommended that future opportunities for evaluation should be seized. They noted

that ‘…monitoring of planned large-scale projects…as part of the Department for

Transport‟s Sustainable Travel Demonstration Towns project may help to provide

more convincing evidence about the effects of such schemes.’ (Cairns et al., 2004, p.

130). A contrasting opinion was put forward in a report to the Australian

Government (AGO, 2006) which concluded that there is „…little further need to

undertake major evaluations of household projects, as…data is in broad agreement’

and in Australia ‘larger household projects routinely show decreases in car use of 4 -

15%’.

A review of Japanese PTP projects (Taniguchi et al., 2007) found an average

reduction in car use of 19% amongst participating individuals in residential PTP

projects. However, the authors noted that projects in Japan have been at a smaller

scale than those in Australia and UK – too small to assess their suitability as a

practical transport policy tool. Möser and Bamberg (2008) recently re-examined

international evidence on the impact of VTBC measures and, although their meta-

analysis of 72 projects (including PTP, travel awareness campaigns and public

transport marketing) yielded a mean increase of 11% in non-car trips (representing a

mean change in the proportion of non-car trips from 34% to 39%), the authors were

concerned about the methodological quality of the evaluations from which the data

had come.




6

The possibility of examining emerging evidence from new PTP projects arose

in 2007 when DfT appointed a team, including the current author, to research the

effectiveness of residential-based PTP in the UK and worldwide. The main aims of

the study were to identify the factors influencing effectiveness of PTP and to produce

guidance for clients and practitioners on how to deliver effective PTP projects. The

study involved detailed examination of eight case study areas in England with

investigation of PTP project delivery processes and outcomes. This was achieved

through interviews with project stakeholders and documentary analysis. Formal

meta-analysis was not attempted due to the relatively small number of projects and

the known heterogeneity of the projects (and their evaluations); rather, the intention

was to use differences between project results to deduce the factors affecting PTP

effectiveness. This paper summarises the findings of the study which relate to the

evaluation evidence and processes. Wider research findings can be found in Parker

et al. (2007) and practitioner guidance based on the study can be found in DfT

(2008).

2. The evaluation of PTP projects

PTP projects have generally been funded by public authorities and the

evaluation of public projects requires systematic investigation of their effectiveness

in achieving benefits for society and whether they represent good use of limited

resources. The main benefits from PTP projects arise from changes in travel

behaviour and the impact that these have on travel conditions, therefore a central part

of the evaluation of PTP projects is the assessment of changes in travel behaviour.




7

Although assessment of the costs of delivery of PTP projects is a major component

of evaluation, this paper focuses on project benefits rather than the costs.

Impact evaluation is concerned with a project’s inputs (resources used), its

outputs (directly attributable effects) and its outcomes (the short-, medium- and long-

term final impacts that arise for society). An impact evaluation assesses whether

intended outputs and outcomes have been achieved and whether a project provides

good value for money. Since PTP projects have no physical outputs of the kind

produced by infrastructure projects, the main emphasis is on evaluating behavioural

outcomes.

Process evaluation is concerned with the way in which a project is

implemented, and received by its target population, and is intended to help

understand how and why certain outcomes were achieved. It involves examining

whether the steps (or preconditions) that are needed for a project to reach its goals

are achieved, or not, and the reasons for this. Process evaluation supports adaptive

learning from experience and enables understanding to be gained which can assist

the design of future projects. Previous comparative evaluations of PTP projects

(Cairns et al., 2004; DfT, 2005; AGO, 2006; Taniguchi et al., 2007; Möser and

Bamberg, 2008) have concentrated on project impacts with less analysis of the

project implementation process and how this affects impacts.

The evaluation of any PTP project needs to reflect its specific objectives.

There may be a variety of motivations to commission a PTP project. For example,

projects might be aimed at outcomes such as increasing the number of trips by public

transport or decreasing the distance travelled by motorised vehicles. Local authorities

in England are required to monitor a large number of performance indicators (DfT,




8

2004; CLG, 2008) of which a number of these could be conceived to be potentially

subject to influence by PTP projects. Table 1 identifies these.

Table 1

Project sponsors may also have the objective of engaging as many members

of the public in PTP projects, or achieving a high level of satisfaction of the public

with the services offered to them in PTP projects. These can be considered as

intermediate outcomes that are likely to influence final impacts of a project.

It is clear that the motivations for PTP projects, and hence the performance

indicators that might be used in evaluations, are diverse. Comparative evaluations of

PTP projects have focused on the small number of indicators which are commonly

available from individual project evaluations. The two indicators providing the main

focus have been mode share (proportion of trips made by different modes) and the

number of car driver trips. The measurement of these indicators has generally been

achieved through travel surveys of individuals in PTP project areas. The distance

travelled by car has also been the subject of analysis in some PTP projects. There has

been little attempt (in comparative evaluations) to assess aggregate-level outcomes

such as traffic levels, congestion and public transport revenue. One exception has

been attempts to estimate aggregate carbon emissions reductions from individually

reported changes in distance travelled by car (AGO, 2006). Estimating aggregate-

level outcomes draws attention to the need to take account of all changes in travel

behaviour attributable to the PTP project – not just those by participants.

In any evaluation it is important that robust results are obtained. A number of

criteria for achieving robust results are now introduced. Validity is a fundamental




9

requirement of any evaluation. To achieve validity attention needs to be paid to

reliability (ensuring that results would be repeatable if conducted again in similar

circumstances), construct validity (ensuring that measurements are assessing what

they are intended to measure), internal validity (ensuring that measured impacts are

caused by intervention) and conceptual validity (ensuring theoretical explanation for

impact is supported by data). Also important is the longevity of impacts.

For evaluation results to apply to other persons, contexts, situations or times

it is necessary for them to have external validity (be generalisable). It is also

important that there is independence of evaluators from project delivery to avoid the

risk that results are biased towards a particular outcome. A final evaluation

consideration is consistency of evaluation designs between projects. The adoption of

consistent designs allows comparison and synthesis of results.

Even though PTP projects have been subject to substantial evaluation efforts

(much more than most other transport measures), the robustness of PTP project

evaluations has been subject of considerable debate with a number of criticisms

raised and rebuttals made. The reliability of project results has been questioned by,

amongst others, O’Fallon and Sullivan (2003) who identify inadequate sample sizes

and failure to report the sampling errors of results in past PTP evaluations. It has

been suggested that use of self-report travel surveys to obtain travel behaviour data

threatens construct validity due to response biases such as the under-reporting of

trips in surveys (identified by Stopher and Bullock, 2003) and social desirability bias

(identified by Bonsall, 2009a, as likely to be a particular problem in PTP

evaluations).

The research designs used in PTP project evaluations have been claimed to

decrease internal validity. Möser and Bamberg (2008) suggest that single-group pre-




10

and post- test designs (before and after surveys in project area without surveys in

control areas) have affected the ability to draw causal inferences. Gärling and Fujii

(2006) note the absence of the collection in travel surveys of ‘process’ measures

(referring to social-psychological measures such as beliefs and attitudes) to

supplement ‘effects’ measures (behaviour) and suggest that this limits the ability to

explain any measured changes in behaviour, thus threatening conceptual validity. It

is often noted that the longevity of PTP project impacts has not been demonstrated,

although public transport boarding data obtained in projects in Australia and

Germany has suggested that increased public transport use may be sustained up to

four years after the PTP projects (Cairns et al., 2004).

Stopher and Bullock (2003) identify a threat to external validity arising from

survey samples that are not representative of the population from which they are

drawn. The survey samples they examined contained a lower proportion of large

households than in the wider population. Their concerns about survey design and

sampling led them to estimate that the reduction in car use associated with IndiMark

PTP projects in Australia was likely to be have been over-estimated by a factor of

two. They also raise the concern that areas selected for PTP projects in the past are

not typical of the population of areas which will need to be considered for future

projects – further threatening the external validity of results. Both of these assertions

have been countered by those involved with the IndiMark projects (Roth et al.,

2003).

The lack of independence of evaluators has been noted by several authors.

Cairns et al. (2004) note that “the fact that those advocating the initiatives are

sometimes also responsible for monitoring them... has led to a lack of confidence in

conclusions amongst some professionals”. Möser and Bamberg (2008) raise an




11

additional issue in their meta-analysis of VTBC measures. They identify the risk of

reporting and retrieval biases where projects for which results are available are

systematically biased (potentially towards those that had favourable results).

The methodological quality of the project evaluations reviewed in this paper

will be assessed based on the above considerations so that a view can be taken on the

robustness of results and so that recommendations can be made for future evaluation

practice.

3. The English case studies

The research study investigated the experience of local authorities that were

making a significant commitment to PTP at the time of the study. The eight selected

local authority areas were Brighton, Bristol, Darlington, Lancashire, London,

Nottingham, Peterborough and Worcester. Table 2 gives details of PTP projects

conducted in the case study areas. It can be seen that the scale of the projects,

measured by the number of households targeted, ranged from about 2,000 to 30,000

(with the projects being delivered in 2005 and 2006 being considerably larger than

the earlier projects) and that, although most projects were associated with major

towns and cities, they varied in terms of their locations within the urban area (inner,

outer and edge locations), average income levels and other socio-economic

characteristics. In some cases the PTP project was the only significant intervention in

the locality whereas in others they were accompanied by physical or economic

improvements to non-car options.

The objectives of the projects involve both intermediate outcomes (such as

number of project participants) and final outcomes and impacts (such as modal




12

share) – although outcome objectives were rarely identified in terms of specific

targets. Many of the local authorities contracted Sustrans or Steer Davies Gleave

(SDG) to deliver their projects. In Brighton and London the projects were managed

in-house but with project support from SDG. There are a number of commonalities

between the TravelSmart®1 approach used by Sustrans and that used by SDG, but a

key difference is that, in TravelSmart, participants are provided with a menu of

information from which to select what they feel will be useful, whereas SDG use

travel advisors to engage potential participants in a short door-step conversation

during which they listen out for characteristics of travel needs and behaviour and key

motivators in order to determine what types of message and information are likely to

be relevant to that participant. The Nottingham Wollaton project, managed by the

city council, was unique in not involving personal travel advisors or a menu of

potentially useful items; letters were sent to residents with personalised travel

information including bus stop-based timetables and a smartcard prepaid with one

day’s free bus travel.

Projects varied in pre-project publicity, wider travel awareness campaign

work and the amount of working with community groups and events to promote

PTP. A particular feature of the projects in Darlington and Peterborough was the

establishment of loyalty clubs designed to engage participants and encourage them to

make pledges. These clubs also acted as a forum for motivational messages referring

to the health benefits of walking and cycling.

The organizations that conducted travel survey-based evaluations of the PTP

projects are shown in Table 3. Socialdata conducted a large proportion of the

evaluations but, in Brighton and London, specialist market research organizations

1 TravelSmart is registered by Sustrans as a trademark in the UK and uses the

IndiMark technique developed by Socialdata and supported by them in the UK




13

were contracted to conduct the evaluations. Often Socialdata has worked in

partnership with Sustrans, who manage the delivery of projects, while Socialdata

conduct evaluations and manage telephone and postal contact with project

participants. For the two recent Bristol projects (in Easton and Clifton/Cotham) the

delivery agent, SDG, also conducted evaluations but these involved small sample

sizes and were not aimed at providing statistically reliable results. For the

Nottingham Wollaton project there were no travel surveys, but bus patronage data

was used to assess impacts.

The research study occurred too early for survey results to be available for all

projects. However, for those projects where behavioural outcomes were not available

it was possible to obtain information on project participation rates and the process of

implementing the projects.

Table 3

4. Evaluation results

4.1 Intermediate outcomes

Table 3 shows that the participation rate (percentage of contacted households

defined as ‘participating’) exceeded 50% in almost all projects. However, the

definition of participating varied between projects. For example, in some projects, a

participating household is one receiving materials (‘materials’ being the various

forms of information, assistance, incentives and motivation), whereas, in the




14

TravelSmart projects, a household would also be defined as participating if it

included anyone using non-car modes even if they did not request materials.

At a more detailed level, project performance can be measured in terms of the

amount of materials distributed. Table 4 shows the distribution of information and

incentives/gifts to participating households across case studies and indicates that bus

timetables were the most commonly distributed information but that several other

forms of information and maps were widely used. Pledge cards were taken up by up

to one in five of participating households where they have been offered (Darlington,

Peterborough and Worcester). Personalised journey plans were less commonly

provided and home visits were very uncommon. Free bus tickets were provided for

everyone in the target area in one project (Nottingham Wollaton project), but only to

a small minority in most other projects.

Table 4

Customer feedback provides a direct way of assessing satisfaction with

services and there was some attempt to measure this as an indicator of project

performance, but sample sizes were small and results could not be considered to be

reliable.

4.2 Final outcomes and impacts

The travel behaviour outcomes reported by the case study projects as

attributable to PTP are shown in Table 5. The travel behaviour outcomes are based

on travel surveys undertaken before and after the PTP projects in the target areas.




15

Sample sizes for the project area surveys are shown in Table 3. Table 5 shows that,

in most cases, surveys were also carried at the same times in a comparison area to

enable the effect of background trends and other external influences to be estimated

(sample sizes are not shown for comparison areas but were at least half the sample

sizes in project area). Before and after surveys were usually conducted at about the

same time of year (although, in Peterborough and Worcester, evidence that levels of

travel activity were similar in Autumn and Spring was used to justify conducting the

after surveys in Spring even though the before surveys took place in Autumn).

The research design used in the evaluations represents a two-group pre- and

post- test design. It is not a randomised control group design since the project area

and comparison area are not randomly selected; it is a quasi-experimental design

where comparison areas are selected to be as similar as possible to the project area.

The trend for the comparison area is applied as a ‘correction’ to the results of the

before survey in the project area to provide an estimate of the counterfactual - what

would have happened in the event of PTP not being implemented in target area. The

changes in mode share and car driver trips and distance travelled by car shown in

Table 5 are estimated from travel diary data after taking into account the

counterfactual (except for Brighton Year 1 and Bristol Southville projects, where

comparison area surveys were not carried out).

The travel surveys were paper-based household surveys with one-day travel

diaries for each household member to complete (except in the case of the Bristol

Southville and Worcester projects where households members were simply asked to

report average weekly frequency of use of different travel modes). Generally,

independent (random) samples have been drawn from the project area and

comparison area populations for both the before and after surveys, although partial




16

matched samples and matched (panel) samples were used in some of the earlier

Bristol and Nottingham projects.

Table 5 includes information on other transport initiatives that took place in

the project and comparison areas between the before and after surveys and, to

provide context for interpreting the results, also includes any notable changes in

mode share measured in comparison areas (representing the counterfactual). Two

sets of results are provided for the Bristol Bishopsworth and Bishopston projects

because after surveys were carried out on two different occasions (unless otherwise

mentioned, results referred to in the main text of this paper are based on the second

after surveys).

Table 5 reports average changes in travel behaviour for the entire populations

of the target areas. Household surveys were distributed to random samples of target

population households and therefore included both project participants and non-

participants. Pre-analysis sample weighting has been applied in Socialdata

evaluations to ensure the same proportion of project participants (as defined by

Socialdata) in the survey samples as in the wider population and to ensure the same

distribution of gender and age in after sample as before sample. This means that

results can be considered to incorporate any diffusion effects from participants to

non-participants. It would have been informative to be able to report results in this

paper separately for project participants and non-participants, but this has not been

possible as it was not reported in project evaluation reports.

Referring to the results for changes in mode share, it is apparent that the share

of trips by car decreased in all projects for which results are available. Taking the

eight PTP projects whose results take account the counterfactual (three in Bristol,

one in Darlington, one in Lancashire, two in Nottingham, one in Peterborough) a




17

decrease in the share of car driver trips of up to 7 percentage points has been reported

with a project arithmetical mean decrease of 4 percentage points. The share of

walking trips is reported to increase by up to 6 percentage points with project

arithmetical mean increase of 3 percentage points. The project arithmetical mean

changes for other modes are 1 percentage point increase for cycling, 1 percentage

point increase for public transport and 1 percentage point decrease for car passenger.

The relative decrease in car driver trips varied from 4% to 13% with project

arithmetical mean decrease of 11%. This is based on 11 PTP projects (the eight

projects identified above plus three Worcester projects). The relative decrease in car

distance travelled varied from a decrease of 8% to a decrease of 15% with project

arithmetical mean decrease of 12% in car distance travelled. This is based on six PTP

projects (three in Bristol, two in Nottingham, one in Peterborough) for which results

were available for car distance travelled.

The statistical significance of changes in travel behaviour is not indicated in

Table 5 because it was not always reported in project evaluation reports and, even

where it was reported, there are some concerns about the basis for the tests used. This

issue is discussed further in section 5.1.

Overall, the results show a degree of consistency in car use reduction across

projects with increased walking often noted. There is no indication that higher

reductions in car use occurred when PTP was accompanied by other measures (such

as in Brighton Year 1 project, two Nottingham projects and Worcester Phase 1 and

Phase 2.2 projects), or that that differences in car use reduction are systematically

related to any other characteristics of projects such as scale or location.

It is perhaps surprising that walking increases substantially more than other

modes, especially considering that some areas included specific attempts to increase




18

cycling (Brighton, Darlington) and bus use (Nottingham) and considering that bus

information was the most popular material requested by project participants. It was

thought that the reported increase in walking might actually be due to an increase in

walking associated with bus trips (respondents were asked to record all modes used

on each trip) but, since such trips have been coded by Socialdata as bus trips, this

explanation can be ruled out.

The Socialdata project evaluations provide more detail on changes in travel

behavior: reductions in car travel did not vary consistently by time of day but were

larger for females than males; and increases in the number of trips within ‘own

neighbourhoods’ have been consistently reported in different projects (a finding

which is consistent with there having been a modal shift from car driver trips to

walking trips).

Cost-benefit analyses have not been conducted for the case study PTP

projects because, although results are available for mode share and car distance

travelled, the aggregate level data needed to estimate changes in indicators such as

travel time, bus patronage, accidents, air pollution and physical activity, have

generally not been available. In the Worcester Phase 1 project, bus patronage data

shows that the bus services promoted by the PTP project experienced higher growth

than other bus services in the town. For the Nottingham Wollaton project, where

travel surveys were not conducted, bus patronage was found to increase 5.5 per cent

on a year-on-year basis six months after the project.

Most of the after surveys were carried out about six to eight months after the

PTP project with no information available about outcomes over a longer period.

After surveys were repeated in two projects in Bristol. Results for Bristol

Bishopsworth suggest an increasing effect of PTP over time with decreases in the




19

share of journeys made by car of 1 per cent after six months and 5 per cent after 20

months. However, there was a counterintuitive increase in bus mode share of 5 per

cent after six months (before the bus upgrade). Results for Bristol Bishopston suggest

impacts were sustained over time (3 percentage point decrease in car driver trip mode

share after 3 months and 4 percentage point decrease after 9 months). It is to be noted

that the relatively small sample sizes of 300-400 in these two projects mean there is

more uncertainty associated with these results than with others reported in Table 5.

4.3 Process evaluation

As well as investigating outcomes (intermediate and final), the research study

examined the process of implementing the projects and how this was received by the

target populations. This was carried out through interviews with project stakeholders

(clients, delivery agents) and documentary analysis. The overall findings are reported

in chapters five and six of Parker et al. (2007). Most of these relate to what can be

done to achieve an effective and cost-efficient delivery process - although the link to

project outcomes is uncertain. The following paragraphs provide a flavour of the

findings.

Experience has shown that careful media planning is required to avoid

negative press coverage which can affect the reception of project by residents. For

example, in Brighton the project engaged a local authority press officer to

communicate with the media. Pre-testing of project branding and materials with

members of the target population has been found to be valuable to ensure they will

be positively perceived. In Peterborough the PTP brand slogan was changed from

Your travelchoice to My travelchoice as a result of pre-testing.




20

To achieve high participation rates, it is necessary for there to be attempts at

contacting households at different times of day and week with written invitations to

participate left with households if these are unsuccessful. Doorstep contact with

target population households can be supplemented by contact via community events.

In the Bristol Easton project the PTP project had a presence at seven community

events during the period of the project. Database management systems are required

to keep records on household participation - enabling efficient and timely processing

to be undertaken of materials requests and thereby increasing the public’s confidence

in the project.

Practitioners claim that PTP has more impact in areas with high car

ownership and use, local facilities, good public transport accessibility and local

recognition of congestion and environmental problems. They also suggest that

impacts per capita are greater when the scale of projects is larger. However, the

outcomes reported previously do not necessarily provide evidence of differential

impacts according to these criteria.

There has been some engagement of bus companies in PTP projects involving

contribution of staff time to the steering of the project and the open sharing of

information to assist the preparation of project materials (bus timetables, route maps

and incentives). Some bus companies have emphasised that they have other effective

methods to market their services. There has been limited evidence of engagement

with other sectors in PTP projects. For example, to engage the health sector it has

been suggested that projects will need to incorporate health considerations at the

outset (e.g. health-related project objectives, funding for health sector project

participation).




21

In Brighton there was an in-depth investigation into the way in which PTP

influenced travel behaviour. An independent researcher was commissioned to

conduct interviews with 25 project participants, exploring their perceptions of the

project, the type of behaviour they changed and the factors inhibiting and motivating

change. A sample was selected which deliberately over-represented participants that

had changed their behaviour. It was found that travel advisors were perceived as

friendly and helpful, changes to travel behaviour were often facilitated by other

changes in people’s lives and local travel was most often changed. Even where no

change in behaviour occurred there was reported to be increased deliberation and

pre-disposition to change.

5. Evaluation methodology

5.1 An assessment of the case study project evaluations

Criteria for achieving robust evaluations were identified in section 2 and the

reported project evaluations are now assessed against these. It is important to note

that limitations in the resources available for project evaluations will have restricted

what was possible and the comments that follow should not be interpreted as

criticisms of the project evaluators.

Reliability

The size of the samples in the project surveys has been insufficient to provide

the prospect for changes in travel behaviour of 5% to be statistically significant at the

95% level. Stopher and Greaves (2007) estimate the need for sample sizes of




22

between 3,000 and 4,000 households to estimate changes in distance travelled by car

with a sampling error of 5% for one day travel diary data and independent before and

after samples. Using matched (panel) samples and longer periods of travel reporting

(e.g. diaries for two or more days) would decrease sample size requirements, but

introduce greater risk of sampling bias from survey non-response and survey drop-

off.

Meta-analysis of results from several studies potentially allows greater

confidence to be placed in the results. Möser and Bamberg (2008) used this approach

to find an effect size (for PTP and related projects) with high precision (effect size of

0.11, with 95% confidence that it is between 0.08 and 0.13). The current study did

not attempt this because of the relatively small number of projects and the known

heterogeneity of the projects (and their evaluations). However, it is considered very

likely that a pooled result for the achieved reductions in car travel would have been

statistically significant.

Some of the PTP project evaluation reports did include statistical tests on the

estimated change in the mode share of car driver trips (calculated based on total trips

reported in the project area surveys). These have usually shown a level of statistical

significance with at least a 90% level of confidence (a 99% level was reported for the

Peterborough Phase 1 project). However, there is ambiguity about the appropriate

sample size to be used in this statistical test with the number of trips in the survey not

strictly appropriate, since multiple trips are made by the same person and trips are

therefore not independent. A more appropriate basis for statistical tests would be the

change in the distance by car travelled per person where it would be clear that the

sample size to use in statistical tests would be the number of persons in the surveys.

Unfortunately, access to the survey data was not available to conduct such a test.




23

Construct validity

The possibility of response bias in self-reported travel surveys should be

checked by obtaining data on travel outcomes using other methods which are not at

risk from this response bias (such as aggregate-level travel counts). This could not be

done in the current study but is discussed further in section 5.2.

Internal validity

The two-group pre- and post- test design used in most of the project

evaluations made it possible to estimate the counterfactual and thus reduce the threat

to internal validity. The estimate of the counterfactual can have a large bearing on the

result obtained (sometimes tending to inflate any measured effect of the PTP and

sometimes tending to deflate it).

The adequacy of the estimates of the counterfactual may be called into

question by the fact that no decrease in car use was estimated in the comparison areas

in those case studies (Darlington, Lancashire, Peterborough and Worcester) in which

area-wide initiatives to promote non-car modes were taking place in the designated

comparison areas as well as in the PTP areas. It should be noted that the estimated

counterfactuals in these case studies were based on limited data and that more data

will be available on town-wide travel trends in the Sustainable Travel Towns at the

end of the five year implementation period. Although confidence in the estimated

counterfactuals would be increased if the survey results could be corroborated by

other data (e.g. aggregate-level data for car travel) for the comparison areas, this has

rarely been undertaken.




24

These comments indicate the importance of careful selection of the

comparison area so that it is as comparable as possible to the project area (an issue

discussed by Stopher et al., 2009). It should also be noted that where data from valid

comparison areas is unavailable the counterfactual can be estimated by extrapolating

past trends or using modelled predictions of outcomes. Such methods were not used

in any of the project evaluations examined in this paper.

Conceptual validity

Gärling and Fujii (2006) have noted that past PTP evaluations have focused

on evaluating measures of travel behaviour effects rather than process which leaves

the question of conceptual validity unanswered. In the project evaluation reports

studied for this paper, travel behaviour results were not disaggregated in terms of

project participation (which, at the most basic level, would distinguish between

participants and non- participants), so it is not possible to test whether project

participation made a difference to outcomes. The main reason offered by evaluators

for not having attempted this analysis is that information on project participation was

not available from the surveys (which had avoided making reference to the PTP

project in case this influences survey response). However, it should be possible in

future projects to link survey respondents to project management databases in order

to be able to identify this.

Information could also be sought in travel surveys on social-psychological

measures (e.g. beliefs, attitudes, social norms, habits). The collection of this type of

information was not attempted in the travel surveys reviewed because of the risks of

over burdening respondents by asking for too much information and of including

subjective questions that might influence self-reported behaviour. However,




25

collecting information on such measures would offer the possibility of better

understanding why behaviour changes. This would be most effective with the use of

matched (panel) samples where social-psychological and behaviour measures are

tracked over time for the same individuals. Social-psychological influences on

behavioual change can also be investigated through qualitative research of the type

conducted in the Brighton project. It is noted though that where the primary purpose

of an evaluation is to assess overall value for money then it is reasonable to

concentrate on estimating the average travel behaviour effects across a target area,

rather than seeking to explain individual behaviour change.

Longevity

The longevity of project outcomes has not been demonstrated. Long-term

monitoring was not available in the project evaluations reviewed in this study, but is

to be collected for Darlington, Peterborough and Worcester to evaluate the impacts

of initiatives in these towns at the end of the Sustainable Travel Towns programme.

External validity

A main threat to external validity is from survey samples which are not

representative of the population from which they are drawn. Random probability

samples were sought in the PTP evaluations. In the Socialdata travel surveys the

electoral register and commercially available telephone databases were used for the

sampling frame. Unfortunately, a significant minority of households are excluded

from these lists (the Small-user Postcode Address File, which lists all addresses

receiving less than 25 items of post per day, is the recommended sampling frame for

surveying households in the UK).




26

A low response rate to a survey presents a serious risk to achieving a random

probability sample and efforts should be made to avoid this. Survey response rates

have been particularly high for Socialdata surveys (between 59 and 74 per cent) and

this helps to reduce survey non-response bias. The pre-analysis weighting of survey

samples that was conducted by Socialdata will have further addressed this. (It would

also be possible to compare characteristics of survey respondents to area population

characteristics (e.g. using census data) and to weight sample on this basis.) It has to

be acknowledged that, despite the steps taken, some households will not respond to

surveys for reasons confounded with the survey objectives (e.g. low level of

mobility, negative attitude towards public transport). This emphasises the desirability

of seeking corroboration of survey results from other data.

External validity is also threatened if areas selected for PTP projects are not

typical of the wider sample population of areas. The consistency of results for project

areas with different characteristics provides some assurance that this threat is low. As

results become available for more of the project areas within the Sustainable Travel

Towns of Darlington, Peterborough and Worcester (where large portions of the

towns are being targeted for PTP), there will be stronger evidence on whether the

results are generalisable.

Independence

Independent evaluators were commissioned in Brighton and Darlington and it

is interesting to note that lower reductions in car use were reported in Brighton and

Darlington than in other projects. It is apparent that local authority clients have

difficulty justifying the large costs required to commission independent evaluators (it

has been suggested that evaluation costs could exceed delivery costs if they did this).




27

Reporting and retrieval biases

The risk of reporting and retrieval biases has been highlighted by Möser and

Bamberg (2008). These are not judged to present an issue in this study, as it proved

possible to obtain results for all the projects that were completed in the selected case

study areas.

5.2 Recommendations for future evaluation

Evaluations need to carefully consider the purpose of evaluation and the

information required (as discussed in section 2) before deciding on data collection

methods or their design. Table 6 sets out how different data collection methods can

be used to meet different information requirements.

Table 6

Secondary data on travel and traffic has generally been under-utilised in past

PTP evaluations. Aggregate-level, secondary data can be used to obtain information

on overall transport and travel outcomes and thus has the benefit of being directly

informative on outcomes such as congestion, emissions and public transport

patronage. It can also corroborate findings from household travel surveys. The costs

of obtaining aggregate-level data are minimal if it is being collected anyway for other

purposes.

Richardson et al. (2005b) have used a variety of aggregate-level data to study

the impact of a PTP project in Melbourne. The scale of the project, which involved




28

30,000 households, lent itself to aggregate-level analysis as it would be expected that

changes in travel behaviour would be evident over a wide geographic area. The study

obtained traffic count data and public transport ticket and satisfaction data and

enabled coherent analysis to be made of project outcomes and for results to be

compared to those from a before and after self-report travel survey (Richardson et al.,

2005a). Although only initial results have so far been reported, this study is a

valuable reference for the employment of aggregate-level data in evaluating PTP

projects.

The challenge in using aggregate-level data is to identify data that is

geographically relevant for the project area and any comparison area. Aggregate-

level data may be available for a period of time before the project, so that assessment

of the impact of PTP can be made in the context of a longer term trend. Aggregate-

level data may also be in the form of continuously collected data over time (in the

case of automatic traffic counts, for example) which enables outcomes to be

monitored during and after the project and enables longevity of outcomes to be

assessed.

For obtaining individual-level travel behaviour outcomes it is suggested that

self-report travel surveys will continue to be required. Odometer readings have been

requested from survey respondents in some Australian project evaluations

(Richardson et al., 2005a; and Seethaler and Rose, 2009) and, if they can be obtained

for long duration periods (of a week or longer), they may allow much smaller sample

sizes to be used in self-report surveys than would otherwise be necessary. However,

the experience of Stopher et al. (2009) with odometer readings indicates that there

are concerns with reporting accuracy and such concerns lead Bonsall (2009b) to




29

suggest that odometer readings should ideally be taken by survey staff or derived

from annual records kept for official purposes.

The use of portable GPS devices has potential to provide more accurate and

precise data on travel than possible from self reporting of trips. Their effectiveness

and feasibility is still being researched (Bonsall et al., 2006; and Stopher et al., 2009)

and it is not yet clear whether sample self-selection and conditioning effects will be

important obstacles.

Chapter 4 of DfT (2008) provides detailed guidance on the design of self-

report travel surveys for evaluating PTP projects and makes recommendations on

sampling strategy, sample size, survey design and survey administration. Although

Möser and Bamberg (2008) suggest a randomised control group post- test design

should be used in future VTBC measure evaluations, the use of the two-group pre-

and post- test design is perhaps more appropriate because it is unlikely to be feasible

in practice to randomly select project and control areas and the inevitable doubts

about the equivalence of control area and project area will make it difficult to avoid

inferring project outcomes which are really due only to differences between the

project and control areas.

It is difficult to decide whether to use independent or matched (panel)

samples; the latter offer much lower sample size requirements (perhaps only one

quarter of the sample size according to Stopher and Greaves, 2007), but may

introduce potentially serious survey drop-off and conditioning biases. Matched

samples offer particular advantages for the analysis of individual behavioural change

(e.g. what were the characteristics of individuals who decreased car use?).

The inclusion of social-psychological questions in travel surveys can help

explain behaviour change, but their use is only recommended when there is a




30

theoretical basis for their influence on behavioural outcomes (see for example

Thøgersen, 2009) and when it is known in advance how they will be analysed. If

included, they need to be carefully selected and limited in number. Respondents need

to be directed to complete details of their trip making before answering these

questions. An alternative research method is to conduct in-depth, qualitative travel

behaviour research using depth interviews or focus groups. By selecting a range of

participants in the research, variations in responses made to PTP can be sought

amongst the project area population. This approach has the advantage of allowing the

factors that influence behaviour to emerge freely during discussion (rather than

requiring them to have been identified in advance), but it has the disadvantage of not

allowing statistical generalizations to be made about the project area population.

Customer feedback surveys can be employed to find out about materials

requested, their perceived usefulness, and the levels of satisfaction generated.

Comments volunteered by participants during such feedback can also provide

insights on how PTP influences travel behavior. Such comments are inescapably

selective and unrepresentative of the project area population but can be useful in

triangulating data from other sources.

The number of participants in a project and the nature of their participation

(e.g. materials requested) can be examined using the project database. The

effectiveness of the overall project management process (or particular parts of it) can

be examined through commissioning an independent evaluation of the process,

including interviews with those involved in the process and analysis of

documentation. More detailed guidance on conducting process evaluation is given in

section 5 of NERA (2006).




31

To be able to address the outstanding evaluation methodological concerns,

future PTP projects will require a higher level of resource than has typically been

available. This is unlikely to be available in all projects (see, for example, Cohen,

2009) and evaluation resources may need to be concentrated on specific

‘demonstration’ projects. Such an approach has been adopted in Melbourne

(Richardson, et al., 2005a). With sufficient resources independent evaluators can be

commissioned and more exhaustive evaluation carried out using multiple research

methods (self-report travel surveys, aggregate-level travel data, qualitative travel

behaviour research, process evaluation) which will allow triangulation of findings.

It is recommended that caution is applied in future in commissioning self-

report travel surveys because, without large sample sizes, it is difficult to obtain

statistically significant results. With further development of expertise in the use of

aggregate-level data in evaluation, it is suggested a greater emphasis is placed on this

in PTP project evaluation.

6. Conclusions

The new evaluation results available from this research study are consistent

with the findings from previous PTP projects in the UK and elsewhere. A reduction

of car driver trips is found in all the reviewed projects with a project average

reduction of eleven per cent. Also, consistent with past findings for the UK, the mode

of travel which experiences the most substantial increase is walking with modest

increases reported for cycling and public transport.

A main aim of the study was to identify factors that influence the

effectiveness of PTP projects. Results have been similar across projects, regardless of




32

the characteristics of the project area and of the other initiatives occurring alongside

the PTP. Results have not been disaggregated according to the type of participation

that individuals have had in projects and so it is not possible to identify how this

influenced outcomes. Despite the consistency of outcomes reported, there are some

concerns about evaluation methodology and reporting which leave open the

possibility that the reported outcomes might be systematically biased.

Möser and Bamberg (2008) identify concerns with research design, survey

sample representativeness, sample sizes and reporting of statistical significance of

individual project evaluations. The projects reviewed here used a quasi-experimental

research design which addresses, as far as is practically possible, the criticisms raised

by Möser and Bamberg (2008). The sampling approaches were generally good but

the sample sizes were often inadequate. It seems that the problem of inadequate

sample sizes can only be overcome by increasing the budget to facilitate larger

samples, collecting data over a longer period - possibly via use of odometer readings

or portable GPS devices, or employing a matched (panel) design – provided that

survey drop-out and conditioning biases can be overcome..

Several of the case studies failed to provide sufficient detail on the statistical

significance of the reported results. It is clearly important that future evaluations use

and report appropriate statistical tests. A remaining concern with survey-based

evaluations is that of possible response biases. Bonsall (2009b) identifies the

particular problem of social-desirability bias in self-reported behavioural data.

Möser and Bamberg (2008) were concerned about reporting and retrieval biases in

obtaining project evaluations but this was not an issue in this study. However, the

independence of evaluators from project delivery was an important issue and it is

essential that independent evaluators are commissioned in future projects.




33

To address the concerns related to survey-based evaluations identified above,

it is a priority in future project evaluations to seek aggregate-level travel data to

supplement survey-based travel behaviour data. Another priority is to utilize

qualitative research methods, or include social-psychological measures within travel

surveys, to increase understanding of how design elements of PTP projects influence

behavioural outcomes.

Improved evaluations will better enable the value for money to be assessed of

PTP relative to other investment options. This is a priority for the DfT who have

stated that “Subject to value for money tests, we will be increasing our investment in

initiatives like these significantly in coming years, and will publish more detailed

plans shortly” (DfT, 2007, p. 49). Such evaluations would, however, place a

significant financial burden on PTP projects and hence are unlikely to be possible for

all future projects. Whilst making the recommendation for improved evaluations, it is

also recommended that careful reflection is made on what is being sought from PTP

projects. They can be aimed at achieving individual travel behavior changes (e.g.

modal shift), broader transport outcomes (e.g. reduction in congestion) or wider

benefits (e.g. physical fitness, carbon emissions reduction). They can be pursued on

their own, or as complementary initiatives alongside physical/economic changes.

They can also be used for creating support for other measures which might be needed

in the long-term, empowering communities to make their own changes, or, they may

simply be seen as projects that are popular in their own right with the public and

politicians.




34

References

AGO, 2005. Evaluation of TravelSmart projects in the ACT, South Australia, Queensland,

Victoria and Western Australia: 2001 – 2005. Australian Greenhouse Office,

Commonwealth of Australia, Canberra. Available online (last accessed on 28 July

2008) at: http://www.travelsmart.gov.au/publications/evaluation-2005.html

Bonsall, P., Wolf, J., Holroyd, S., 2006. Review of the Potential Role of ‘New

Technologies’ in the National Travel Survey. Report to Department for Transport.

Available online (last accessed on 17 August 2008) at:

http://www.dft.gov.uk/pgr/statistics/datatablespublications/personal/methodology/nts

reports/

Bonsall, P.W., 2009a. What is so special about surveys designed to investigate the

sustainability of travel vehaviour? In: Bonnel, P., Lee-Gosselin, M., Zmud, J.,

Madre, J-L., (Eds.), Transport Survey Methods: Keeping Up with a Changing

World. Emerald Press, Bingley, UK, (Chapter 3).

Bonsall, P., 2009b. Do we know whether personal travel planning really works? Transport

Policy (this issue).

Cairns, S., Sloman, L., Newson, C., Anable, J., Kirkride, A., Goodwin, P., 2004. Smarter

Choices – Changing the Way We Travel. Report to Department for Transport.

Available online (last accessed on 28 July 2008) at:

http://www.dft.gov.uk/pgr/sustainable/smarterchoices/ctwwt/

CLG, 2008. National Indicators for Local Authorities and Local Authority Partnerships:

Handbook of Definitions. Department for Communities and Local Government,

London. Available online (last accessed on 28 July 2008) at:

http://www.travelsmart.gov.au/publications/evaluation-2005.html

http://www.dft.gov.uk/pgr/statistics/datatablespublications/personal/methodology/ntsreports

http://www.dft.gov.uk/pgr/statistics/datatablespublications/personal/methodology/ntsreports

http://www.dft.gov.uk/pgr/sustainable/smarterchoices/ctwwt




35

http://www.communities.gov.uk/publications/localgovernment/finalnationalindicator

s

Cohen, T., 2009. Evaluating personal travel planning: If it is prohibitively expensive to get a

robust answer then what should we do? Transport Policy (this issue).

DfT, 2004. Full Guidance on Local Transport Plans: Second Edition. Department for

Transport, London. Available online (last accessed on 28 July 2008) at:

http://www.dft.gov.uk/pgr/regional/ltp/guidance/fltp

DfT, 2005. Personalised Travel Planning: Evaluation of Fourteen Pilots Part Funded by the

DfT. Department for Transport, London. Available online (last accessed on 28 July

2008) at http://www.dft.gov.uk/pgr/sustainable/travelplans/ptp/

DfT, 2007. Towards a Sustainable Transport System. Department for Transport, London.

Available online (last accessed on 29 July 2008) at

http://www.dft.gov.uk/about/strategy/transportstrategy/

DfT, 2008. Making Personal Travel Planning Work: Practitioners’ Guide. Department for

Transport, London. Available online (last accessed on 1 January 2009) at:

http://www.dft.gov.uk/pgr/sustainable/travelplans/ptp/

Gärling, T., Fujii, S., 2006. Travel behavior modification: Theories, methods and programs.

Resource paper for the IATBR Conference, Kyoto, Japan, August 16-20.

Möser, G., Bamberg, S., 2008. The effectiveness of soft transport policy measures: A critical

assessment and meta-analysis of empirical evidence, Journal of Environmental

Psychology 28, 10-26.

NERA, 2006. The Evaluation of Major Local Authority Transport Projects: A Guide for

DfT. NERA, MVA, and David Simmonds Consultancy. Available online (last

accessed on 2 January 2009) at:

http://www.dft.gov.uk/pgr/evaluation/evaluationguidance

http://www.communities.gov.uk/publications/localgovernment/finalnationalindicators

http://www.communities.gov.uk/publications/localgovernment/finalnationalindicators

http://www.dft.gov.uk/pgr/regional/ltp/guidance/fltp

http://www.dft.gov.uk/pgr/sustainable/travelplans/ptp

http://www.dft.gov.uk/about/strategy/transportstrategy


http://www.dft.gov.uk/pgr/evaluation/evaluationguidance




36

O’Fallon, C., Sullivan, C., 2003. Personalised marketing – improving evaluation. Revised

version of paper presented at 26th

Australasian Transport Research Forum, 1-3

October 2003, Wellington, New Zealand. Available online (last accessed on 2

January 2009) at:

http://www.pinnacleresearch.co.nz/research/personalised_marketing.htm

Parker, J, Harris, L., Chatterjee, K., Armitage, R., Cleary, J., Goodwin, P., 2007. Making

Personal Travel Planning Work: Research Report. Integrated Transport Planning Ltd.

Report to Department for Transport. Available online (last accessed on 28 July 2008)

at: http://www.dft.gov.uk/pgr/sustainable/travelplans/ptp/

Richardson, A.J., Davis, M.B., Harbutt, P.L., 2005a. Using before and after household

surveys to evaluate a TravelSmart program. Paper presented at 28th

Australasian

Transport Research Forum, 28-30 September 2005, Sydney, Australia. Available

online (last accessed on 2 January 2009) at: http://www.patrec.org/atrf/index.php

Richardson, A.J., Roddis, S., Arblaster, D., Attwood, D., Newman, J., 2005b. The role of

trend analysis in the evaluation of a TravelSmart program. Paper presented at 28th

Australasian Transport Research Forum, 28-30 September 2005, Sydney, Australia.

Available online (last accessed on 2 January 2009) at:

http://www.patrec.org/atrf/index.php

Roth, M., Ker, I., James, B., Brög, W., Ashton-Graham, C., Ryle, J., Goulias, K.,

Richardson, E., 2003. A dialogue on Individualised Marketing: Addressing

misperceptions. Paper presented at 26th

Australasian Transport Research Forum, 1-3

October 2003, Wellington, New Zealand. Available online (last accessed on 2

January 2009) at: http://www.patrec.org/atrf/index.php

Seethaler, R., Rose, G., 2009. Using odometer readings to assess VKT changes associated

with a voluntary travel behaviour change program. Transport Policy (this issue).

http://www.pinnacleresearch.co.nz/research/personalised_marketing.htm








37

Stopher, P.R., Bullock, P., 2003. Travel behaviour modification: A critical appraisal. Paper

presented at 26th

Australasian Transport Research Forum, 1-3 October 2003,

Wellington, New Zealand. Available online (last accessed on 2 January 2009) at:


Stopher, P.R., 2005. Voluntary travel behavior change. In: Button, K.J., Hensher, D.A.,

(Eds.), Handbook of Transport Strategy, Policy and Institutions. Elsevier,

Amsterdam, pp. 561-579.

Stopher, P.R., Greaves, S.P., 2007. Guidelines for samplers: measuring a change in

behaviour from before and after surveys, Transportation 34, 1-16.

Stopher, P.R., Cifford, E., Swann, N., Zhang, Y., 2009. Evaluating voluntary travel

behaviour change – suggested guidelines and case studies. Transport Policy (this

issue).

Taniguchi, A., Suzuki, H., Fujii, S., 2007. Mobility management in Japan: Its development

and meta-analysis of travel feedback programs (TFPs). In: CD-ROM Proceedings of

86th

Transportation Research Board Annual Meeting 2007.

Thøgersen, J., 2009. Promoting public transport as a subscription service: Effects of a free

month travel card. Transport Policy (this issue).





38

Table 1

Performance indicators monitored by English local authorities

Transport objectives Wider objectives

o Local public transport

journeys

o Cycling trips

o Journey to school mode share

o Area-wide road traffic mileage

o Peak-period traffic flows to

urban centres

o Congestion (vehicle delay)

o Air quality

o Road accident casualties

o Bus services running on time

o Bus satisfaction

o Accessibility to jobs, services

and facilities by public

transport, walking and cycling

o Carbon emissions per capita

o Obesity in primary school

children

o Mortality rate from all

circulatory diseases

o Self-reported health/overall

well-being

o Small businesses showing

employment growth

o General satisfcation with local

area

o Children’s satisfaction with

parks and play areas




39

Table 2

Case study project details

Local

authority

area

Project Delivery

agent(s)

Project

Date

Main objective(s) Scale

(house-

holds)

Description of area Other initiatives in

project area

Brighton Year 1 (of 3)

Portslade & Hangleton

BHCC /

SDG Jun-Oct 06

Long-term modal shift

(towards meeting LTP targets for cycling, walking and

congestion)

10,000 West edge of city (average

income)

Cycling infrastructure

improvements

Bristol Bishopsworth Sustrans Sep 02 Offer PTP to 2,500 people in each area

Modal shift

(Assess relative impact on bus mode share of PTP

with and without bus upgrade)

1,192 South-west edge of city

(low income) Bus service upgrade

Bristol Hartcliffe Sustrans Oct-Nov 03 1,200 South edge of city (low

income)

Bus service upgrade

(one year after PTP)

Bristol Bishopston Sustrans Apr-Jun 03

Offer PTP to 5,000 people

Modal shift

(Assess short and medium-term modal shift)

2,254 North (inner-) outer area of

city (high income, students) Bus service upgrade

Bristol Southville Sustrans Apr-May

05

Offer PTP to 5,000 people

Modal shift 2,535

South inner area of city

(average income) None

Bristol Easton SDG Jan-Mar 06

2,000 people participating

10% decrease in car trips

200 car drivers decrease veh. emissions

30 new members of car club

3,469 East inner area of city (low

income, ethnic diversity) None

Bristol Clifton/Cotham SDG Aug-Nov

06

2,000 people participating

10% decrease in car trips

Improvements in air quality through changes to

driving practice

5,629 North-west (inner-) outer

(high income, students) None

Darlington Phase 1 (of 3) SDG May-Aug

05 Contribution to LTP targets for modal shift, traffic

growth and school trips by car

11,500 North and north-west outer

areas of town STT initiatives

Darlington Phase 2 (of 3) SDG May-Aug

06 12,000

Inner west end and north-

east outer area of town

STT and CDT

initiatives

Lancashire South Ribble Sustrans May-Jul 06

Offer PTP to 25,000 h’holds in each area

10,500 Outlying areas to south and

east of Preston

EU CIVITAS

initiatives

Lancashire Bare, Torrisholme and

Skerton Sustrans Autumn 06 8,500

Outer areas of Lancaster

and Morecambe

EU CIVITAS

initiatives




40

London Kingston (New

Malden)

TfL /

SDG

May-Aug

06

Modal shift (and congestion and carbon emssions

decreases and health improvements)

22,299 Outer areas of London (high

car use, good public

transport access and local

facilities)

None London Harringey (Alexandra

Palace)

TfL /

SDG

Sep-Nov

06 31,324

London Sutton (Worcester

Park)

TfL /

SDG Sep-Oct 06 6,545

Nottingham Lady Bay Sustrans Sep-Nov

03 Offer PTP to 1,000 people

Modal shift

353 South (inner-) outer area of

city (high income) Bus service upgrade

Nottingham Meadows Sustrans Sep-Nov

03 538

South inner area of city

(low income, ethnic

diversity)

Bus service upgrade

Nottingham Wollaton City Card

Pilot NCC 2006 Increase in patronage on Number 30 bus service 2,130

Mix of neighbourhoods on

bus route west of city centre None

Peterborough Phase 1 (of 5) Sustrans Sep-Dec 05 Offer PTP to 30,000 h’holds (all phases)

69% of population informed about travel choices by

2008/09

Contribution to LTP targets for modal shift

Broader outcomes (health, etc.) (no specific targets)

6,500 North-east of town

STT initiatives Peterborough Phase 2 (of 5) Sustrans Apr-Jul 06 6,103 North of town

Peterborough Phase 3 (of 5) Sustrans Sep-Dec 06 5,653 North-west of town

Worcester Phase 1 (of 3) Sustrans Sep-Dec 05 Offer PTP to 60% of town’s citizens

Contribution to STT targets for modal shift

Improved perception of bus services

6,300 North-east of town STT initiatives and

new bus service

Worcester Phase 2.1 (of 3) Sustrans Apr-Aug

06 4,775 East of town STT initiatives

Worcester Phase 2.2 (of 3) Sustrans Sep-Nov

06 3,829 South of town

STT initiatives and

new bus service

Key: BHCC = Brighton & Hove City Council, CDT = Cycle Demonstration Town, EU = European Union, LTP = Local Transport Plan, NCC = Nottingham City Council, SDG

= Steer Davies Gleave, STT = Sustainable Travel Town, TfL = Transport for London




41

Table 3

Case study project populations and survey samples

Project Delivery

agent(s)

Evaluator Before : after

survey dates

(format:

mm/yy)

Project area populations/survey samples

Ta

rget

ed

ho

use

hold

s

Co

nta

cted

ho

use

hold

s

Pa

rtic

ipa

tin

g h

ou

seh

old

s

Bef

ore

su

rvey

sa

mp

le

size

(p

erso

ns)

Aft

er s

urv

ey s

am

ple

siz

e

(per

son

s)

Brighton Year 1 BHCC/SDG SRA 11/06 : IA 10000 9800 3150 1968 2000

Bristol: Bishopsworth Sustrans Socialdata 5/02 : 3/03, 5/04 1192 867 473 378 363*, **

Bristol: Hartcliffe Sustrans Socialdata 3/03 :/ 5/04 1200 959 619 374 332

Bristol: Bishopston Sustrans Socialdata 3/03 : 9/03, 2/04 2254 1761 1221 456 478*, **

Bristol: Southville Sustrans Socialdata 4/05 : 6/05 2535 2053 1664 2053 779***

Bristol: Easton SDG SDG 12/05 : 12/06 3469 2112 1538 54 32***

Bristol: Clifton/Cotham SDG SDG 7/06 : IA 5629 2917 1667 IA IA

Darlington Phase 1 SDG Socialdata 11/04 : 11/05 11591 7779 4579 1237 1156

Darlington Phase 2 SDG Socialdata 11/04 : 11/06 10744 7599 5205 1246 1224

Lancashire: South Ribble Sustrans Socialdata 4/06 : 3/07 10713 9813 6907 IA 634

Lancashire: Bare Sustrans Socialdata 4/06 : IA 8500 6941 5265 IA 561

London: Kingston TfL/SDG Synovate IA : IA 22299 15386 7503 IA IA

London: Harringey TfL/SDG Synovate IA : IA 31324 19122 10722 IA IA




42

Project Delivery

agent(s)

Evaluator Before : after

survey dates

(format:

mm/yy)

Project area populations/survey samples

Ta

rget

ed

ho

use

hold

s

Co

nta

cted

ho

use

hold

s

Pa

rtic

ipa

tin

g h

ou

seh

old

s

Bef

ore

su

rvey

sa

mp

le

size

(p

erso

ns)

Aft

er s

urv

ey s

am

ple

siz

e

(per

son

s)

London: Sutton TfL/SDG Synovate IA : IA 70000 IA IA IA IA

Nottingham Lady Bay Sustrans Socialdata 7/03 : 5/04 353 232 172 601 450***

Nottingham Meadows Sustrans Socialdata 7/03 : 5/04 538 285 188 535 402***

Nottingham: Wollaton NCC NCC NS 2130 2130 2130 NS NS

Peterborough Phase 1 Sustrans Socialdata 10/04 : 3/06 6500 5336 3267 1073 1228

Peterborough Phase 2 Sustrans Socialdata 10/04 : IA 6103 4981 2530 IA 1111

Peterborough Phase 3 Sustrans Socialdata 10/04 : IA 5653 4573 2611 IA IA

Worcester Phase 1 Sustrans Socialdata 10/04: 3/06 6300 5247 3210 978 962

Worcester Phase 2.1 Sustrans Socialdata 10/04 : 4/07 4775 3913 2127 IA 784

Worcester Phase 2.2 Sustrans Socialdata 10/04 : 4/07 3829 3133 1762 IA 805

Key: BHC = Brighton & Hove City Council, IA – information awaited and not available at time of study, NCC = Nottingham City Council, NS – no survey, SDG = Steer Davies Gleave, SRA =

Social Research Associates, TfL = Transport for London, * After survey sample size applies to second after survey. Sample size for first survey was similar, ** Partial matched sample, *** Matched

(panel) samples.




43

Table 4

Information and incentives/gifts distributed across case studies

Total distributed

(% of households participating)

Minimum Average Maximum

Information

Bus timetables 10% 85% 157%

Cycle information 11% 47% 102%

Walking information 11% 39% 77%

Other bus information 5% 35% 100%

Cycle maps / routes 9% 32% 73%

Local Travel Maps 8% 31% 68%

Walking maps 9% 30% 89%

Rail timetables 9% 23% 36%

Bus stop timetables 9% 23% 28%

City Travel Maps 14% 19% 23%

Greener / Eco driving 5% 15% 23%

Pledge card 1% 14% 21%

Oystercard information 5% 8% 12%

Personal Journey Plans (all modes) 1% 6% 25%

Tube information 0.3% 4% 8%

Home visits 1% 1% 2%

Incentives / gifts

Shopping Bag 32% 40% 49%

Other (retail vouchers etc) 15% 24% 33%

Free bus tickets (day tickets) 0.1% 21% 100%

FM Radio 7% 17% 24%

Alarm Clock 6% 13% 19%

Pen 5% 10% 16%

Mug 6% 10% 15%

Pedometer 0.3% 0.4% 0.7%

Cycle computer 0.3% 0.4% 0.4%

Note: Figures for each item only apply to projects where they were offered




44

Table 5

Changes in travel behaviour attributed to PTP in case study projects

Project Months

after

project

of

survey

Other

initiatives

in project

area

Initiatives

in

compar-

ison area

Trend in

comparison

area

Change in mode share (%) Change in

car travel

(%)

Wa

lkin

g

Cy

clin

g

Pu

bli

c

tra

nsp

ort

Ca

r d

riv

er

Ca

r

pa

ssen

ger

Ca

r d

riv

er

trip

s

Ca

r d

riv

er

dis

tan

ce

Brighton Year

1† IA

Cycling

measures

No

compar-ison area

No

comparison area

14→19

+5

1→2

+1 IA

54→50

-4 IA -6 IA

Bristol:

Bishopsworth 6 None None

Walking

increase

23→21

-2

0→0

0

8→13

+5

44→43

-1

25→22

-3 NR NR

Bristol:

Bishopsworth 20

Bus

upgrade

Bus

upgrade PT increase

20→23

+3

0→1

+1

11→12

+1

45→40

-5

23→23

0 -9 -8

Bristol:

Hartcliffe 6

Bus

upgrade

Bus

upgrade PT increase

20→23

+3

1→2

+1

12→13

+1

45→40

-5

21→21

0 -12 -11

Bristol:

Bishopston 3

Bus

upgrade

Bus

upgrade Stable

38→40

+2

4→5

+1

5→7

+2

37→34

-3

15→14

-1 -10 NR

Bristol:

Bishopston 9

Bus

upgrade

Bus

upgrade Stable

38→40

+2

4→5

+1

5→7

+2

38→34

-4

14→14

0 -11 -13

Bristol:

Southville 2 None

No

compar-ison area

No

comparison area

NA NA NA NA NA -10° NA

Darlington

Phase 1 6

STT

initiatives

STT

initiatives Stable

23→23

0

1→1

0

12→14

+2

45→44

-1

19→18

-1 -4 NR

Lancashire:

South Ribble† 8

CIVITAS

initiatives

CIVITAS

initiatives

Car driver

increase

14→20

+6

1→2

+1

6→7

+1

57→50

-7

21→21

0 -13 IA

Nottingham

Lady Bay 7

Bus

upgrade

None

except city trends

Walking

decrease,

car driver

increase

25→30

+5

7→8

+1

8→10

+2

41→36

-5

19→16

-3 -12 -12

Nottingham

Meadows 7

Bus

upgrade

None

except

city trends

Walk

decrease,

car driver increase

32→34

+2

3→4

+1

25→26

+1

29→26

-3

11→10

-1 -10 -10

Peterborough

Phase 1 6

STT

initiatives

STT

initiatives Stable

20→24

+4

5→6

+1

7→8

+1

43→38

-5

25→24

-1 -13 -15

Worcester

Phase 1 6

STT

initiatives

and bus upgrade

STT

initiatives Stable NA NA NA NA NA -12° NA

Worcester

Phase 2.1† 6

STT

initiatives

STT


Worcester

Phase 2.2† 6

STT

initiatives

and bus

upgrade

STT





45

Key: IA – information awaited and not available at time of study, NA – not available due to nature of surveys and

evaluation performed, NR – not reported, † detailed evaluation reporting has not been available in time for this

report but headline evaluation results have been provided, ° inferred based on changes in average weekly

frequency of use of transport modes




46

Table 6

Data collection methods and how they address information requirements

Data collection

method

Information

requirement

Secondary

travel/traffic

data

Self-report

travel survey

Qualitative

travel

behaviour

research

Customer

feedback

survey

Project

database

Interviews &

documentary

analysis

Overall transport

and travel outcomes

Can be

inferred

Individual travel

behaviour outcomes

Can help

corroborate

travel survey

But not

representative

results

Individual social-

psychological

measures

If relevant

questions

included

Can provide

selective

insights

Project participant

satisfaction

Project participation

Project management

effectiveness

Documents

A Comparative Evluation of Large-Scale Personal Travel