Ministero della Giustizia

Direzione Generale di Statistica

Customer Satisfaction Surveys conducted in the pilot courts of Turin and Catania

Tbilisi, February 10, 2012

Italian Ministry of JusticeOrganization, Human Resources and Services Department Department of Statistics

Background

The idea of conducting a satisfaction survey in Italy aimed to Court users takes its stands from the guidelines prepared by the Quality Working Group of the CEPEJ.

The idea was proposed and enthusiastically accepted by the Heads of the Courts of Turin and Catania.

How satisfied or dissatisfied are you with

your personal experience of the court

today?

3

Objectives

Main objectives of the survey are:

To evaluate the overall satisfaction level of Justice amongst thefinal users

To establish Key Performance Indicators (KPIs) to be tracked throughout time

To pinpoint the areas of improvement and (propose to) take action accordingly

To analyze the satisfaction amongst specific sub-samples (e.g. males/ females, age classes, …)

To plot Performance vs Importance diagramMaintain Focus

Low priority

Improve

Performance

Importance

4

Working group

A key success factor towards the success of such initiative is to involve as much as possible the stakeholders of the world-justice.

Thus, the working group to design and conduct the survey was composed by: Ministry of Justice representatives(and as members of the CEPEJ)StatisticiansHeads of the Courts of Appealand TribunalsJudges and LawyersProfessors of the University(Law, Politics and Economic Sciences)

5

Media Attention

The initiative was reported on main national newspapers and local news broadcast

Courts under survey

Last year we conducted the survey in Turin and Catania

Depending on the goodness of the results this kind of survey might be extended to other courts across the Country

In Turin the survey was conducted only with users while in Catania it was done also with other two targets: Lawyers and Court staff

Turin

Catania

Members of the CEPEJ were responsible for managing and coordinating the initiative in both cities

7

Constraints and methodology

Since this Satisfaction Survey is carried out without a budget, we have decided to:maximize the use of internal resources and free external resources

guarantee the quality of the project through involvement of internal professional statisticians and university professors

select a group of around 25 students per city to conduct the interviews

8

Additional slides are reported in the second part of this presentation on how to conduct interviews

The sample

Interview Citizens visiting the Courts of Turin and Catania (preferably people who have already got a contact with the service provided) during February-March 2011.

Preferably a “random” sample of 600 usersper Court

Alternatively a sample composed of atleast 380 users per Court

Statistical note within an unlimited population:A sample of 600 users guarantees that the standard error is lower than 4%A sample of 380 users guarantees that the standard error is lower than 5%

9

The target

The target has been defined according to the following criteria:

Criteria of inclusion

• Parties• Witnesses• Interpreters, experts• Relatives of the Parties, of

witnesses, etc.

• Lawyers• Judges, Prosecutors and staff

of the Court• Policemen, Bailiffs, etc.

Criteria of exclusion *

(*) In Catania two specific satisfaction surveys aimed at Lawyers and Court staff have been carried out separately.

10

The questionnaire

The questionnaire has been designed according to the CEPEJ guidelines, taking into account specific needs of the Courts in Turin and Catania. For this reason we have used two slightly different questionnaires.

Both questionnaires are divided into 3 different sections:• Screening questions

• Main Questions. Satisfaction of: Court premises, organization, clean environment Punctuality of hearings, length of proceedings Judge professionalism, competence, cordiality Information points within the Court

and web-sites Court location and ways of transport

• Demographic Questions

11

Promoting the initiative

Before the interviews started, many posters promoting this initiative were stuck to the walls of the Courts premises.

This simple action had a great contribution on the respondents’ willingness to take part into this survey.

Additional publicity has been obtained through:•Lawyers in Turin and Catania: they were asked to talk about this initiative among their clients. •Judges: they were all formally informed about this initiative.

Catania has requested the CEPEJ to provide some gadgets to be distributed to the people who participated to the survey

12

Data acquisition and results

The data collection is performed through an on-line application.

At the end of the day all questionnaires are being loaded-up into a central database by the interviewers using an on-line form.

The results (charts and graphs) are available in real-time to the members of the working group.

Statistical Note:

In order to ensure good quality standards, a selection of random paper questionnaires will be double-checked against the online data to verify that the questionnaires have been transmitted correctly.

13

Project timing

14

The whole project lasted about 8 months, from very beginning to presentation of results

The results: Overall Satisfaction

Overall satisfaction with the court experience

TURIN

CATANIA

Note: Scores are calculated applying a weighted mean. Weights range from 1 (Very Dissatisfied) to 5 (Very satisfied).

Score:

Score:

Satisfaction vs Importance diagram

TURIN

Satisfaction vs Importance diagram

CATANIA

Key suggestions on how to conduct interviews and collect the data

Tbilisi, February 10, 2012

Italian Ministry of JusticeOrganization, Human Resources and Services Department Department of Statistics

Interviewers training

Prior to the fieldwork a number of motivational sessions have been carried out.

Moreover, in order to ensure good quality standards, interviewers were carefully briefed on the following:

Phases of the interview (approach, questionnaire, closure)

Behavior and good conduct

What to do and what to avoid

Questionnaire flow (screening, main and demographic questions)

19

Behavior and good conduct

Kindness, discretion, self-confidence, inspiring trust

Neutral behavior, attention to non-verbal communication

Read the questions in the exact order they are reported into the questionnaire

Availability to read questions and all possible answers even more than once

Take note of all the comments coming from the interviewed person

20

INTRODUCTION Good morning / afternoon, my name is ___________________ and I’m here today because the Ministry of Justice is currently conducting a survey aimed at court users such as parties, witnesses, experts, interpreters, etcThe Ministry of Justice is interested in your own opinion and in your experience of this court in order to improve the overall quality of the service.This is a statistical survey and therefore your answers will be handled with care. At the end of this survey data will be analyzed in the utmost anonymous way. Can I ask you a few questions? Strict confidentiality of your answers is guaranteed. This survey is anonymous.

The questionnaire - Introduction by the interviewer

Introduction by the interviewer

Screening questions Core questionnaire Demographic questions Closing

Conducting one interview takes approximately 15 – 18 minutes(it is important to notice the person about the actual duration of the interview)

The questionnaire - structure

Data has been collected using free tools on the web, called Google Docs

Google Docs is a collection of web-based documents that let users edit the same file at the same time so you always have the latest version

In particular, an online form was designed in order to collect the data submitted by the interviewers

The main advantages of using an online form for collecting data are:

Quality of data is guaranteed Answers can be accessed in real-time Data is ready for analysis at any time The workgroup can easily track the

interviewing process

SW utilized for data collection – the tool

TURIN

CATANIA

Designing the questionnaire

Collecting the data

Summary report

Exporting data into Excel for further analysis.

11

22

33

44

A summary report with charts and graphs is available in real-time.

A form (i.e. questionnaire) can be easily designed using single and multiple choice questions, scales, grids, text boxes etc.

Once the form is online, interviewers can load their questionnaires via web. The data submitted is stored in an online spreadsheet document.

Data can be exported into excel format (amongst the others) and eventually into spss format for further analysis.

SW utilized for data collection – the steps

PIAZZA GIOVANNI VERGA, Palazzo di Giustizia (1)

VIA FRANCESCO CRISPI, ex Pretura (2)

VIA VERONA, Sezione Lavoro (3)

1

3

2

Sede Uffici Numerosità campionaria

Palazzo di Giustizia Corte d’Appello e Tribunale

382

Via Crispi (ex Pretura) Tribunale 164

Via Verona (Sezione Lavoro) Tribunale 54

Taking care of all details in the organization

The workgroup must consider all the details related to the organization, in this slide, for example, we report the map shown to students to make them aware of the courts premises and location.

In our experience, in a working morning, e.g. from 9.00 to 14.00, an interviewer can conduct up to 6 interviews

Interviewers were organized into sub-groups of 6-8 people with an agreed calendar over a period of 6-8 weeks

It is important to recommend the interviewers that the choice of the person to be interviewed must be absolutely random

At the end of each day, each interviewer was requested to report filled-in questionnaires into the on-line application

Timing and other practical informations

Ministero della Giustizia

Direzione Generale di Statistica