Health promotion for disadvantaged youth presentation_IF… · Presentation Max Saurug. Alfred Strassegger. Ancona, 28.09.2011. The Project HEALTH25 has received funding from the

The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.

Health promotion for disadvantaged youthWP 3 | Evaluation

Quality Management Concept

PresentationPresentation

Max SaurugAlfred Strassegger

Ancona, 28.09.2011


Health 25 | Introducing IFA Steiermark

Our OrganisationOur OrganisationIFA Steiermark, seated in Graz, Austria (5 employees) was founded in 1989.

IFA is a research institution working for the common good of society. Our organisation is aimed at

assisting politically responsible people, decision makers in public corporations and private companies,

persons involved in projects, special interest groups and so on.

Research = Accompaniment + Optimisation



Our ServicesOur Services

Evaluations

Short term analyses

Consultation and development

Our work includes questions concerning labour market and social policy, regional development,

vocational education for companies and personnel and the integration of people with special needs into

the labour market.



Our ExperienceOur ExperienceOver many years, IFA has acquired a vast wealth of experience in

the planning, conception and

undertaking of practical scientific investigations and in accompanying consultations about programs and

measures in the social field.

Through numerous all‐embracing national and transnational research and development projects

conducted throughout Europe (more than 100), IFA is able to fall

back on a wealth of both national and

international experience.

To learn more: www.ifa‐steiermark.at


Health 25 | WP 3: Evaluation | 1. Quality Management Concept

Quality Management Quality Management ConceptConceptThis quality management concept will give you detailed description of the quality management and an

overview of the evaluation strategies in regard to the quality management of

the project process,

the project product (outputs and outcomes) and

the project impact.

In respect of the assessment of the quality of the Health 25‐project the quality indicator identification

process and its outcomes should be started as one of the next steps.

from http://ec.europa.eu/eahc/management/Fact_sheet_2010_05.html


Health 25 | WP 3: Evaluation | Quality Management Concept

The objectivesThe objectives# 1 Literature review and survey

# 2

Development of the health training and intervention programme for

the target group

# 3

Development of pedagogical‐psychological concept for the target

group

# 4 Synthesis of the holistic health training and intervention programme

for 16‐20 year old people without a job and not in school education

# 5 Pilot implementation


Health 25 | WP 3: Evaluation | Quality Management Concept | Overview

Criteria Based EvaluationCriteria Based EvaluationFor a successful venture the quality management approach has to rely on specific quality criteria. This

concept relies on the following characteristics:

Scientifically an methodologically correct

Transparent

Flexible

Realistic

Possible to control

Targeted



Description of the modelDescription of the modelThe quality management concept is based on a triangulate evaluation approach. In the frame of this

approach there are three different dimensions to focus on during

the evaluation:

Project process

Project products

Project impact

Only by taking into account these three dimensions the project quality can be estimated in a holistic

approach. The model contains so called Critical Success Factors (CSF’s) for all three dimensions, which are

partly given in advance, but mostly tailored to the project.



DescriptionDescription of of thethe modelmodel

This triangulate Quality Management Approach implies that all three quality dimensions

interdependently interfere, which allows to ensure quality through a holistic approach. Furthermore, this

method is absolutely transparent and the evaluation data is accessible anytime to all the project partners.

The quality management method estimates the actual status quo by covering the targets set; it

recognizes deviances and suggests possible steering measures, which allows to keep the project on track

from the start and to reach the project quality proposed.



QM of the project processQM of the project process

One of the components of this quality management strategy involves the monitoring of the project in

terms of its processes. In this context the following processes will be observed:

Communication among partners

Communication between the contractor and the partners

The level of information and awareness of the partners towards the contents of the project (also goals,

schedule and budget)

The degree of involvement of the different partners in the project activities




The reports serves as monitoring tool for the coordinator as its part of the quality Management and

Evaluation Strategy. The questionnaire developed focuses on the following areas:

Processes during the meeting

Organisation of the meeting

Suggestions, recommendations and personal comments

Project progress reports

Implementation of the peer evaluation system




Criteria collection for the product evaluation

Methods and instruments for the evaluation


Health 25 | WP 3: Evaluation | Quality Management Concept | QM of the project products

Criteria collection (CSFCriteria collection (CSF’’s)s)

In the process of ensuring the quality of the Health 25‐project all participating partners will be assigned

the identification of applicable quality indicators and subsequently asked to rate these items. The partner

responsible for the WP3 – Evaluation of the project –

will collect and analyse the data provided by all

organisations involved in the Health 25‐project.

Stage 1 | Initial Phase –

Definition of the CSF’s

Stage 2 | Rating the elaborated items

Stage 3 | Processing the contributions


Health 25 | WP 3: Evaluation | Quality Management Concept | QM of the project products

Evaluation methods and instrumentsEvaluation methods and instruments

The proposed instruments used for the evaluation of the Health 25‐project are based on the experience

of the evaluator in the area of quality management. In order to assess the individual CSF’s of the Health

25‐project it is recommended to use

(mainly) questionnaires and

guided Interviews (if reasonable and necessary)

as methods most suitable for the evaluation purposes.

The use of questionnaires along guided interviews within a partner meeting is intended to produce a

combination of qualitative and quantitative information which should support the evaluation of the

Health 25‐project in

respect of the work process and the product itself.


WP 3: Evaluation of „Health 25“ | The Quality Management Concept | Overview

QM of the project impactQM of the project impact

Similar to the evaluation of the product quality, a method of impact evaluation is recommended; where in

some case partly standardised CSF’s may be useable. These CSF’s will be developed similar to those in the

product quality involving all partners and stakeholders.

It is suggested to evaluate the impact of the product on representatives of the target groups in the

partner countries at the end of the project; the Health 25‐project can also be assessed in respect to its

short term impact. There could/should be mentioned measurement criteria like physical measurements

(baseline and at the end of the training programme) in relation to the Health Training and Intervention

programme as well as (set and fulfilled) expectations, knowledge, skills, attitudes and behaviour.


Thank you for your attention


Health 25 | WP 3: Evaluation

Evaluation Evaluation –– Basic approachBasic approach

Objective of the planned research project is the evaluation of the cooperation “Health 25”

on a

transnational level. The central tasks of the research assignment are to analyse the implementation

process and the attainment of the common set objectives.



Levels of analysis Levels of analysis and key questionsand key questions

Level 1:

This concerns outputs of work packages and the ”Health 25”

project as a whole: Specifying tasks

and duties ‐

What are the central common aims of ”Health 25”

? What is each member is in charge of?

How do the partners describe their roles within the transnational partnership? What are the

expectations? What are the duties?

Level 2:

Specifying the implementation steps: What working steps, methods and tools are to be used to

attain the common objectives?

Level 3:

Analysing the implementation process: What are the general conditions and factors that can

promote and impede the cooperation process?



Levels of analysis and key questionsLevels of analysis and key questions

Level 4:

Analysing goal‐attainment: To what extent were the set objectives (outputs/outcomes) attained?

Could the expected impact on the target groups be reached?

Level 5: SWOT analysis and analysis of ”lessons learned”: What are the strengths and the weaknesses of

this transnational cooperation? What is the potential for improvement? Are there any recommendations

for future partnerships?

In addition the project partners of the participating countries are to be assisted in evaluating their project

activities (within their work packages) themselves by putting at

their disposal research instruments

developed for this evaluation assignment. The results from this self‐assessment are to be made available

to the evaluator and incorporated into the evaluation as a whole.

Special task: A Quality Management Handbook has already been worked out (and also distributed to the

partners)



Duration, methods and working stepsDuration, methods and working steps

36 months (February 2011‐

January 2014)

Both qualitative and quantitative instruments are to be

used for this accompanying evaluation. Internet‐

based questionnaires for quantitative and partly standardised surveys are to be used.



Supporting the lead partner and reportingSupporting the lead partner and reporting

Periodic written reports about the progress of Health 25 are to be presented at every transnational

meeting. The evaluator is a member of the steering committee and

is to support the project lead partner

BBRZ Austria.

The evaluator is obliged to attend all transnational meetings of Health 25.

Representatives of the transnational project partnership and the lead partner are to be kept regularly up‐

to‐date orally and by periodic short reports after the meetings.

These reports ensure that the evaluation results can be used as a basis for project management.

Written

reports are to be submitted every six months as well as the

final results of the evaluation are to be drawn

up in a final research report.


Health 25 | WP 3: Evaluation | Working Steps

Presentation of the evaluation concept at the

transnational meeting in Ancona/It

(September 2011)

Presentation of the evaluation concept at the

transnational meeting in Ancona/It

(September 2011)

Development of questionnaires for self‐assessment of the

work packages of the project partners

(October/November 2011)

Development of questionnaires for self‐assessment of the

work packages of the project partners

(October/November 2011)

Development of an internet‐based questionnaire in order

to evaluate the progress of target attainment and the

implementation process of the project partners and the

project “Health 25”

as a whole (October/November 2011)

Development of an internet‐based questionnaire in order

to evaluate the progress of target attainment and the

implementation process of the project partners and the

project “Health 25”

as a whole (October/November 2011)

Participating in the 1st working group meetings of

WP 5, WP 6, WP 8

Participating in the 1st working group meetings of

WP 5, WP 6, WP 8

Drawing up the Final Report | January 2014

Presentation of the final results | January 2014

Drawing up the Final Report | January 2014

Presentation of the final results | January 2014

Participating observation at all transnational meetings; 6

qualitative interviews with leading representatives of the

partner organisations (Ongoing)

Participating observation at all transnational meetings; 6

qualitative interviews with leading representatives of the

partner organisations (Ongoing)

Evaluating and reporting the self‐assessment of the project

partners in the final evaluation report

Evaluating and reporting the self‐assessment of the project

partners in the final evaluation report

Periodic evaluation of the progress of the project including

short evaluation reports after every transnational meeting

(including first SWOT analyses)

Periodic evaluation of the progress of the project including

short evaluation reports after every transnational meeting

(including first SWOT analyses)

Elaboration the Quality Management Handbook (June

2011)

Elaboration the Quality Management Handbook (June

2011)


Thank you for your attention

Documents

Health promotion for disadvantaged youth presentation_IF… · Presentation Max Saurug. Alfred Strassegger. Ancona, 28.09.2011. The Project HEALTH25 has received funding from the