QA Conference Sarajevo © Mag. FH Bernadette Frech04.04.2008 Quality Assurance Strenghts at BiH Universities Monitoring Results of the Tempus Project „Strenghtening

© Mag.FH Bernadette Frech 04.04.2008QA Conference Sarajevo

Quality Assurance Strenghts at BiH Universities

Monitoring Results of the Tempus Project „Strenghtening Quality Assurance at

Universities in BiH“

Mag.FH Bernadette FrechResearcher & Lecturer

FH JOANNEUM

Department of International Magement

8020 Graz, Austria

[email protected]


„When the winds of change blow

some build walls

and some build windmills“

(BMBWK, 2004)


QA in the emerging EHEA

„the primary responsibility for QA

lies with each institution itself and

this provides the basis for real

accountability of the academic system within the NQF“

Berlin Ministeral Meeting (2003)

Bergen (2005)

European standards and

guidelines


QA in Higher Education – A BiH Perspective


Evaluation Objectives

1. Compliance of the results of the project “Strengthening Quality Assurance in Bosnia-Herzegovina” with output indicators defined in the JEP_19074_2004 proposal

2. Putting special emphasis on the sustainability of university quality assurance initiatives on university and faculty level after the project duration.

3. Accordance of quality assurance initiatives with the Bologna process, and more precisely with the ENQA standards for quality assurance.


Evaluation Framework

n Evaluation Object

EvaluationPlace

Interview Partners

1 University of East Sarajevo

Sarajevo / Rp. Srpska (24/01/2008)

Prof.dr. Zoran Ljuboje (Vice-rector) Mr. Nenad Markovic (QA coordinator)

2 University of Zenica

Zenica (24/01/2008)

Prof. dr. Darko Petkovic (Vice-rector) Mr. Ibrahim Plancic (QA coordinator)

3 University of Mostar

Mostar (25/01/2008)

Mrs. Prof. Drazena Tomic, Phd (Vice-rector)Mr. Jadranko Batista (QA coordinator)

4 University Dzemal Bijedic of Mostar

Mostar (25/01/2008)

Prof. dr. Fuad Catovic (Rector) Prof.dr. Ibro Popic (Vice-rector) Mr. Alim Abazonic (QA coordinator)


Interviews based on evaluation indicators

Evaluation Model

Member/sboard of directors

QA coordinator

External evaluator

Lejla SahacicSUS BiH


Evaluation Matrix


Evaluation ModelPillars of evaluation:• Overall quality assurance system and strategy

formulations at university and faculty level as well as organizational integration of the QA office;

• Capacity building through the provision of equipment, space and the employment of a QA expert functioning as QA coordinator on university level;

• Implementation of best practice seen based on seminar inputs received at the Karl-Franzens University Graz, Austria and the University of Ljubljana, Slovenia;

• Transfer of knowledge from QA coordinators to student unions, academic as well as administrative staff;

• Cooperation with stakeholders such as students, teaching and administrative staff, public authorities, and industry;

• Dissemination and valorization of project results through up-dated weblinks, brochures and open day seminars


QASQuality Assurance System and Organizational Integration

ID Indicator

QAS_1 Organizational integration of QA offices

QAS_2 Development and integration of QA system

QAS_3 Responsibilities of management, faculties, staff, stakeholders

Overall quality assurance system and strategy formulations at university and faculty level as well as organizational integration of the QA office


QASQuality Assurance System and Organizational Integration

Points of strengths Points for future development

High QA awareness

Preparation of a sustainable QA strategy

Establishment of internal student evaluations

Highly motivated staff

Phase of change

Striking roots in formal structures and procedures as well as in people’s minds

National legislation

Integration of all measures in a holistic QA strategy.


CB - Capacity Building

ID Indicator

CB_1 Provision of equipment and space

CB_2 Employment of QA Coordinator

Level of capacity building through the provision of equipment, space and the employment of a QA expert functioning as QA coordinator.


CB - Capacity Building


project aimed at establishing a fully equipped QA office and to employ an expert responsible for the coordination of QA activities

This capacity-building measure was fully implemented by all assessed universities.

Project objectives for capacity building are fully met.

Continuous provision of necessary resources in terms of space and time even after the project duration.


BP – Implementation of Best Practice

ID Indicator

BP_1 Implementation of best practices from seminar in Graz

BP_2 Implementation of best practice examples from seminar in

Ljubljana

Degree of implementation of best practice examples based on inputs received at seminars at Karl-Franzens University Graz, Austria and the

University of Ljubljana, Slovenia.


BP – Implementation of Best Practice


The project objectives regarding the provision of best practice seminars in Graz and Ljubljana were exceeded.

Besides active participation many universities have implemented QA measures at their universities.

The project enabled the establishment of national and international networks of QA

This exceptional openness for new QA ideas and ambition for constant enhancement at BiH universities should be retained in the future.


TK – Transfer of Knowledge

ID Indicator

TK_1 Transfer of knowledge to students

TK_2 Transfer of knowledge to academic and administrative staff

Extend knowledge is transferred from QA coordinators and top management to student unions, academic as well as administrative staff.


TK – Transfer of Knowledge


The project being in line with ENQA standards strives for constant transfer of knowledge from top management and QA coordinators to university staff and students.

Installed meeting and communication cultures, open-door policies as well as constant knowledge transfer to stakeholders at all levels underline the fulfilment of the project’s endeavour.

Structured information flows

Regular and documented form of knowledge management


CS – Corporation with Stakeholders

ID Indicator

CS_1 Cooperation with students

CS_2 Cooperation with faculty members

CS_3 Cooperation with public authorities

CS_4 Cooperation with industry

Degree of cooperation with stakeholders such as students, teaching and administrative staff, public authorities, and industry


CS – Corporations with Stakeholders


All universities have a regular involvement of university staff and students

Some universities actively involve ministries and the labour market when developing curricula etc.

Contact to ministries and public authorities

Labour market perspectives


DIS – Dissemination and Valorization

ID Indicator

DIS_1 Updating web-links, printing and sending brochures

DIS_2 QA events (open days or other)

DIS_3 Future activities

Degree of cooperation with stakeholders such as students, teaching and administrative staff, public authorities, and industry


DIS – Dissemination and Valorization


Project results were disseminated through manifold channels

Brochures, updated weblinks, seminars, press coverage, and direct communication

Continuous dissemination and exploitation of project results

Updating of website links


Overall Project Evaluation

012345QAS

CB

BP

TK

CS

DIS

BiH Universities (mean value)

A


Best Practice Examples within BiH

• University of East Sarajevo: Being involved in seven QA projects the university has reached already a high degree of awareness for QA among important stakeholders. The university is prepared for installing a university wide QA system. The constant communication flow between top management, QA coordinator, teaching staff and students ensure high support at all levels for planned QA measures.

• University of Zenica: The university of Zenica can be regarded as a best practice example within the BiH tertiary education sector as well as on a European level. Quality assurance is an integral part of the university’s daily work agenda and profound initiatives are already established which ensure sustainability and professionalism of QA in the future.


Best Practice Examples within BiH

• University of Mostar: One central point of strengths at the University of Mostar is the consciousness of importance of QA for future university development. Though the implementation of novel initiatives such as a student information system for assessments they strive to become a university known for its innovative measures in QA.

• University Dzemal Bijedic of Mostar : The main point of strengths of the University of Dzemal Bijedic of Mostar is their inspiring and enthusiastic team striving for high quality in the implementation of sustainable internal as well as external QA activities. The university is fully devoted towards the development, implementation, monitoring and dissemination of QA measures.


Visions

• University of East Sarajevo: … to have a QA system with defined criteria by 2010 with its standards and methods of monitoring and quality data, comprising also methods for data gathering and analysis…

• University of Zenica:… to become the first quality certified university in Bosnia and Herzegovina…


Visions

• University of Mostar: … to have high quality graduates … driven by an integrated university where a QA strategy is connected to the university strategy …

• University Dzemal Bijedic of Mostar:… to become a fully integrated university, to implement the planned QA strategy and to formally institutionalize QA …


Thank you for your attention!


Contact:

Mag.FH Bernadette FrechResearcher & Lecturer

FH JOANNEUM

Department of International Management

8020 Graz, Austria

[email protected]


QA in Higher Education – An Austrian Perspective


Higher Education Landscape

Higher education

in Austria

22 Publicuniversities

19 Universities of applied sciences

Post-secondary colleges

Academies


Diploma Programmes

EUA 2007


QA at Universities and U. of Applied Sciences

UNIVERSITIES FACHHOCHSCHULEN

Students 187,559 17,500

Number of studies 180 145

Curricula - Curricula commission

- Qualification profile

contents

Scientific approach

- Job-relatedness

- Internship

Scientific-practice approach

Evaluation and quality assurance

- Quality management system

- Evaluation of whole performance spectrum

- Evaluation of teaching quality by students

- Quality management system

- Scientific evaluations

- Evaluations of teaching quality by students

- Admission examinations


Quality Defined at Universities

• Goals of university studies (AHStg):

– The development of science and young scientific talents

– The scientific vocational education

– The education through science

– The further education of the graduates


Quality Defined at Fachhochschulen

• Goals at Fachhochschulen:

– A hands-on higher education

– The supply of skills and capabilities in order to fulfill specific work tasks

– To support the work flexibility of students


FH JOANNEUM

• One of Austrian‘s leading Universities of Applied Sciences

• Founded 1995

• ~ 3500 students


QA at FH JOANNEUM

Intern ExternResearch

Teaching and Studying


QA at FH JOANNEUM

• Internal Quality Assurance: Teaching and Studying

• Annual internal audit by Department of Personnel, Legal Services, and Internal Auditing

• Annual knowledge bilance

• Student entrance examinations

• Student evaluations / semester

• Teacher‘s evaluations / academic year

• Employabity evaluations / project based

• Programme evaluation /academic year (management board, advisory committee)

• QM Handbook


QA at FH JOANNEUM

• Internal Quality Assurance: Research– Process Handbook

– Current objective: EN ISO 9001 for R&D

– QM Handbook


CuStomer

Deviations from the plannedcourse

improvements

AssignmentContractExamination otthe contract

Project approvalend reviewhanding out

Project approvalend reviewhanding out

Project prolog & Pre-OfferMarketingacquisition

Project epilogMeetingreflexionCustomer care

Project epilogMeetingreflexionCustomer care

Offer / proposalstage

project specificationfirst planningcalculation

formulation ofoffer/proposal

ImplementationDetailed planning

processingcontrolling

Valid process instructions

PZB „formulating an offer to contracting“ PZB „settlement of the project“

Storage of project documents

PZB…Process descriptionVA…..process instructionAA…..working instruction

+ diverse VA´s and AA´s for R&D relevant acticities

CuStomer


QA at FH JOANNEUM

External Quality Assurance– Accrediation procedures (Public Authorities, FHK, experts from

industry and science)

– External Auditing by AQA (Austrian Quality Assurance System)

– Financiers (National government, provinces, municipalities)

– Labour market evaluations / project based

– Alumni evaluations / project based


Accrediation Procedure

• New degree programs need to apply for accreditation (§12 FH Studies Act)

• First Accreditation (based on quality standards) granted for a maximum of 5 years (FH Council)

• Re-accreditation before the end of the approval period by FH Council


Requirements for Application

• Severe analysis of the needs of the labour market (demand, acceptance, coherence..)

– Detailed and consistent description of the vocational fields of activity the program aims at

– A qualification profile of the future graduates; they shall be able to fulfill the tasks of the professional field in a manner which is adequate for higher education level

• Formal Requirements (curriculum, examinations regulations, didactic concept)

– A curriculum; i.e. the required qualification profile must be reflected in the curriculum

– Duration of study: Bachelor: 3 years ( = 180 credits); Master: 1 to 2 years ( = 60 to120 credits); Diploma: 4 to 5 years ( = 240 to 300 credits)

• Committments concerning resources and financing


A system-oriented perspective of quality

• Compulsary entrance procedure for students limited number of students per year and program are accepted

– Submission of a workmap (for design & art programs)– Written examination, personal interviews

• Evaluation of single courses Per term; students represenatives make proposals; improvement measures

• Annual „internal evaluation“ of all programs by the management and members of the advisory board

5% 5%10%

25%50%

5%

work experience

form of application

grade point average

personal interview

written examination

advanced training

QA in the organisation


Labour market evaluations - TAQC

• TAQC: Creating an Observatory on Europe-wide TAQC (Transparency of Academic Qualifications and Competences) for Catching the MOLE (Mobility of Labor in Europe) and Filling in the GAP (Generalized Academic Policy)

• (DIN EN ISO 9001 certified)


Labour market evaluations - TAQCSample

July to September 2006 a total of 863 valid questionnaires were collected via judgemental sampling technique in 8 European countries

General structure of the respondents

6170

81 75 75 7561

70

53

75

92

75

0

20

40

60

80

100

Austria Bulgaria France Lithuania Romania Slovakia

Graduates Employers


• Employers‘ Suggestions for Improvement

Labour market evaluations - TAQC

knowing company

structures

Learn customer related thinking

learn more about the real job life and not about theory

learning by doing

More autonomy (elimination of fixed curricula,

more specialization)


Labour market evaluations - MISLEM

• To advance a model or step-by-step procedure for assessing the match between skills and competencies developed by educational institutions and those required by labour market

• To refine and test meta-level indicators (focused on the labour market linkage) for assessing quality in universities and vocational training institutions (VETs)

• To combine expertise from vocational training instutitions and universities in the area of quality assessment

• To foster collaboration among different types of partners from education, industry and government

• To promote transnational cooperation which will lead to the sharing of best practices and potentially greater convergence and comparability across countries‘ educational quality systems


MISLEM - Final study sampleCOUNTRY/Groups Graduates

(N=596)

Employers

(N=304)

Romania n= 102

80 univ., 12 FHs,

10 VETs

n= 97

86 univ., 10 FHs,

no VETs

Austria n= 106

48 univ., 37 FHs,

21 VETs

n= 56

29 univ., 23 FHs,

4 VETs

England n= 178

146 univ., 32 VETs

n= 60

51 univ., 9 VETs

Slovenia n= 210

64 univ., 100 FHs,

46 VETs

n= 91

40 univ., 34 FHs,

17 VETs


Meta-Level indicators (%)Graduates

Business Knowledge

Graduates

Business Competencies

Employers

Business Knowledge

Employers

Business

Competencies

Indicator #1

(valuable)

41.56 70.50 41.32 78.09

Indicator #2

(relevant)

48.72 82.11 51.16 82.15

Indicator #3

(capable)

54.83 63.40 38.63 53.03

Indicator #4

(gap)

15.05 20.69 18.94 17.00


Evaluation

• To guarantee the fulfillment of the educational mission and to ensure the quality of the vocational training evaluations are made (evaluation regulations EvalVO 5/2004).

• standardized process– Internal evaluation (internal report) basis for ->– External evaluation by a review team– Final inspection and assessment by the FH Council

– Two types of evaluation• Institutional evaluation: strategy & organisation; quality

management; program portfolio, R&D, resources, infrastructure…, internationalization, cooperations..

• Degree program related evaluation: educational goals, students, organisation and Q-assurance, personal, infrastructure, R&D


External Auditing

Documents

QA Conference Sarajevo © Mag. FH Bernadette Frech04.04.2008 Quality Assurance Strenghts at BiH Universities Monitoring Results of the Tempus Project „Strenghtening