Embed Size (px)
Citation preview
!
Training Experts for Quality Assurance:
Lessons Learned
Mark!Frederiks,!Stephanie!Zwiessler,!Maria!E.!Weber!
!
Training Experts for
Quality Assurance:
Lessons Learned The results of
the E-TRAIN Pilot Trainings
Copyright*©*2012*by*the*European)Consortium)for)Accreditation)in)higher)education*
Author(s):*Mark*Frederiks,*Stephanie*Zwiessler,*Maria*E.*Weber*
ISBN/EAN:)978H94H90815H07H3)
All*rights*reserved.*This*information*may*be*used*freely*and*copied*for*nonHcommercial*
purposes,*provided*that*the*source*is*duly*acknowledged**
Additional*copies*of*this*publication*are*available*via*www.ecahe.eu.*
!
!
Table of content !
1.! Introduction,...............................................................................................................,9!
2.! Training,Practices,of,ECA,agencies,............................................................................,12!
2.1.! Starting!point:!when!is!someone!an!expert!and!a!good!panel!member!.............!13!
2.2.! What!can!and!should!be!accomplished!instead?!................................................!16!
3.! The,Pilot,Trainings,....................................................................................................,18!
3.1.! The!pilot!trainings!for!European!experts!.............................................................!18!3.1.1.! The!first!pilot!training!of!European!experts!...........................................!20!3.1.2.! The!second!pilot!training!of!European!experts!......................................!24!
3.2.! The!pilot!trainings!for!QA!staff!............................................................................!28!3.2.1.! The!first!pilot!training!of!QA!staff!...........................................................!28!3.2.2.! The!second!pilot!training!of!QA!staff!......................................................!30!
4.! Conclusions,..............................................................................................................,33!
4.1.! The!training!of!European!experts!........................................................................!33!
4.2.! The!training!of!QA!staff!(train!the!trainers)!.........................................................!35!
5.! Annexes,...................................................................................................................,37!
Annex!1:!Detailed!overview!of!training!characteristics!of!QA!agencies!
Annex!2:!Preparatory!material!
Annex!3:!Programme!of!the!training!
Annex!4:!Programme!of!the!first!pilot!training!of!QA!staff!
Annex!5:!Recommendations!(final!draft)!
Annex!6:!Programme!for!the!training!days!
!
!
!
”!
!“!
“European cooperation in higher
education calls for internationally
experienced and trained experts.”*
*
!
!
1. Introduction
!fundamental! part! of! external! quality! assurance! procedures! is! peer! review.! The!
connection! between! the! quality! of! the! procedure! and! the! ‘quality’! of! the! team!
members! is! apparent:! no! matter! how! competently! the! quality! standards,!
frameworks! and! procedures! are! composed! –! if! the! experts! are! not! conversant! with! the!
application!of! these! standards,! the!quality!of! the!procedure!might! suffer.! This! is!one! reason!
why!the!European,Standards,and,Guidelines,for,Quality,Assurance,(ESG)! insist!that!“(…)!the!
experts! undertaking! the! external! quality! assurance! activity! have! appropriate! skills.”!
Additionally! the! ESG! point! out! that!Quality! Assurance! Agencies! should! provide! ‘appropriate!
briefing!and!training!for!experts”.!!!
Within! the! higher! education! sector,! collaboration!
across! regional! and! national! borders! is! more! and!
more! the! rule! instead! of! the! exception! [! especially!
within!the!European!Higher!Education!Area!–!(EHEA),!
and! correspondingly! the! field! of!work! for! experts! is!
no! longer!restricted!to!their!own!national!system(s).!
This! fact! is! also! stressed! by! the! ESG’s! emphasis! on!
“the!use!of! international!experts”!and!would!lead!to!
the!conclusion!that!the!training!of!experts!should!not!
be! restricted! to! the! national! level,! either.! Drawing! on! international! expertise! can! help! to!
ensure! the! independence! of! the! panel! activity! from! the! influences! of! national! conflicts! of!
interests! and! foster! the! orientation! towards! European! and! international! developments! for!
standards!and!procedures.!!
A
No matter how competently the
quality standards, frameworks and
procedures are composed – if the
experts are not conversant with the
application of these standards, the
quality of the procedure might
suffer.
!
!10
!
Supported!by!EU,funds!the!European!Consortium!for!Accreditation!in!higher!education!(ECA)!
launched! in!October!2010!the!“European!Training!of!QA!Experts! (E[TRAIN)”!project.!ECA’s!E[
TRAIN!project!intends!to!strengthen!the!European!dimension!in!quality!assurance!by!making!it!
possible! for! Quality! Assurance! Agencies! to! identify! trained! and! experienced! experts! from!
countries!within!the!EHEA.!
The! overall, aim! of! E[TRAIN! was! to! facilitate! the! sharing, of, trained,, knowledgeable,, and,
internationally, experienced, experts! who! will! be! better! equipped! to! participate! in! quality!
assurance! (QA)! procedures,! to! the! benefit! of! higher! education! institution! institutions! under!
review.!Two!objectives!were!specified!in!order!to!ensure!that!the!overall!aim!of!E[Train!would!
be!met:!!
!
1. to!develop!a!European!training!programme!for!experts!in!QA!procedures!and!
2. to!develop!a!database!to!share!experts!among!Quality!Assurance!Agencies!in!Europe.!!
!
The! first!objective! is! the!most! important!one! for! this!handbook:! the!provision!of! training! for!
staff!member!of!quality! assurance!agencies!and! for! training!of!experts! (peers/auditors)!who!
will!be!part!of!panels!in!QA!procedures.!According!to!the!first!objective!two!sorts!of!trainings!
have!been!conducted!since!E[TRAIN!was!launched:!
!
1. training!for!quality!assurance!agency!staff!–!the!train[the!trainer[session;!and!!
2. pilot!training!sessions!for!experts!
!
The!focus!within!the!pilot!training!sessions!was!not!upon!specific!approaches!for!the!training!
of! panel! chairs! and! panel! secretaries.! Both! types! of! trainings! delivered! were! open! for!
participation! for! staff! of! ECA! member! agencies! but! as! well! for! experts! nominated! by! the!
agencies.!Participating!agencies!and!experts!involved!in!the!pilots!received!practical!hands[!on!
approaches!for!the!training!of!experts.!!
!
!11
The! first! pilot! session! for! staff! members! of! quality! assurance! agencies! was! held! on! 3rd!
November!2011!in!Vienna!and!the!second!was!held!on!10th!May!2012!in!Madrid.!The!first!pilot!
session!for!experts!was!held!on!6th!October!2011!in!The!Hague!and!the!second!one!on!23!and!
24!April! 2012! in!Vienna.! In! advance! to! these! sessions! the! agencies!were! asked! to!nominate!
experts!to!participate!in!these!pilot!sessions.!!
This!publication! includes! a! summary!of! the!experiences! and!achievements! acquired! through!
the!various!pilot!sessions!during!the!E[TRAIN!project.!!
Chapter! 2! provides! a! general! overview! on! the! actual! status! quo! of! training! practices!within!
agencies.!Chapter!3!summarizes!the!trainings!actually!delivered!within!the!E[TRAIN!projects.!!
Since! the! handbook! first! and! foremost! deals! with! the! aspect! of! how! to! train! experts!
undertaking!external!quality!procedures,!a!second!publication
!
!12
2. Training Practices of ECA agencies
he! following! information! is! derived! from!a! survey! amongst!QA!agencies!which!was!
conducted! in! order! to! explore! which! training! practices! exist! among! the! quality!
assurance! agencies! and! to! identify!what! the! agencies! consider! to! be! good! training!
practices.! It! served! as! an! important! input! and! inspiration! for! the! design! of! the! European!
training!course.!The!data!were!complemented!during!the!first!pilot!staff!training!in!November!
2011!where!staff!from!ten!different!European!agencies!elaborated!on!their!current!practices.!!
The!survey!has!revealed!common!elements!of!the!agencies’!training!practices.!!
Almost!all!the!agencies!carry!out!some!kind!of!training!experts.!The!focus!areas!of!the!trainings!
are!comparable!and! involve! the!quality!assurance!process,!national!higher!education!system!
and!practical!skills.!The!format!varies,!though.
Some! agencies! use! online! trainings,! some! arrange!
seminars,! some! have! trainings! which! lead! to!
certification,! and! some! combine! the! training! with! a!
meeting!right!before!the!site!visit.!!
When!looking!at!what!is!considered!to!be!good!practice,!
quite!a!few!agencies!find!it!appropriate!to!specialise!the!
training! according! to! the! background! or! role! of! the!
experts,! e.g.! distinguishing! between! newcomers! and!
experienced! experts,! or! training! secretaries! and! chairs!
separately, or! having! separate! student! trainings.! Furthermore,! the! practical! elements!
(interactive!elements!such!as!case!studies,!role[plays!etc.)!are!often!highlighted!and!this!is!very!
T
The national context is relevant for
the needs and means of quality
assurance and influence the
outlook of trainings delivered;
nevertheless the main focus of the
trainings is laid on QA standards &
criteria, the higher education
system, practical skills.
!
!13
much!appreciated!by!the!participating!experts.!This! is!one!reason!why!the!European!experts’!
training!attaches!some!importance!to!integrating!practical!elements!into!the!training.!!
Considering! these! findings! of! the! survey! amongst! agencies! and! taking! into! account! the!
European!dimension,!the!European!training!programme!of!experts!has!been!developed.!!
The!table!in!ANNEX!1!gives!a!more!detailed!overview!of!training!characteristics!of!QA!agencies.!!
2.1. Starting point: when is someone an expert and a good panel member
Before!starting!to!create!a!training!programme!for!international!experts!two!very!basic!
questions!had!to!be!answered:!!
1. What!constitutes!a!‘good’!expert?!And!can!the!skills,!qualities!and!competences!which!
might!be!needed!by!such!an!expert!be!acquired!through!training!at!all?!
2. What!is!a!good!panel!member?!Of!course!there!is!no!clear!or!easy!answer!to!this!
question:!the!‘quality’!of!an!expert!depends!on!a!variety!of!aspects.!!
!
In!the!following!it!is!attempted!to!depict!a!short!set!of!characteristics!of!external!experts.!Due!
to! the!experience! in! the!pilot! sessions!within! the!E[TRAIN!project,! the! following!outlines!are!
solely!focussing!on!so!called!‘academic!experts’.!Nevertheless!various!agencies’!procedures!do!
not!only! include! ‘academic!experts’!but!also!experts! from!other! relevant! stakeholder!groups!
(e.g.! students,! experts! with! relevant! professional! background).! Several! procedures! have! to!
take!into!account!the!different!target!groups!and!based!on!that!the!needs!of!information!and!
skills!of!panel!members.!!!
Hence,! if! the! expert! is! an! academic,! a! high! level! of! expert! knowledge! in! their! academic!
discipline!or!field!should!be!a!given!–!she/he!should!be!up[to[date!on!developments!within!her!
or!his!subject!area,!she/he!should!have!a!broad!scope!of!experience!in!research!and!teaching,!
she/he! should! have! experience! with! the! development! and! implementation! of! study!
programmes.!Where!the!expert! is!a!senior!HE!administrator! it! is!expected!that!he!or!she!has!
been! responsible! at! a! senior! level! in! the! institution! for! educational! policy! and! processes.!!
!
!14
She/he!should!have!been!involved!at!a!senior! level! in!their! institution!with!quality!assurance!
processes!and!assessment!processes.!
Student! experts! will! be! expected! to! be! familiar! with! educational! and/or! quality! assurance!
processes,! and!probably! to! have! (or! have!had)! some! representative! role! in! their! institution.!!
Quality!assurance!agencies!can!check!these!basic!requirements!for!working!in!an!expert!panel!
rather! easily! by! scrutinising! documents! such! as! the! curriculum! vitae,! list! of! publications,!
personal!statements,!of!the!expert!candidate!in!question.!!
There! are! ‘practical’! skills! which! competent! experts! possess.! These! comprise! for! example!
communication! techniques,! interview! techniques,! knowledge! about! how! to! write! reports!
evidence! based,! knowledge! about! relevant! national! frameworks,! awareness! for! the! specific!
role!in!the!group,!awareness!about!cultural!differences,!the!ability!to!be!good!team!workers!or!
the! ability! to! accurately! analyse! strengths! and! weaknesses! of! the! study! programme/!
institution!under!review.!
!
!
! !
What!experts!should!possess…!
• Awareness!of!cultural!differences!
• Ability!to!work!in!teams!and!groups!
• Subject!specific!knowledge!
• Analytical!skills!and!competences!
• Communicational!skills!&!interview!techniques!
• Knowledge!of!QA!standards!and!procedures!&!national!frameworks!
!
!
!15
That!is!why!training!sessions!normally!include!a!practical!component:!!
1. Testing!and!enhancing!these!skills!e.g.!by!using!(parts!of)!the!higher!education!
institutions’!self[evaluation!reports!to!be!analysed!in!case!studies.!!
2. Role[plays!in!which!interview[situations!with!representatives!of!the!institution!are!
simulated!and!thus!possible!(problematic)!situations!can!be!anticipated.!Role[plays!are!
also!a!good!training!method!to!focus!the!team!aspects!(various!roles,!diversity!
aspects).!!
!
Obviously!one!of!the!main!aims!of!(national)!training!courses!for!experts!is!to!guarantee!that!
the!experts!are!well!acquainted!with!the!standards!and!guidelines!to!be!applied!in!the!review!
process.!They!should!know!the!legal!framework!of!the!procedure,!their!role!in!the!review!and!
the! limits! of! their! decision[making!powers.! This! is!why! a! training! programme! should! ensure!
that!experts!are!familiar!with!the!framework!conditions!of!the!review!process.!
Developing!a!pan[European!training!course,!a! first! important! realisation!regarding! the!target!
group! and! the! aims! of! the! pilot! training! session! was! that! it! could! not! possibly! cover! all!
European! external! quality! assurance! systems! with! their! varying! standards,! criteria! and!
procedures.!!
1. Firstly,!this!would!be!unfeasible!regarding!the!sheer!extent,!!
2. Secondly!not!every!potential!expert!is!interested!in!each!national!system,!!
3. Thirdly!it!is!the!core!task!of!each!regional/national!QA!agency!to!make!their!experts!
acquainted!with!the!respective!standards.!
!
A!European!training!programme!cannot!and!should!not!try!to!accomplish!this.!
! !
!
!16
2.2. What can and should be accomplished instead?
As!noted!above,! academic! subject! knowledge! is! an! important!prerequisite! for!experts!but! is!
not!teachable!in!the!context!of!a!training!programme.!Always!bearing!in!mind!that!the!target!
group!are!experts!who!already!have!some!experience!in!external!quality!assurance!procedures!
at! the! national! and! international! level,! this! would! leave! practical! and! communication! skills.!
Since!the!training!course!is embedded!into!a!European!dimension,!the!communicational!skills!
may!have!to!be!complemented!by! input!on! intercultural!awareness!and!communication.!The!
practical!skills!may!include!reflection!on!the!role!of!the!expert!within!the!procedure,!and!work!
on!a!case!study.!
The!knowledge!dimension,!as!previously!discussed,!cannot!comprise!extensive!information!on!
all! European! systems! but! could! focus! instead! on! a! brief! but! representative! overview! of!
external! QA! approaches,! of! different! ‘philosophies’! across! Europe! (for! more! details! see!
chapter!3).!
What!a!European!Training!Programme!should!comprise!…!
• Practical!/!Communicational!Skills!!
• Analytical!skills!and!competences!
• Knowledge!about!external!QA!approaches!in!Europe!
• Intercultural!Awareness!
!
!
!
3. The Pilot Trainings
he! focus! of! this! chapter! is! to! summarise! the! experiences! from! the! trainings!
delivered!for!European!experts!and!for!staff!of!QA!agencies.!First,!the!experiences!
with!the!two!pilot!trainings!for!European!experts!are!described,!followed!by!the!
two! pilot! trainings! for!QA! staff.! For! each! of! the! pilot! trainings! a! brief! description! of! the!
target!group! (participants),!objectives,! learning!outcomes,! content!and!didactics! is!given.!
This!is!followed!by!the!main!results!of!the!evaluation!of!each!of!the!pilot!trainings.!!
3.1. The pilot trainings for European experts
Who!are!the!European!experts!that!should!be!trained!by!E[TRAIN?!The!E[TRAIN!proposal!
indicates! that! experts! who! are! part! of! assessment! panels! in! national! external! QA!
procedures!could!take!part!in!the!pilot!trainings.!!
In! its! meeting! on! 5th! May! 2011! the! E[TRAIN! Stakeholder! Group! (representing! the! QA!
agencies)! specified! that! as! institutional! reviews! were! emerging! in! several! countries! it!
would! be! most! useful! if! the! trainings! focused! on! experts! who! could! participate! in!
institutional!reviews.!!
!
The! experts! trained! by! E[TRAIN! should! bring! international! experience! to! national! panels!
and!also!help!to!legitimize!decisions!made!when!institutional!reviews!are!carried!out.!!
T
The! experts! trained! by! E[TRAIN! should! bring!
international!experience!to!national!panels!and!also!help!
to! legitimize! decisions!made!when! institutional! reviews!
are! carried! out.! ! They! should! also! be! able! to! indicate!
ways! in! which! the! institutions! can! improve! their!
practices.!
!
!19
They! should! also! be! able! to! indicate! ways! in! which! the! institutions! can! improve! their!
practices.!It!was!argued!that!in!order!for!the!experts!to!reach!this!high!level,!they!need!to!
have!solid!experience!from!QA!already!before!entering!the!E[TRAIN!pilot!training.!Hence,!
the!focus!of!the!training!should!be!on!a!higher!level!than!basic!elements!of!QA.!However,!
the!E[TRAIN!Focus!Group! (representing!experienced!experts)!had!warned!against!a!“new!
class! of! elitist! QA! professionals”.! It! would! threaten! the! nature! of! peer! review! and!
ultimately!the! legitimacy!of!the!review!process! itself! if!a!small!number!of!trained!experts!
travelled!around!Europe!to!take!their!seats! in!review!panels!as! if! it!were!a!profession.!To!
avoid!this!threat,! it!was!suggested!that!the!trainings!should!also!be!open!to!experts!with!
little! or! no! experience! in! QA.! A!mediating! solution! was! found! in! the! agreement! by! the!
Stakeholder!Group!that!less!experienced!experts!might!join!the!trainings!after!undergoing!
on!line!E[TRAIN!training!modules!leading!to!the!entry!level.!Moreover,!it!was!decided!that!
the!experts!for!the!pilot!trainings!should!be!proposed!by!the!national!agencies!(i.e.!the!ECA!
members).!Nominated!experts!should!have!some!experience!with!the!national!procedures!
in! the! country! concerned.! It! was! agreed! that! it! would! be! possible! to! nominate! student!
experts.!The!trainer!should!receive!information!from!the!participants!on!their!background!
and!expectations!in!order!to!customize!the!training!according!to!needs.!It!was!agreed!that!
Professor!Nick!Harris! (independent! consultant! and! former!QAA!Director!of!Development!
and!Enhancement!Group)!would!act!as!trainer!for!the!first!pilot.!With!regard!to!the!content!
of!the!training!the!Stakeholder!Group!endorsed!the!distinction!made!by!the!Focus!Group!
(and! included! in! the! “Guidelines! for! Training! of! Experts”)! between! the! awareness!
dimension,! the! cognitive! dimension! and! the! interactive! dimension.! With! a! view! to! the!
training! of! experts! for! institutional! reviews! the! Stakeholder! Group! emphasized! the!
following:!
!
1. The!awareness!dimension;!there!should!be!additional!focus!on!intercultural!
attitudes!(this!can!include!research!papers!on!international!communication).!!
2. The!cognitive!dimension;!here!additional!focus!should!be!on!the!European!
dimension!and!on!variations!in!institutional!quality!management/!autonomy!in!
higher!education!institutions!in!different!countries.!After!completing!the!training!
the!experts!should!be!up!to!date!on!international!standards.!!!!
!
!20
3. The!interactive!dimension;!the!experts!should!e.g.!learn!about!intercultural!
communication,!effective!interviewing!skills,!and!other!“soft!skills”.!
!
It!was!against!this!background!that!the!preparations!for!the!first!pilot!training!of!experts!
started.!
3.1.1. The first pilot training of European experts
For!both!budgetary! reasons!and!busy!agendas!of!experts! it!was!decided!to! limit! the! first!
pilot!training!to!1!day!with!a!dinner!on!the!previous!evening.!The!trainer!and!the!project!
partners! responsible! for! the! training! (ÖAR1! and! NVAO)! met! once! to! prepare! the! pilot!
training.! The!envisaged! content! for! the! training!was!derived! from! the!Guidelines! for! the!
Training!of!Experts:!
1. What!should!experts!be!aware!of?!
- Different!organisational!cultures,!different!quality!cultures!!
- Being!biased!(prior!perceptions/prejudices)!
- Intercultural!awareness/Intercultural!communication!!
2. What!should!experts!know?!!
- Knowledge!of!the!European!dimension!of!QA!!
- Knowledge!of!the!national!dimension!of!QA!!
- Knowledge!of!cultural!differences!!
3. What!should!experts!be!able!to!do?!!
- Audit!techniques/”tools!&!tricks”,!e.g.!how!to!ask!good!questions!
- Team!work:!how!to!effectively!work!together!as!a!review!team!
- How!to!write!a!(joint)!report!that!includes!the!opinions!of!all!!
- “Do’s!and!Dont’s”!at!the!site!visit!
- How!to!act!in!intercultural!situations!
!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1!With!the!legal!validity!of!the!HS[QSG!a!new!and!single!agency!(Agency!for!Quality!Assurance!and!Accreditation!Austria)!was!
established!which!is!responsible!for!external!quality!assurance!in!the!HE!sector!(Public!Universities,!Universities!of!
Applied!Sciences,!Private!Universities).!The!‘Qualitätssicherungsrahmengesetz’!(QSRG)!entered!into!force!on!1st!March!
2012.!The!ÖAR!!persisted!until!31st!August!2012.!
!
!21
It!was!quickly!agreed! that! this!agenda!was! too!much! for!a!one!day! training.! Instead,! the!
trainer! had! made! a! proposal! which! included! several! dimensions! of! QA! training.! A!
distinction!was!made!between!knowledge!and!understanding!which!could!be!provided!by!
on!line!training,!and!skills!(including!roles!and!practice)!which!would!be!the!main!focus!of!
the!training!day.!Each!of!these!dimensions!consisted!of!a!break!down!into!4!categories!on!
an! introductory! level! and! 4! subsequent! levels! of! achievement.! This! resulted! in! a!matrix!
which!would!have!to!be!further!developed.!Learning!outcomes!should!be!included!for!each!
of!the!dimensions,!categories!and!levels.!It!turned!out!that!this!was!a!too!complicated!task!
to! achieve! in! time! for! the! start! of! the! pilot! training.! Instead! the! trainer! would! ask!
participants! at! the! start! of! the! training!what! they! expected! as! learning!outcomes!of! the!
training.! The! first! question! in! the! evaluation! form! was! then! whether! these! learning!
outcomes!had!been!achieved! in! the! view!of! the!participants.! The!other!questions! in! the!
evaluation! form! dealt! with! opinions! on! the! strong! and! weak! points! of! the! training.! In!
addition,! the! participants! were! specifically! asked! to! rate! the! following! aspects! from! 1!
(excellent)!to!5!(poor):!
!
- Clarity!of!session!goals!
- Trainer’s!presentation!skills!
- Relevance!of!information!
- Clarity!of!information!
- Scope!(coverage)!of!training!
- Relevance!of!materials!presented!during!training!
- Organisation!of!training!
- Overall!rating!of!training.!
!
The!knowledge!base!with!on!line!materials!for!experts!was!not!available!by!the!time!of!the!
first! training! (this! was! meant! to! be! an! outcome! by! the! end! of! the! E[TRAIN! project).!
Therefore,! it!was!decided! to! start! the! training!day!with!a!discussion!on! two!assignments!
which!the!experts!were!asked!to!complete.!One!assignment!on!stakeholders!perspectives!
was!to!be!accomplished!by!the!experts!prior! to!the!meeting,!and!another!assignment!on!
placing!the!expert’s!own!national!QA!system!in!a!matrix!of!systems!would!be!included!as!a!
!
!22
plenary!exercise!during!the!training!day.!The!attention!in!the!project!partners’!preparatory!
meeting!then!turned!to!the!skills!dimension!and!the!role!playing!exercises!which!would!be!
the!main!part!of!the!training.!First!the!training!programme!with!the!different!sessions!was!
designed!and!then,!after!the!preparatory!meeting,!the!training!materials!were!finalised!by!
the!trainer!and!sent!to!the!participants.!The!role!play!would!be!introduced!to!participants!
during! the! dinner! on! the! night! before! the! training.! This! would! give! some! time! for!
participants!to!prepare!the!roles!which!were!written!down!for!them.!Unfortunately!not!all!
participants!were!present!at!the!dinner!and!so!the!preparation!did!not!take!place!then,!but!
had!to!be!delayed!until!the!next!day!of!the!training.!!
Eight! experts,! including! one! student! expert,! were! proposed! by! the! agencies! and!
participated! in! the! training.! All! experts! had! experience!with! national!QA!procedures! but!
not! all! of! them!with! institutional! reviews.! It! turned!out! that! among!ECA!members! there!
was! less! experience! with! institutional! reviews! in! practice! than! envisaged! by! the!
Stakeholder! Group.! In! addition! to! these! eight! experts! to! be! trained,! four! experienced!
experts!from!the!Focus!Group!(including!one!student!member)!participated!in!the!training.!
These!experts!participated!fully!in!the!training!but!also!had!a!role!as!observers.!The!Focus!
Group!met! immediately! after! the! training! to! discuss! their! observations.! The! Stakeholder!
Group!had!decided!that!each!agency!could!send!an!observer!to!the!pilot!training.!This!led!
to!an!additional!group!of!7!observers!from!agencies.!The!group!of!organisers!consisted!of!
the!trainer!and!4!members!of!staff!of!ÖAR!and!NVAO!who!also!had!a!role! in!the!training!
(particularly!in!the!role!play).!
The! training! was! held! on! 6th! October! 2011! at! the! NVAO! offices! in! The! Hague! (with! a!
preceding!dinner!on!October!5th).!A!background!paper!introducing!the!programme!for!the!
training!day!and!the!preparatory!material!was!sent!to!participants!in!advance:!ANNEX!2.!!
The! pilot! training!was! evaluated! by! the! Focus! Group,! the! Stakeholder! Group! and! by! an!
observation!report!of!the!responsible!project!partner!ÖAR.!The!main!conclusions!from!this!
evaluation!were:!
1. The!overall!rating!of!the!first!pilot!training!was!3!on!a!scale!from!1!(excellent)!to!5!
(poor).!The!ratings!of!the!different!aspects!and!comments!made!by!participants!
made!it!clear!that!significant!improvements!were!needed!for!the!second!pilot!
training.!
!
!23
2. The!training!focused!mainly!on!the!interactive!dimension!(role!play),!less!on!
knowledge,!and!very!!little!on!awareness.!It!was!concluded!that!the!second!training!
should!have!a!stronger!focus!on!the!latter!dimensions,!particularly!on!intercultural!
awareness.!
3. More!structure!for!the!training!was!needed:!explicit!learning!outcomes!for!each!of!
the!3!dimensions!should!be!developed.!
4. There!should!be!more!clarity!and!coherence!with!regard!to!didactics!and!materials!
used.!!
5. The!role!play!needed!a!real!case[study.!!
6. A!clearer!profile!of!the!trained!experts!is!needed!(e.g.!programme!or!institutional!
reviews).!!
7. There!were!too!many!observers;!there!were!as!many!agency!staff!present!as!there!
were!experts,!and!within!the!whole!group!the!8!experts!to!be!trained!formed!only!
a!third!of!the!group.!
8. One!training!day!is!too!short.!
!
*
Keeping!in!mind!the!critical!results!of!this!evaluation!the!E[TRAIN!Steering!Group,!ÖAR!and!
NVAO! in!particular,! took!responsibility! for!designing!a!new!structure! for! the!second!pilot!
training.!
!
The!first!pilot!training!of!experts!focused!mainly!on!the!
interactive!dimension!(role!play),!less!on!knowledge,!and!
very!!little!on!awareness.!It!was!concluded!that!the!
second!training!should!have!a!stronger!focus!on!the!
latter!dimensions,!particularly!on!intercultural!
awareness.!
!
!24
3.1.2. The second pilot training of European experts
It! was! decided! that! the! training! should! be! prolonged! to! 1! ½! days! to! enable! a! better!
coverage!of!the!3!dimensions.!The!target!group!of!experts!to!be!trained!in!the!second!pilot!
training! were! described! as! “experts! who! can! participate! in! institutional! external! quality!
assurance! procedures! (audits/accreditations/evaluations)! in! different! national! systems”.!
The! training! was! structured! along! the! lines! of! the! three! previously! agreed! dimensions!
which!were!specified!as!the!following!aims!for!the!training:!
•! To! increase! experts’! knowledge! of! the! European! dimension! of! external! quality!
assurance! and! of! the! characteristics! of! different! European! external! quality!
assurance!systems;!
•! To! raise! intercultural! awareness! when! working! as! an! expert! in! distinct!
(sub)national!external!quality!assurance!systems;!
•! To!enhance! the!audit! skills!needed! to! function!effectively!as!an!expert! in!various!
(sub)national!external!quality!assurance!systems.!
The! training! was! divided! into! three! modules:! European! external! QA! approaches;!
Intercultural! awareness;! and,! Audit! skills! for! diverse! national! settings.! Each! of! these!
modules!corresponded!with!one!or!more!of!the!overall!aims!of!the!training.!For!each!of!the!
modules!objectives!and!intended!learning!outcomes!were!drawn!up.!These!objectives!and!
learning!outcomes!were!based!on!the!overall!aims.! !Based!on!the! learning!outcomes!the!
didactical!approach!and! the!desired!profile!of! the! trainer!were!described.!Therefore,! the!
description! of! each! of! the! modules! followed! the! same! format:! Objectives! –Learning!
Outcomes! –! Didactical! Approach! –! Trainer’s! profile.! We! will! limit! ourselves! here! to! an!
overview!of!the!learning!outcomes:!
!
!
!25
!
!
A., European,external,QA,approaches,
1.! discuss!the!position!of!external!quality!assurance!in!the!European!Higher!Education!
Area;!
2.! describe!how!higher!education!systems!and!their!specific!policy!choices!impact!the!
external!quality!assurance!methodologies;!
3.! classify!(sub)national!models!of!external!quality!assurance;!
!
B., Intercultural,awareness,
4.! position! one’s! own! national! culture! in! a! generic! frame! of! reference! for! cultural!
differences!and!characteristics;!
5.! frame! questions! related! to! the! quality! of! higher! education! in! the! perspective! of!
cultural!differences!and!characteristics;!
6.! perceive!culture!dependent!frames!of!reference!in!order!to!avoid!being!mislead!by!
them;!
!
C., Audit,skills,for,diverse,national,settings,
7.! communicate! effectively! and! appropriately! with! all! actors! in! diverse! external!
quality!assurance!settings;!
8.! employ! advanced! communication! techniques! for! successful! and! relevant!
information!gathering!in!external!quality!assurance!procedures;!!
It!was!decided!that!the!training!should!be!prolonged!to!1!
½!days!to!enable!a!better!coverage!of!the!3!dimensions.!
The!target!group!of!experts!to!be!trained!in!the!second!
pilot!training!were!described!as!“experts!who!can!
participate!in!institutional!external!quality!assurance!
procedures!(audits/accreditations/evaluations)!in!
different!national!systems”.!
!
!26
9.! assume! suitable! behaviour! for! particular! expert! roles! in! diverse! intercultural!
settings!of!external!quality!assurance!procedures;!!
10.! cooperate! effectively! and! appropriately! in! decision[making! processes! of! an! audit!
team.!
The! trainers!had! to!meet! the! trainer’s!profile!and!were!carefully! selected.!A!preparatory!
meeting!of! the!3! trainers!and! the! responsible!E[TRAIN!project!partners! (ÖAR!and!NVAO)!
was!organised.!At! this!meeting!the!content!of! the!programme!was!agreed,! the!didactical!
approach!was!discussed!and!the!assignments!and!materials!were!agreed.!The!pre[training!
assignments!for!participants!consisted!of:!reading!a!selected!case[study!which!was!used!for!
the!role!play;!a!text!on!European!QA;!short!profiles!of!3!national!agencies;!and,!a!matrix!on!
which! experts! were! asked! to! position! their! own! national! QA! system.! This!matrix! would!
then!be!discussed!during!the!training.!!!
The! programme! followed! the! sequence! of! the!modules! but! not! exactly;! to! increase! the!
span! of! attention! during! the! day! it! was! decided! to! break! the! intercultural! awareness!
module! in! two! sessions.! Intercultural! awareness! part! 1!was! scheduled! before! lunch! and!
Part!2!before!the!end!of!the!day.!For!the!programme!of!the!training!see!ANNEX!3.!
!
The!training!was!held!on!23!and!24!April!2012! in!Vienna.!Ten!experts!participated! in! the!
training.!As!in!the!first!pilot!training!they!were!nominated!by!the!ECA!member!agencies.!In!
addition,! three! experienced! experts! from! the! Focus! Group! participated! in! the! training.!
They!also!evaluated!the!training!immediately!after!the!end!of!the!training.!The!organising!
team!consisted!of!the!three!trainers,!two!staff!members!of!the!Austrian!agency!(of!which!
one!acted!as!observer)2,!and!two!staff!members!of!NVAO.!As!a!consequence!there!was!a!
mixed!group!of!people!from!Austria,!Belgium,!Germany,!the!Netherlands,!Norway,!Poland,!
and!Spain.!!
As!with!the!first!pilot! training!the!second!one!was!thoroughly!evaluated.!During!the!final!
session! participants! were! asked! to! fill! in! an! evaluation! form! and! these! were! collected!
before! going! to! lunch.! As! a! consequence! there! was! a! very! high! response! rate.! The!
achievement!of!the!learning!outcomes!were!rated!and!a!number!of!other!questions!were!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2!AQ!Austria!
!
!27
asked.!After! lunch!the!Focus!Group!shared!their!observations,!and!Notburga!Damm!from!
AQ!Austria,!wrote!an!observation!report.!The!main!results!of!these!evaluations!were:!
!
•! The!training!was!clearly!appreciated!by!the!participants;!the!overall!rating!was!7.8!
on!a!scale!from!1!(poor)!to!10!(excellent).!
•! Most!participants!felt!that!1!½!days!of!training!was!sufficient!and!feasible.!
•! A!group!size!of!10!experts!worked!well,!especially!in!the!group!work!(2!groups!of!5!
experts).!When!splitting!in!2!groups!for!a!role!play!or!discussion!(e.g.!when!making!
judgements)! there! are! always! 2! trainers! needed! to! give! maximum! feedback! to!
participants.!!
•! The! structured! approach! taken! worked! well;! the! content! structure! (e.g.! the!
sequence!of!the!sessions)!could!even!be!further!refined!and!improved.!
•! The! case[study! and! role! play! generated! a! lot! of! enthusiasm! and! there! was!
agreement!that!these!were!the!most!beneficial!parts!of!the!training.!
•! Intercultural!awareness!should!be!integrated!in!the!role!play/case[study.!
•! More! attention! is! needed! for:! the! classification! of! national!QA! systems;! the! role!
and!attitude!of!the!auditor;!panel!chair!skills.!
!
!
!
!
The! structured! approach! taken! in! the! second! pilot!
training!of!experts!worked!well.!The!case[study!and!role!
play! generated! a! lot! of! enthusiasm! and! there! was!
agreement! that!these!were! the!most!beneficial!parts!of!
the!training.!
!
!28
3.2. The pilot trainings for QA staff
Originally!in!the!E[TRAIN!project!proposal!the!staff!trainings!were!meant!to!train!QA!staff!
members! to! become! trainers! of! European! experts! after!which! they! could! go! on! to! train!
European! experts! themselves.! However,! it! turned! out! that! there!was! no! clear! profile! of!
these!European!experts!which!was!shared!by!the!agencies!at!the!start!of!the!project.!As!a!
consequence,! a! lot! of! time! and! energy! went! into! discussions! among! the! agencies!
(Stakeholder! Group! and! Steering! Group)! and! experts! (Focus! Group)! on! reaching! an!
agreement!about!who!should!be! trained!and!what! this!would!mean! for! setting!up! these!
trainings.!Moreover,!it!gradually!became!apparent!that!agencies!preferred!so!called!‘train!
the!trainers’!sessions!which!would!help!staff!members!to!become!better!national!trainers.!
The! sequence!of! trainings!was! therefore! reversed:! first! the!pilot! training!of!experts! took!
place,!followed!by!the!training!of!staff.!!
3.2.1. The first pilot training of QA staff
The!Stakeholder!Group!agreed!that!the!first!pilot!training!of!QA!staff!training!should!take!
place!after! the! first!pilot! training!of!experts.!This!would!allow! for! the! involved!parties! to!
reflect!upon! the!experts! training!before! carrying!out! the! staff! training.! The! staff! training!
would!function!as!a!“train!the!trainers”!meeting.!The!original!idea!was!that!observers!from!
the!agencies!who!were!present!at!the!first!pilot!training!of!experts!could!participate!in!the!
staff! trainings! so! that! they! could! act! as! future! trainers! of! European! experts.! However,!
when!these!staff!members!were!asked!what!their!preference!for!the!first!pilot!training!was!
it!turned!out!that!the!vast!majority!wanted!a!training!which!would!help!them!improve!as!
national! trainers.! Therefore,! it! was! decided! that! the! first! pilot! training! should! have! the!
format!of!a!workshop!of! trainers!where! they!could!share!how!they! train!experts! in! their!
agencies.! Mutual! learning! and! sharing! good! practices! were! important! goals! for! this!
workshop.!Moreover,!there!should!be!overarching!outcomes!for!the!group!of!agencies! in!
the! form! of! an! overview! of! training! practices! in! agencies,! drafting! recommendations!
concerning!national!trainings,!and!initiating!the!development!of!a!skills!profile!of!trainers.!!
The!workshop!took!place!on!3rd!November!2011!in!Vienna.!For!the!programme!of!the!first!
pilot!training!of!QA!staff!see!ANNEX!4.!
!
!
!29
!
!
There! were! in! total! 15! participants! in! this! workshop! representing! (prospective)! trainers!
from!Austria,!Denmark,!Germany,! the!Netherlands! and! Flanders,! Spain,! Switzerland,! and!
the! UK.! The! survey! of! good! practices! in! experts’! trainings! had! shown! that! the! Quality!
Assurance!Agency!UK!had!a!long!lasting!experience!with!an!extensive!training!programme,!
particularly!in!the!field!that!was!mostly!new!but!of!increasing!relevance!for!ECA!members,!
namely! the! training! of! institutional! reviewers.! The! first! part! of! the! programme! was!
therefore! reserved! for! a! presentation! by! Dr! Gillian! King! (consultant! and! former! Deputy!
Director! of! the! Reviews! Group! at! QAA)! of! the! QAA! training! programme.! Then! the!
participants!presented!the!training!practices! in! their!own!agencies! in!which!they! focused!
primarily!on!the!following!aspects:!
!
•! Description!of!experts!profile!(which!experts!are!trained)!!
•! Content!and!programme!
•! Preparation!and!materials!
•! Conduction!of!the!training!
•! Learning!outcomes!
•! Didactic!approach!and!methods!
•! Skills!of!trainers!
!
The! first! pilot! training! of! staff! members! from! QA!
agencies! should! have! the! format! of! a! workshop! of!
trainers!where! they! could! share!how! they! train! experts!
in! their! agencies.! Mutual! learning! and! sharing! good!
practices!were!important!goals!for!this!workshop.!
!
!30
For!an!overview!of!the!training!programme!characteristics!of!these!agencies!see!ANNEX!1.!
Although!there!are!some!significant!differences!in!how!these!trainings!are!conducted!there!
are!also!commonalities.!It!became!apparent!that!most!agencies!have:!
!
•! The!same!training!or!briefing!for!all!experts;!!
•! A!focus!on!the!review!framework;!!
•! Legal!aspects!as!preparatory!materials;!!
•! One!day!of!training!or!less;!!
•! A!tendency!to!focus!didactics!on!presentations!and!discussions,!although!the!use!of!
case!studies!is!increasing;!!
•! Trainers!from!staff!or!the!management!of!the!agency.!!
!
The!participants!also!discussed!ten!draft!recommendations!to!agencies!for!the!training!of!
experts.!A!final!draft!of!these!recommendations!is!included!as!ANNEX!5.!
The!workshop!was! evaluated!which! resulted! in! an! overall! rating! of! 2! on! a! scale! from! 1!
(excellent)! to! 5! (poor).! During! the! concluding! session! the! participants! expressed! their!
preferences! for!the!second!pilot! training!to! include!guidance!on!the!structural!aspects!of!
setting! up! a! training! and! for! opportunities! to! advance! their! communication! skills! as! a!
trainer.!!
!
3.2.2. The second pilot training of QA staff
All!staff!members!who!are!responsible!for!the!training!of!the!panel!members!within!their!
agency! have! an! extensive! knowledge! of! the! subject,! i.e.! external! quality! assurance,! and!
they! are! experienced!with! the! practicalities! of! external! quality! assurance! procedures! as!
well.!At! the!same!time!the!majority!have!not!undergone!any!preparation! for! the!specific!
task!of!converting!that!knowledge!into!a!training!course.!They!had!no!structured!training!in!
how! to! train! panel! members! but! follow! more! or! less! a! “Do[it[yourself”! instead! of! a!
didactic[methodical!approach.!This!is!why!it!was!decided!that!the!second!pilot!training!for!
QA!staff!should!be!designed!as!a!“Train[the[trainers”!workshop,!conducted!by!professional!
trainers.!The!staff!members!would!be!given!tools!which!they!can!use!when!designing!their!
!
!31
training!course!back!home.!On!10!and!11!May!2012!the! train[the[trainers!workshop!was!
held!in!Madrid.!The!workshop!was!divided!in!two!parts;!on!the!first!day!Gillian!King!was!the!
trainer!for!the!structural!part!of!the!training!and!on!the!second!day!the!Polish!consultants!
Przemyslaw! Kotecki! and! Anna! Giniewicz! were! responsible! for! the! communication! skills!
training.!The!learning!outcomes!and!the!content!of!the!two!training!days!were!developed!
by!the!trainers.!
The!programme!for!the!two!training!days!is!included!as!ANNEX!6.!The!content!of!the!first!
training! day! is!mainly! included! in! the! publication! ‘Handbook! for! the! training! of! experts’.!
Fifteen!learning!outcomes!were!developed!for!the!two!training!days.!
!
Table 1. Learning Outcomes for the two days training of staff members Day,1,
1. be!aware!of!their!own!skills!as!a!trainer!and!have!identified!areas!for!development!
higher!education!area!
2. be!aware!of!the!need!to!know!their!participants!and!the!material!to!be!
communicated!
3. be!able!to!write!session!aims!and!outcomes!
4. be!able!to!choose!delivery!methods!which!suit!the!participants!and!the!material!
5. be!able!to!develop!training!materials!(notes,!visuals)!which!deliver!the!outcomes!
6. consider!assessment!tools,!if!required!
7. understand!the!importance!of!evaluating!the!programme.!
Day,2,8. be!able!to!identify!the!barriers!in!communication!
9. be!able!to!break!down!the!communication!barriers!
10. be!aware!of!different!aspects!of!communication!
11. be!able!to!work!with!the!group!dynamics!!
12. be!able!to!work!with!“difficult”!participants!!13. understand!the!role!of!auto[presentation!in!delivering!the!training!
14. be!able!to!use!auto[presentation!techniques!for!delivering!of!the!training!
15. be!aware!of!methods!and!strategies!for!stress!management!
!
There!were!10!participants! from!the!same!countries!as! in!the!first!pilot!training.!A!needs!
questionnaire! was! sent! to! participants! asking! them! for! their! experience! as! a! trainer,!
whether!they!had!been!trained!as!a!trainer!before,!their!strengths!as!a!trainer,!and!areas!
for!development.!!
!
!32
The! training! was! evaluated! by! participants! filling! in! an! evaluation! and! by! one! of! the!
participants!who!had!acted!as!observer.!The!evaluation!led!to!the!following!results:!
• The!overall!rating!on!a!scale!from!1!(poor)!to!6!(excellent)!was!:!5.2!for!day!1!and!
4.8!for!day!2.!
• Participants!considered!day!1!to!be!particularly!useful!and!well[structured.!!
• The!agendas!on!both!days!were!a!bit!overloaded;!the!time!pressure!was!
noticeable.!!
• A!possible!solution!could!be!that!the!first!day!would!be!stretched!into!2!days!and!
some!of!the!objectives!of!the!second!day!integrated!in!the!first!day.!
! !
The! second! pilot! training! of! staff! members! from! QA!
agencies!was!a!train!the!trainers!workshop.!It!was!valued!
highly!by!the!participants.!!
!
!33
4. Conclusions
he! experiences! with! the! pilot! trainings! and! the! discussions! during! the! E[TRAIN!
dissemination!conference!held!in!Madrid!on!14!and!15!June!2012!gave!very!useful!!
feedback!on!a!possible!follow[up!to!the!project.!The!following!observations!may!
guide!the!proposals!for!future!trainings!of!experts!and!staff!organized!by!ECA:!
4.1. The training of European experts
• Expert!trainings!should!be!“tailor[made”!depending!on!the!target!group,!e.g.:!!
o Experts!for!institutional!reviews!!
o Experts!for!assessments!of!joint!programmes!(single!accreditation!
procedures)!
o Experts!for!the!assessment!of!HEIs!or!programmes!requesting!the!ECA!
certificate!for!internationalisation!(CeQuInt!project).!
• A!community!of!trained!experts!should!be!established!by!carefully!considering:!
o Recruitment:!to!prevent!an!elite!corps!of!(retired)!experts!trainings!should!
not!solely!rely!on!nominations!of!experienced!experts!but!be!open!to!
newcomers!with!less!experience!in!external!QA.!Recruits!should!share!the!
agencies’!values,!ethos,!and!ideas!of!professionalism.!!
o Inclusion!in!the!European!database,!preferred!participation!in!panels,!and!
maintenance!of!communication!(e.g.!newsletter,!providing!opportunities!
for!updating!skills,!expression!of!appreciation)!
o Performance!of!experts!should!be!reviewed!
o Student!experts!should!be!included!in!trainings!and!in!the!community!of!
experts!but!they!may!require!some!specialised!content.!
• Structure!and!content!of!the!training:!
o The!structured!approach!taken!in!the!second!pilot!training!should!be!
maintained!and!refined!
o The!content!of!the!training!should!be!adapted!according!to!the!profile!and!
experience!of!experts!
T
!
!34
o Knowledge!of!European!external!QA!approaches!should!be!enhanced!by!
expanding!the!ECApedia!and!making!useful!references!to!materials!in!the!
ECApedia!throughout!the!training!and!in!a!designated!assignment!
specifically!
o Intercultural!awareness!should!be!integrated!in!the!discussion!on!
European!QA!systems!and!in!the!training!of!audit!skills!(including!a!case[
study!and!role!play)!
o Improve!the!learning!effect!of!the!role!play!by!providing!country[specific!
information!on!local!higher!education!!
o More!attention!needed!for:!the!role!and!attitude!of!the!auditor;!panel!
chair!skills;!enhancement[led!vs.!assessment[focused!reviews;!creating!an!
atmosphere!of!trust!and!openness.!
• Organisation!and!finances:!
o Trainings!should!be!organized!on!an!easily!accessible!location,!if!possible!in!
cooperation!with!other!networks!to!increase!European!support!and!the!
funding!base!
o Financial!sustainability!can!be!enhanced!by!sharing!costs!between!involved!
networks,!agencies,!and!experts!
o Organize!a!1!½!day!training!for!approximately!10!experts!who!can!be!split!
in!2!groups!guided!by!2!trainers!
o Trainings!should!be!organised!at!least!twice!a!year!and!more!often!when!
the!need!occurs.!
!
!
Expert! trainings! should! be! “tailor[made”! depending! on!
the! target! group! and! a! community! of! trained! experts!
should!be!established.!
!
!35
4.2. The training of QA staff (train the trainers)
• Train!the!trainers!workshops!of!2!days!should!be!organised!annually!for!trainers!of!
national!agencies!
• The!train!the!trainers!workshops!may!adapt!the!objectives!and!learning!outcomes!
according!to!the!needs!of!the!participants!
• Future!train!the!trainers!workshop!may!focus!on!the!development!of!a!community!
of!experts!and!on!increasing!the!effectiveness!of!recruitment!and!performance!of!
experts!
• A!community!of!trainers!should!be!established!by!a!regular!exchange!of!trainers!
among!national!agencies!and!maintenance!of!communication!(e.g.!newsletter,!
providing!opportunities!for!updating!skills,!etc.)!
• The!costs!of!the!training!should!be!shared!by!the!agencies!involved.!
!
!
!
! !
Train! the! trainers! workshops! of! 2! days! should! be!
organised!annually!for!trainers!of!national!agencies.!
!
!36
!
!
!37
5. Annexes
!
!
Annex 1:
Detailed overview of training characteristics of QA agencies
!
Qua
lity
Ass
uran
ce
Age
ncy
Tra
inin
g’s
targ
et
audi
ence
T
rain
ing
cont
ent
Prep
arat
ory
mat
eria
l T
rain
ing
proc
ess
Lea
rnin
g ou
tcom
es
Did
actic
s T
rain
ers’
ski
lls
AC
E
Den
mar
k Sa
me
trai
ning
for
repr
esen
tativ
es o
f al
l sta
keho
lder
gr
oups
Ass
essm
ent
fram
ewor
k;
accr
ed. c
rite
ria;
ro
les
and
task
s of
exp
erts
No
exce
ssiv
e pr
epar
atio
n (le
gal f
ram
e-w
ork,
gui
des,
ex
ampl
e of
a
repo
rt)
1 da
y tr
aini
ng;
lunc
h as
an
impo
rtan
t soc
ial
com
pone
nt
Kno
w a
sses
smen
t fra
mew
ork
and
crite
ria/
inte
rpre
tatio
n;
min
imal
thre
shol
d le
vel;
know
ro
les
and
task
s of
exp
erts
; so
cial
com
pete
nces
Pres
enta
tion
base
d tr
aini
ng;
disc
ussi
ons;
lit
tle c
ase
wor
k,
expe
rien
ces
of
expe
rts
Tw
o tr
aine
rs,
teac
hing
ex
peri
ence
, ac
cred
itatio
n ex
peri
ence
s
AN
EC
A
Sam
e tr
aini
ng fo
r re
pres
enta
tives
of
all s
take
hold
er
grou
ps +
in
tern
atio
nal
expe
rts
Sele
ctio
n cr
iteri
a fo
r ch
oosi
ng
expe
rts;
gen
der
and
terr
itori
al
bala
nce
assu
red
Eva
luat
ion
con-
te
xt +
proc
ess;
to
ols;
acc
red.
cr
iteri
a an
d th
eir
inte
rpre
tatio
n;
AN
EC
As
role
, pr
actic
al is
sues
Stan
dard
ized
m
ater
ial (
lega
l as
pect
s,
guid
elin
es a
nd
stan
dard
s,
eval
uatio
n pr
otoc
ols,
cas
e st
udy
for
smal
l gr
oup
wor
ks)
1 da
y tr
aini
ng
Prac
tical
or
ient
atio
n, w
ith
feed
back
loop
s
Kno
w le
gal a
spec
ts; k
now
ev
alua
tion
proc
ess;
ski
lls to
an
alys
e a
degr
ee;
inte
rpre
tatio
n of
cri
teri
a; s
oft
skill
s; e
thic
s an
d pr
ofes
sion
alis
m, t
eam
pla
y;
empa
thy
Lec
turi
ng a
bout
ev
alua
tion
proc
ess;
W
orks
hop
I:
stan
dard
s an
d gu
idel
ines
; in
terp
reta
tion
of
crite
ria
Wor
ksho
p II
: sm
all g
roup
s,
case
stu
dies
, si
mul
atio
n of
ev
alua
tion
Exp
erts
from
A
NE
CA
s IQ
M
team
with
ex
peri
ence
s in
ev
alua
tion;
ac
adem
ics
with
Q
A
expe
rien
ces;
ex
peri
ence
s in
m
anag
emen
t of
degr
ees
AQ
U
Cat
alun
ya
Sam
e tr
aini
ng fo
r re
pres
enta
tives
of
all s
take
hold
er
grou
ps;
Add
ition
ally
“ad
ho
c” c
ours
es fo
r st
uden
ts
Sele
ctio
n cr
iteri
a fo
r ch
oosi
ng
expe
rts
Eva
luat
ion
pro-
ce
ss, c
onte
xt,
accr
ed. c
rite
ria,
IT
tool
s Sp
ecia
l con
tent
fo
r st
uden
t co
urse
s:
purp
ose
AQ
U,
stud
ents
rol
e in
co
mm
ittee
s, e
tc
Man
agem
ent o
f ex
pert
s is
par
t of
inte
rnal
QA
sy
stem
1 da
y tr
aini
ng
Ad
hoc
cour
ses
for
stud
ents
0.5
da
y
Impr
ovem
ent o
rien
ted
aspe
cts;
ap
plic
atio
n of
acc
red.
cri
teri
a;
team
wor
k an
d so
ft s
kills
Lec
turi
ng;
disc
ussi
on o
f ex
ampl
es; c
ase
stud
ies
(PB
L)
incl
udin
g fe
edba
ck
Stra
tegi
c as
pect
s ta
ught
by
AQ
U
man
agem
ent;
so
ft s
kills
, pr
oble
m b
ased
, im
prov
emen
t sk
ills
by p
roje
ct
man
ager
s
EV
A
All
expe
rts a
re
trai
ned
(exc
ept
exis
ting
expe
rts)
Se
lect
ion
is b
ased
on
ped
agog
ical
kn
ow h
ow a
nd Q
A
expe
rien
ces
EV
A’s
con
cept
; ac
cred
itatio
n as
m
etho
d, a
ccre
d.
crite
ria;
rol
es
and
task
s of
expe
rts
Let
ter
with
ta
sks o
f ex
pert
s;
Leg
isla
tive
text
s; L
O
mat
eria
l
3 ho
urs t
rain
ing
Kno
wle
dge
of a
ccre
dita
tion
as
met
hod;
rol
e of
exp
erts
, kno
w
asse
ssm
ent c
rite
ria
Pres
enta
tions
an
d pl
enar
y di
scus
sion
s (c
ase
stud
ies
plan
ned
for
futu
re)
EV
A
man
agem
ent +
ex
tern
al
coun
sello
r w
ith
thor
ough
ex
peri
ence
s in
accr
edita
tion
FHR
1 W
orks
hops
for
coun
cil m
embe
rs
and
acad
emic
m
embe
rs
Sele
ctio
n cr
iteri
a fo
r ex
pert
pan
el
mem
bers
Purp
ose
of
asse
ssm
ent;
as
sess
men
t fr
amew
ork;
na
tiona
l con
text
, ro
les o
f exp
erts
an
d st
aff,
“goo
d qu
estio
ns”
No
form
al
trai
ning
yet
N
o fo
rmal
tr
aini
ng y
et
Nee
ded
LO
acc
ordi
ng to
su
rvey
s: K
now
how
abo
ut
asse
ssm
ent f
ram
ewor
k an
d cr
iteri
a/in
terp
reta
tion;
kno
w
role
s and
task
s of e
xper
ts;
know
how
to a
sk sm
art
ques
tions
; kno
w h
ow to
wri
te a
re
port
No
form
al
trai
ning
yet
N
o fo
rmal
tr
aini
ng y
et
FIB
AA
ac
adem
ics,
prof
essi
onal
s st
uden
ts: r
egul
ar
trai
ning
s of n
ew
revi
ewer
s (2
per
anno
) and
se
min
ars f
or
expe
rien
ced
expe
rts (
2 pe
r an
no)
New
com
ers:
le
arn
abou
t as
sess
men
t pr
oced
ure,
na
tiona
l re
gula
tions
and
E
urop
ean
stan
dard
s as
sess
men
ts
fram
ewor
k.
Exp
erie
nced
ex
pert
s: n
ew
regu
latio
ns,
stan
dard
s and
tr
ends
, co
nsis
tenc
y of
pr
oced
ures
Bac
kgro
und
docu
men
ts
(reg
ulat
ions
);
FIB
AA
as
sess
men
t gu
ide
1 da
y tr
aini
ng; 4
tr
aini
ngs p
er
year
org
aniz
ed
New
com
ers l
earn
rol
es a
nd
task
s of e
xper
ts; d
iffer
ence
s co
nditi
ons/
reco
mm
enda
tions
; as
sess
men
t met
hods
, nat
iona
l an
d E
urop
ean
regu
latio
ns.
Exp
erie
nced
exp
erts
lear
n ab
out c
hang
es o
f reg
ulat
ions
, in
terp
reta
tion
of r
egul
atio
ns
and
abou
t spe
cial
item
s (i.e
. jo
int p
rogr
amm
es)
Pres
ent c
ase
stud
ies;
dis
cuss
ex
ampl
es,
wor
king
on
text
s, re
port
ing
of o
wn
expe
rien
ces
Exp
erie
nced
st
aff o
f age
ncy;
ex
peri
ence
s in
QA
pol
icie
s and
pr
oced
ures
1"W
ith"the"legal"validity"of"the"HS2QSG"a"new"and"single"agency"(Agency"for"Quality"Assurance"and"Accreditation"Austria)"was"established"which"is"responsible"for"external"quality"assurance"in"the"HE"sector"(Public"
Universities,"Universities"of"Applied"Sciences,"Private"Universities)."The"‘Qualitätssicherungsrahmengesetz’"(QSRG)"entered"into"force"on"1st"M
arch"2012."T
he"FH"Council"persisted"until"31st"August"2012.""
N
VA
O
Tra
inin
g of
top
leve
l per
sons
for
inst
itutio
nal
audi
ts; s
epar
ate
trai
ning
s for
st
uden
ts a
nd
secr
etar
ies
Sele
ctio
n cr
iteri
a fo
r ch
oosi
ng
expe
rts
QA
in w
ider
pe
rspe
ctiv
e;
Aud
it fr
ame-
w
ork;
as
sess
men
t cr
iteri
a an
d in
terp
reta
tion;
le
ader
ship
; au
dit t
rails
; rol
e of
exp
erts
; ju
dgem
ent;
di
vers
ity
Aud
it fr
ame-
wor
k; r
ealis
tic
case
, exa
mpl
es
of a
udits
trai
ls;
Lite
ratu
re o
n le
ader
ship
and
on
cul
tura
l di
ffer
ence
s
2 da
ys tr
aini
ng
prog
ram
me
(sm
all g
roup
le
arni
ng);
3
times
a y
ear
Und
erst
and
audi
t fra
mew
ork;
ab
le to
lead
aud
it te
am;
awar
enes
s of c
ultu
ral
diff
eren
ces (
Net
herl
ands
and
Fl
ande
rs)
Lec
turi
ng;
grou
p di
scus
sion
s;
case
stud
ies;
D
iner
pen
sant
, Ic
e- b
reak
ers;
Fo
lder
with
ar
ticle
s and
st
udy
mat
eria
l
NV
AO
seni
or
staf
f + se
nior
ex
tern
al
cons
ulta
nt;
both
exp
erts
on
audi
ts in
HE
OA
Q
Bri
efin
g fo
r al
l m
embe
rs o
f exp
ert
pane
ls; s
epar
ate
trai
ning
for
stud
ents
Se
lect
ion
crite
ria
for
choo
sing
ex
pert
s
Fram
ewor
k of
as
sess
men
t;
role
s and
task
s of
exp
erts
and
st
aff o
f age
ncy
Leg
al
docu
men
ts;
Gui
des;
Q-
Stan
dard
s;
tem
plat
e fo
r ex
pert
rep
ort
3 ho
urs
trai
ning
/bri
efin
g fo
r al
l exp
erts
1.
5 da
y tr
aini
ng
prog
ram
me
for
stud
ents
Not
spec
ified
; kno
w a
sses
smen
t fr
amew
ork
and
crite
ria/
inte
rpre
tatio
n;
min
imal
thre
shol
ds; k
now
ro
les a
nd ta
sks o
f exp
erts
Pres
enta
tion,
di
scus
sion
s am
ong
expe
rts,
shar
ing
of
expe
rien
ces
Stud
ent t
rain
ing
inte
ract
ive
with
ro
le p
lays
,etc
.
Seni
or st
aff
/man
agem
ent o
f ag
ency
QA
A
Sam
e tr
aini
ng fo
r re
pres
enta
tives
of
all r
evie
wer
-st
akeh
olde
r gr
oups
St
rict
sele
ctio
n pr
oces
s:
nom
inat
ion
by
inst
itutio
ns,
appl
icat
ion
form
s, se
lect
ion
crite
ria.
Ass
essm
ent/
judg
emen
t pr
oces
s, ro
les
and
task
s, et
hica
l ele
men
t of
pee
r re
view
s
Rea
ding
m
ater
ial
Qui
z Pr
epar
ator
y ex
erci
ses
Onl
ine
prep
arat
ory
wor
k. 3
day
s re
side
ntia
l tr
aini
ng; n
o m
ass t
rain
ing
– tr
aine
d on
ce
assi
gned
to a
sp
ecifi
c re
view
. R
efre
sher
tr
aini
ng a
nd
CPD
.
Und
erst
and
obje
ctiv
es o
f as
sess
men
t pro
cess
and
pr
oces
s its
elf;
kno
w r
oles
and
ta
sks o
f exp
erts
, Eth
ics;
soci
al
skill
s, co
nsen
sus f
indi
ng
Inpu
ts, s
mal
l gr
oups
, si
mul
atio
n,
plen
ary
disc
ussi
on,
refle
ctio
n sp
ots,
vide
os
Exp
erie
nced
st
aff o
f age
ncy
(QA
A a
ssis
tant
di
rect
ors)
and
ex
peri
ence
d re
view
team
m
embe
rs
ZE
vA
Tra
inin
g fo
r al
l m
embe
rs o
f exp
ert
pane
ls e
xcl
.stud
ents
(the
y ha
ve b
een
trai
ned
else
whe
re)
Nat
iona
l and
re
gion
al
regu
latio
ns,
asse
ssm
ent
fram
e- w
ork,
ex
amin
atio
n re
gula
tions
, cr
iteri
a fo
r no
n-st
anda
rd st
udy
prog
ram
mes
Sta
ndar
dize
d m
ater
ial o
n re
gula
tions
(g
uide
lines
and
st
anda
rds,
tem
plat
es e
tc.).
Pr
esen
tatio
n m
ater
ial.
One
-day
-tr
aini
ngs f
or
expe
rts a
nd
stud
ents
, abo
ut
thre
e tim
es a
ye
ar.
Shor
t tra
inin
gs (2
ho
urs)
bef
ore
each
spec
ific
revi
ew.
Kno
wle
dge
of r
egul
atio
ns a
nd
eval
uatio
n fr
amew
ork;
kno
w
ledg
e in
cha
nge
man
agem
ent,
mod
ule
and
prog
ram
me
desi
gn
Exe
rts t
alks
, op
en d
iscu
ssio
n gr
oup
wor
k
(in fu
ture
e-
lear
ning
m
ater
ial,
role
pl
ays,
awar
enes
s of
cultu
res)
Spea
kers
with
ev
alua
tion
expe
rien
ces.
Ta
ble
1.
Ta
ble
1. O
verv
iew
of
cu
rre
nt
pra
cti
ce
s in
tra
inin
g e
xpe
rts
- S
um
ma
ry o
f Q
ua
lity
Ass
ura
nc
e A
ge
nc
ies
de
live
red
in S
taff
Tra
inin
g
Wo
rksh
op
!
!
Annex 2:
Preparatory material
!
!Background!and!preparatory!material!!
P a g e !|!1!
!
1st!Training!Session!for!Panel!Experts!
The!Hague!5th!and!6th!October!2011!
Background!
E(TRAIN!will!develop!materials!and!a!methodology!for!training!both!members!for!Expert!Panels!and!QA!
agency!staff!in!a!range!of!aspects!concerned!primarily!with!evaluation!at!the!level!of!HE!institutions.!It!will!include!two!main!components:!on!line!materials!and!a!‘face!to!face’!day!concerned!with!all!aspects!
of!the!evaluation!of!HE!institutions.!!The!focus!will!be!on!the!‘European!context’;!particular!attention!will!
be! paid! to! the! variety! of! institutional! evaluation! procedures! used! and! also! language,! including! terms!
that! are! frequently! employed! ! but! may! be! interpreted! in! (slightly)! different! ways! depending! on!
individual’s! own! backgrounds.! The! first! sentence! for! example! includes! two! words! –! training! and!evaluation! (! that!may! be! interpreted! in! different!ways;!whilst! training! is! a! somewhat! generic! term! it!
might,!probably!would,!!be!regarded!by!some!as!slightly!anachronistic!when!put!with!‘experts’.!It!would!
for!some!be!more!appropriate!to!speak!of!induction!within!a!European!context!(rather!than!training)!for!
those!who!have!already!gained!substantial!experience/expertise!within!a!national,!regional!or!sectoral!
role.!The!training!/!induction!will!focus!on!‘evaluation’!as!the!core!activity!since!this!is!a!generic!term!for!
the!wide! range! of! accreditation,! review,! audit,! etc!methods! that! are! used!within! different! countries,!
regions!or!sectors.!!
Purpose!of!the!day!
The!purpose!of!this!part!of!the!E(TRAIN!programme!is!to!share!experiences!and!also!to!extend!
knowledge,!understanding!and!practical!expertise!amongst!established!experts,!related!to!the!
wide! range! of! contexts! and! approaches! used! across! the! EHEA! for! the! evaluation! of! HE!
institutions.!!
As!it!is!also!the!first!such!session!and!thus!a!‘pilot’,!importantly!it!should!also!identify!the!extent!
to! which! the! contents! and! approaches! could! be! improved! for! subsequent! similar! events.!
Session!5! is! thus!perhaps!a! little! longer! than!might!normally!be!expected,!and! the!preceding!
Session!4!a!little!shorter,!in!order!to!maximise!participant’s!fed(back!on!this!pilot!day.!
Those! who! attend! the! first! session! will! have! already! gained! significant! experience! in! some!
forms!of!evaluation!but! it! is! anticipated! that! they!will! depart!with!a!much!wider!and!deeper!
understanding!(and!practical!knowledge)!that!can!in!future!be!applied!with!benefits!to!a!wider!
range!of!contexts.!!
!Background!and!preparatory!material!!
P a g e !|!2!
We!may!wish!to!consider!the!extent(s)!to!which!experts!who!undertake!the!training!/!induction!
(whether! on! line! or! in! ‘face(to(face’)!would! find! it! helpful! to! have! some! form!of! assessment!
either! ‘internal’! (that! is,! by! themselves),! or! ‘external’! ..! assessed!and!perhaps! ‘accredited’!by!
someone!else!–!but!..!by!whom!?!and!for!what!?!!
Timetable!for!the!day!
The!day,!which!requires!the!active!involvement!of!all!participants!in!all!sessions,!will!start!with!a!brief!introduction!and!then!cover!the!following:!
• an! initial! session! designed! to! demonstrate! the! different! contexts! in! which! institutional!
evaluation!takes!place!within!the!HEA,!and!the!different!purposes!and!expectations,!priorities,!
and!outcomes!!that!different!‘stakeholders’!have!of!the!quality!assurance!of!higher!education.!
• a! second! session! will! involve,! playing! different! roles,! in! some! panel! sessions.! Each! will! be!
assigned!a!role!in!either!the!evaluation!panel,!or!the!team!from!the!HEI!being!interviewed,!or!as!
panel! secretaries,! and! after! an! initial! period! roles! will! be! ‘reversed’.! There! will! then! be! a!
discussion.!!
• after!lunch!the!third!session!will!consider!the!requirements!and!dynamics!of!making!judgements!
within!a!panel,! including!consideration!of! the!different! roles!and!responsibilities! that!different!
members!have!within!an!evaluation!panel.!!
• The! penultimate! session!will! be! concerned!with! reports,! how! they! can! be!written,!what! they!
should,! and! perhaps! should! not! contain.! There! will! be! discussion! about! how! any!
recommendations! may! be! reached! and,! importantly,! how! it! may! be! best! to! present! them,!
depending!on!the!context!of!the!evaluation!and!its!consequences.!!
• The! final! session! will! involve! discussion! on! the! successes! and! failures,! the! strengths! and!
weaknesses!of!the!day’s!‘training’,!bearing!in!mind!that!this!is!a!‘first!run’!in!the!development!of!
the!overall!E(TRAIN!project.!
!Background!and!preparatory!material!!
P a g e !|!3!
!
09:00!! ! Summary!introduction!to!the!day!09:15! Session!1a! Presentation!on!participant’s!systems:!position!on!matrix!!!!!!!!!!!Annex!1a!10:15! Session!
1b!Meeting!the!different!purposes!and!priorities!of!institutional!evaluation;!!Results!of!a!survey!and!a!discussion!of!the!various!themes!!!!!!!!!!Annex!1b!
10:45! ! Break!11:00! Session!2! Role!playing!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Annex!2!12:00! ! Plenary!discussion!on!the!role!playing!13:00! ! Lunch!13:30! Session!3! Making!judgements!and!the!roles!within!the!panel!!!!!!!!!!!!!!!!!!!!!!!!Annex!3!14:30! ! Break!14:45! Session!4! Writing! reports! and!making! recommendations! and! understanding! their!
consequences!!!!!!15:30! Session!5! Concluding!session!!(including!evaluation!of!the!day!and!its!materials)!16:00! ! End!of!the!training!!
Session!1!!
will!include!two!parts!each!based!on!responses!provided!by!participants!to!material!provided!before!the!event,!see!the!exercises!in!Annexes!1a!and!1b.!!!
IT! IS! ESSENTIAL! THAT! ALL! PARTICIPANTS! COMPLETE! THESE! EXRCISES! BEFORE! THE! EVENT! and!important! that! participants! send! a! completed! return! for! Annex! 1b! ! to! the! organisers! AT! LEAST! 24!HOURS!BEFORE!THE!START!OF!THE!EVENT,!!
Participants! should! also! come,! having! looked! and! thought! about! at!Appendix! 1a,! and!be!prepared! to!identify!the!position!of!their!national/regional!/!sectoral!evaluation!system(s)!on!the!summary!diagram!AND!..!explain!briefly!(MAXIMUM!5!MINUTES)!any!important!background!context(s)and!likely!change(s).!!
Where!there!is!more!than!one!person!from!a!particular!country!it!is!hoped!that!they!will!combine!with!their!colleagues!BEFORE!the!event!to!agree!on!what!they!would!want!to!say! in! just!ONE!presentation!per! country! (apologies! to! those!who!come! from!countries!with!autonomous! regions!or! some! form!of!‘competitive!market’!but!we!don’t!really!have!time!for!more!than!5!mins!/!country!system!!!!
Session!2!!
You!will!be!assigned!a!‘role’!(student,!professor,!rector,!evaluator,!employer,!etc)!when!you!arrive,!but!this!will!only!be!‘valid’!for!the!first!part!of!Session!2;!you!will!change!roles!for!the!second!part!!But!if!you!wish! to! think!about!how!you!might!gather! information,! respond!to!an!evaluation!panel!with!different!roles!/!responsibilities,!the!topic!to!be!discussed!will!be!:!!
!Background!and!preparatory!material!!
P a g e !|!4!
How.does.the.institution.know.that.the.work.it.students.present.is.assessed.consistently.and.fairly?!!
In!other!words!–!what!internal!quality!assurance!system/!steps!does!the!institution!have!to!ensure!that!students,!in!for!example!philosophy!and!chemistry,!are!treated!in!comparable!(although!not!necessarily!exactly!the!same)!ways?!!It!may!be!tricky!for!some!of!you!but!it!is!an!integral!part!–!perhaps!one!of!the!most!important!parts!–!of!institutional!evaluation!in!some!countries.!!!
Session!3!
Material!for!Session!3!will!be!presented!to!you!at!the!meeting;!there!will!thus!be!some!‘time!pressure’!on!you!to!reach!a!judgement!–!perhaps!with!not!all!of!the!information!you!would!ideally!have!liked.!!
Session!4!
The! key! principles! involved! in! setting! verifiable! evidence! against! specific! standard! and! criteria! and!reaching!conclusions!will!be!described!as!some!techniques!discussed!for!helping!to!reach!agreements!/!concensus! in! panels.! Participants! will! be! expected! to! share! their! experiences! and! views! about! these!matters! and! about! making! recommendations,! bearing! in! mind! that! different! systems! often! have!different!‘scales’!for!judgements!and!different!requirements!for!recommendations.!The!latter!may!also!come!with!a! ‘scale’,! in! the!UK! for!example! there!are! ‘essential’!and! ‘desirable’!amongst!others.!There!can!be!‘shades!of!judgement’!within!a!judgement!!!
Session!5!..!!
fairly!self!explanatory!and!hopefully!very!valuable!to!all.!!
!
!
Annex 3:
Programme of the training
!
A G E N D A E - T R A I N S E C O N D E X P E R T S T R A I N I N G 2 3 - 2 4 A P R I L 2 0 1 2
V I E N N A Time Item
DAY 1: MONDAY 23 APRIL
9.00 Welcome and introduction
9.15 QA approaches in Europe
Helmut Konrad
10.45 Coffee break
11.00 Intercultural awareness Part 1
11.30 Role and attitude of the auditor
- Dialogue on the participants earlier experiences as auditors
- The art of asking questions
Stephan van Galen and Paul Wouters
12.30 Lunch
13.30 The audit: case work (2 groups)
- Case work and discussions
- Role play
- Lessons learned
Stephan van Galen and Paul Wouters
15.45 Coffee break
16.00 Intercultural awareness Part 2
Paul Wouters
17.30 End of day 1
19.30 Dinner
Page 2 | Agenda
Time Item
DAY 2:TUESDAY 24 APRIL
9.00 The audit team: making judgements (2 groups)
- Returning to the case
- Making and formulating judgements
- Making recommendations
Stephan van Galen
11.00 Coffee break
11.15 Evaluation of the programme
12.00 Lunch
End of experts training
!
!
Annex 4!
!
Programme of the first pilot training of QA staff
!
A G E N D A E - T R A I N F I R S T Q A S T A F F T R A I N I N G W O R K S H O P 3 N O V E M B E R 2 0 1 1
Ö A R , P A L A I S H A R R A C H , F R E Y U N G 3 , S T I E G E 3 , 2 N D F L O O R , V I E N N A
WEDNESDAY 2 NOVEMBER 2011
Time Item 19.30 Dinner
THURSDAY 3 NOVEMBER 2011
Time Item
9.00 Welcome
9.15 The QAA training of reviewers
Presentation by Gillian King, QAA UK
Discussion
10.00 Presentations on participants’ national training systems1, concerning:
- Description of experts profile (which experts are trained) - Content and programme - Preparation and materials - Conduction of the training - Learning outcomes - Didactic approach and methods - Skills of trainers
11.00 Break
11.15 Continuation of presentations
13.00 Lunch
13.30 Trainings of experts: strengths/weaknesses/recommendations
14.30 Break
14.45
15.30
Development of a “skills profile” for trainers
Concluding session
16.00 End of the training workshop 1 The following agencies are invited to give a maximum 15 minutes presentation: ACE Denmark, ANECA, AQU Catalunya, EVA, FIBAA, FHR, NVAO, OAQ, ZEvA.
!
!
Annex 5
Recommendations (final draft)
! !
Recommendations for training of experts
!Source: ECApedia!
This set of recommendations has been developed within ECA's E-TRAIN project. The recommendations are the result of staff training workshops.!
1. Good knowledge, skills and competences of experts are a key factor for success in peer review processes. QA/accreditation agencies should organise regular trainings in order to assure the needed capabilities and capacities of their external experts and to promote consistency of decisions in external QA.!
2. Training has to be objective-driven and has to take the national and international context into account .!
3. Training should be offered to all stakeholder groups represented in expert panels; experts can be trained all together or in specific programmes for specific target groups (e.g. students).!
4. Training courses should include the following learning elements: knowledge of external QA, understanding and interpretation of assessment criteria, understanding of different kinds of experts and their roles, team working, audit/review and reporting skills and soft skills (communication and social and skills).!
5. Structured preparatory material should be sent out to participants before the training; interactive communication between trainers and experts is recommended.!
6. The didactic concept of the training should encompass both presentations and case studies/discussions of real life examples of external QA procedures (opportunities for the active involvement of experts).!
7. The training should be evaluated (both evaluation of experts and feedback to experts).!
8. Through the various national trainings a community of well trained experts should be created with the option to share them on an international level.!
9. A „Train the Trainers“ programmes offered to all ECA members will ensure overall consistency.!
A regular exchange of national trainers among ECA countries will contribute to further mutual learning.
!
!
Annex 6!
!
Programme for the training days
!
A G E N D A E - T R A I N Q A S T A F F T R A I N I N G 1 0 - 1 1 M A Y 2 0 1 2
M A D R I D
THURS DAY 10 MAY 2012 Time Item
9.00 Introductions. Workshop aims and objectives
9.15 Trainer skills and attributes • Training styles • Attributes of an effective trainer • The participant’s main objectives for this workshop • The participant’s project for today • Choosing a buddy
9.45 Participant profile and needs • Who are the participants? • What do they need to know? • How will they learn most effectively?
10.45 Coffee break
11.00 From subject matter to learning objectives • How to construct an effective learning objective
12.00 From objectives to session design • How to structure and sequence training • How to decide how to deliver the training • What resources are necessary?
13.00 Lunch
13.45 Energizer
14.00 Matching training objectives and delivery methods to materials
15.30 Coffee break
16.00 Does the training need to be assessed?
16.30 It’s not over until the evaluation is done! • The training cycle • Transfer of training to the real review situation
17.00 Putting learning from this workshop into action
17.30 Action planning
18.00 Close
20.00 Dinner
Page 2 | Agenda
FRIDAY 11 MAY 2012
9.00 Introduction and presentation Outline of the workshop aims, objectives and programme
9.30 Effective communication as a key for delivering the training • What is communication? • What are the differences between verbal and non-verbal communication? • How to use communication for our advantage?
10.00 Communicational barriers • Tools for improving effective communication • Assertive communication - introduction
10.30 Coffee break
10.45 Assertive communication - practical use of the techniques
11.30 Working with difficult participants
11.50 Working with the group dynamics • Motivating the participants to be active during the presentation
12.15 Lunch
12.45 Auto-presentation and its role while delivering the training • Attributes of the good presenter • Presentation skills – practical exercises
14.15 Coffee break
14.30 Presentation skills – practical exercises Stress management:
• Mechanisms • Typical strategies • Relaxation techniques
15.30 Summary
16.00 Close
!
!
!
www.ecahe.eu
