Embed Size (px)
Citation preview
1
2
3
EU KOSVET V Development of Quality Assurance, Accreditation and Development of NQA & NQF in Kosovo
Contract No. 2009/216-809
The publication has been produced with the assistance of the European Union.The content of the publication is the sole responsibility of the contractor, PEM GmbH,
and can in no way be taken to reflect the views of the European Union.
4
5
CONTENTInternal Verification Generic Model for Assessment and Verification Internal Verification and Verifier Guidelines Selection of Assessors
Internal Auditing Recording Processes Coherent Internal Verification Sampling Strategy Sampling assessments Standardising of Assessment Judgements Monitoring assessment Verification sampling strategy Interim sampling Summative sampling Sampling plan
Experience Workload Specialist assessors Location
Standards Meeting External Quality Assurance RequirementsAnnex 1 School Internal Verification School Internal VerificationInternal Verifier Sampling form Assessor Sampling Plan Assessor Induction AgreementInternal Verifier Observation and Questioning Checklist Internal Verifier Record – Observation of Assessor Internal Verifier Record – Feedback to Assessor Countersigning report form Annex 2 Appeal Process Appeal FlowchartLearner/Candidate Appeal Report FormLearner Appeals Procedure
678889999
10111111121313141414151617181819212223242527
6
IntERnAL VErifiCaTiON
The National Qualifications Handbook provides information on how Education and Training Programmes are to be Assessed and Verified, and the important role of ensuing that these assessments of learners work is of the appropriate standard to achieve the outcomes of the curriculum and also to meet the expected “Occupational Standards” required by the Industry Sector.
This Practical guide will give additional information on the Internal Verification Process and examples of the “Forms for Evidence” that your institution will need to provide when you are Externally Verified and Monitored.
7
GEnERIC MOdEL FOR assEssmENT aNd VErifiCaTiON
ASSESSMENT AND VERIFICATION PROCESS
Candidate’s performance
MES
T
Exte
rnal
ver
ifier
sIn
tern
al v
erifi
ers
Teac
hers
/Tra
iner
s
VET
Scho
ols
/ VE
IVE
T Sc
hool
/ V
EI
External verification
Internal verification
Assessment
8
IntERnAL VERIFICAtIOn And VErifiEr GuidEliNEs
sElECTiON Of assEssOrs
iNTErNal audiTiNG
The role of the Internal Verifier is to ensure that Assessments are carried out consistently by Assessors to a level in order that learners/candidates achieve the National Standards, and the Internal Verification Process must effectively support this and provide the means to “Evidence” that is both transparent and appropriate.It is important that any process can be “audited” and that all documents for the Internal Verification Process are relevant, so the focus for the internal verifier is ensure that the school’s internal processes demonstrably meet external (Moderate/Verifier) requirements.
When selecting the team of assessors to be responsible for assessing learner/candidate performance it is important that they are the right individuals for the task, and that they have been properly trained in how to assess.
There must be a clearly documented system that is understood by all Internal Verifiers and Assessors.
It is also important that there are enough assessors to meet the needs of the learners/candidate and that the learners/candidates have equal access to assessment.
The appeals/complaints procedures must be clear and accessible to the learners/candidates. (Annex 2 Appeal Procedure)
This includes having plans in place that identify criteria for:• selecting assessors• comprehensive internal auditing and recording
processes• coherent internal verification sampling strategy
development• sampling assessments• standardising assessment judgements • implementation and evaluation of continuous
improvement.
The Annex 1 includes sample forms needed for recording the internal verification processes.
9
rECOrdiNG PrOCEssEs
samPliNG assEssmENTs
COhErENT iNTErNal VErifiCaTiON samPliNG sTraTEGy
sTaNdardisiNG Of assEssmENT JudGEmENTs
Feedback to Assessors on the assessment practice must be clear and supportive and documented. Records/reports should be relevant and appropriate and the monitoring of assessors in action should be recorded.
The assessment sampling process must be robust enough to identify:• Inappropriate practice.• Unequal learner/candidate access to assessment.• Under/over evidencing by learner/candidate.• Problem Units/Modules not being assessed.
It is important that a standardised process be in place and that the assessors are receiving appropriate timely support.
All assessments judgements by assessors must conform to an agreed standard and approach, and this should apply to programme/course levels expected for
Senior Management should be informed of any developments or issues that require their attention.
• “End-loaded” sampling taking place.• No sampling strategy in place• Unsatisfactory/inadequate recording/reporting of
assessments.
The assessment sampling strategy must be “Transparent” to ensure that that good practice may be shared and poor assessment practices eliminated.
learner achievement, and that learner/candidates have equal access to the assessment process.
10
Finally it is important that the Internal Verifier has sufficient time to manage the process!
mONiTOriNG assEssmENT
The internal verifier should aim to monitor all assessors within a given period of 6 -12 months, depending on the size and level of activity of the assessment team. They also need to select different assessment methods to monitor, the most obvious being observation of learner/candidate Performance, professional discussion and simulated exercises.
When selecting the monitoring sample the internal verifier must be aware of safety, confidentiality, security and any additional organisational requirements which could be affected by their presence. Any issues should be discussed with assessors before arranging to monitor them.
The quality and accuracy of assessment plans (review and feedback) as well as assessor records should be included in the review. The internal verifier should observe the individual assessor’s assessment and interpersonal skills, with a view to noting any areas which would benefit from advice or further development.
Following the monitoring exercise, it is important that constructive and positive feedback is given to enable assessors to develop. The discussion should be held in private and should not be relayed to the candidates or other assessors. The internal verifier should keep a record of this and ensure that It forms part of the assessor’s individual development plan.
Monitoring also creates an opportunity to liaise with candidates to ensure that their assessment needs are being properly met, in particular the internal verifier should check that learners/candidates:
• know which module they are working towards, and have a copy of the standard
• know what progress they have made towards achievement
• are aware of their rights to access unit certification• understand their role in evidence generation,
collection and cross referencing• have confidence in their assessor• understand the appeals process• have no current unmet learning needs.
11
iNTErim samPliNG
summaTiVE samPliNG
VErifiCaTiON samPliNG sTraTEGy
It is vital that the internal verifier participates in the assessment process undertaken with candidates at different stages in that process. This includes reviewing learner/candidate work perhaps (a) before decisions have been made on any unit and (b) looking at portfolios when one or two units are completed. It will entail checking the progress review reports given to candidates by assessors. This will enable the internal verifier to evaluate the quality of formative guidance on assessment and the effectiveness of candidate
Correct summative sampling should entail reviewing the quality of the assessment judgement by evaluating how assessors have reached those decisions. The internal verifier must be able to follow an audit trail which clearly demonstrates that assessors have checked that the
The internal verifier must record and report on all sampling undertaken in sufficient detail to justify their decisions. Underpinning all internal verifier activity is the validity of the verification sample. It is not a good use of internal verifier resources to maintain 100% checks on assessment decisions: it implies a lack of confidence in both assessment practice and the verification strategy.
assessment planning, review and feedback. Interim verification will also enable the internal verifier to pick up problems at an early stage and so avoid the situation of disagreeing with assessment judgments. It will also highlight individual assessor needs for support or training which in turn may be used to develop the assessment team as a whole. Similarly it provides an opportunity to identify and share good practice within the school, particularly where one or more of the assessors has wider experience.
evidence presented – whatever its format – meets the rules of evidence. This includes evidence collated via portfolio, computer-aided compilation, video/audio tape or evidence identified via an alternative assessment.
The assessment process must be left to assessors following feedback by the internal verifier. Poor practice is where sampling is interpreted as exclusively reviewing summative assessment
12
Evidence must be confirmed by assessors as:• valid – relevant to the standards for which
competence is claimed• Authentic – produced by the learners/candidates• Current – sufficiently recent for assessors to
be confident that the same level of skills, understanding or knowledge exists at the time of claim
• Sufficient – meets in full all the requirements of the standards.
The sampling strategy for each school will vary according to the needs of the school and the particular programme being delivered, but in all cases must be agreed with the external verifier. In defining the strategy all the following should be taken into account.
Factors you need to consider
Factors You need to consider
Candidates Ethnic origin, age, gender, other factors, special needs etc
Assessors Experience and qualifications, workload, occupational experience
Methods of assessment Questioning, observation, testimony, APL, use of simulation, product evidence, assignments, projects and tests
Elements within the NVQ Problem areas, special requirements
Records Reports from assessors, correct assessment practices, IV records,Learner/candidate portfolios and files
Assessment Workplace assessments, college and off-the-job training, otherassessment locations
samPliNG PlaN
To help monitor the overall sampling strategy it is strongly recommended that the School produces and maintains an overall sampling plan which will be implemented by the internal verifier. (An example of an Internal Verification Sampling Plan is included in this document, which has a dual function in that it allows schools to record their overall sampling plan and the internal verifier to comment on its implementation and any variations).
Ultimately, the sample must represent a sufficiently accurate picture of the quality of assessment for the school to be confident that those decisions not sampled also meet national standards. It is important that the internal verifier looks at the decisions of the entire assessment team in any given period (possibly on a calendar basis, or by learner/candidate cohort).
13
ExPEriENCE
If assessors are qualified/trained and experienced it may not be necessary to look at more than one or two decisions per candidate and in some sectors, not all learner/candidates. If assessors have less than 12 months’ experience, are new to either the school or out of practice, the internal verifier will need to sample substantially more of their decisions for the first 6 – 12
months. This ensures that the internal verifier reviews assessor decisions on all modules in any programme.This is particularly important where there may parts of the programme/module where evidence specifications require very particular evidence or the standards to be achieved by the learner/candidate are often misinterpreted.
Once the other sampling factors are taken into account, the sample size will almost decide itself. It is essential that the process follows this route rather than deciding on a set sample size and trying to fit a strategy to it. The sampling process must not be determined by any rule of thumb such as 10% or a square root.
The internal verifier must sample the full range of assessment methods used,e.g. direct observation of performance, witness testimony, professional discussion, questioning, assessment of portfolio evidence – products, APL, personal reports and assessment of simulated exercises.
The range and frequency of use of particular assessment methods may vary depending on the level and the skills involved. For example, Engineering or
Industrial Electronics may draw heavily on observation of learners/candidates’ performance, whereas a level in Business and Administration may primarily use products from the workplace and witness testimony. It is likely that most portfolios will contain a mixture of evidence, and therefore assessment activity and decisions. The internal verification sample must take this into account, and it is clearly imperative that the internal verifier knows the subject thoroughly and the likely range of assessment methods and evidence sources that could be used.
All assessors should be included in the sample, but a number of other factors must be considered:
WOrklOad
The internal verifier needs to look at the workload of each assessor so that the total sample also fairly reflects the number of decisions being made by individual assessors. Increasing ratios of candidates to assessors may also indicate assessment problems.
This requires the attention of the internal verifier as it can result in diminishing the quality of assessment. In these cases, sampling should be increased.
14
sPECialisT assEssOrs
It may be that some assessors may only assess certain units in a programme/module such as specialist areas, e.g. on first aid or energy management. In some schools there may be the opportunity of some assessors carrying out observations; whilst others look at other types of evidence and this can be a highly effective
use of assessment time and expertise. In these cases, once the internal verifier is satisfied that assessor judgements are consistent, it is reasonable to reduce the number of their decisions included in the overall sample.
lOCaTiON
The location and type of assessor is also a factor in deciding the sampling strategy. In some centres, assessment is widely dispersed, both in terms of geographical location and type of employer at which the learner/ candidates may be located. Some schools may have satellite centres (organisations which link to the main school to facilitate access to certain modules of the programme, but who may have varying approaches to assessment).
The internal verification sample must capture a picture of assessment across the whole school; in effect the strategy should be repeated for all assessment sites.
If these arrangements appear too complex, then the strategy needs to be carefully thought through, and it is a good idea to discuss it with the external verifier who can offer advice about the frequency of sampling across the assessment sites.
Some assessors, often in training providers, are peripatetic and visit across a number of assessment sites. And the internal verifier must look at the pattern of assessment and take this into account in their sample.
sTaNdards
The sample also needs to take into account the standards themselves. In some qualifications, there is a lack of clarity – most frequently in the criteria or the evidence specifications – which, unless fully discussed by the assessment team, inevitably leads to confusion and differing assessment decisions. Clearly, therefore, it is imperative that both assessors and
internal verifiers know the standards well, and also agree their interpretation of aspects such as evidence specifications, range and knowledge and understanding requirements through standardisation activities.Equally, there may be changes to assessment requirements when standards are revised.
15
MEEtInG ExtERnAL QualiTy assuraNCE rEQuirEmENTs
The internal verifier also has a wider management role, which is to provide a link between their school/institution/organisation, and the regulatory bodies, such as MEST/NQA. The External Moderator/ Verifier’s first port of call is the internal verifier, who will be expected to have the following range of information to hand:• numbers of current registered candidates per N/SVQ
and level• candidate centre enrolment and awarding body
registration details• assessor details – specifically CVs, CPD records,
assessor qualifications, development plans and workloads
• candidate progress reviews and achievements, special assessment requirements and equal opportunities monitoring information
• details for assessment records, plans, reviews and tracking sheets
• available candidate support resources• details of assessment sites• details of any other assessment site arrangements• the internal verification sampling strategy• internal verifier records including feedback to
assessors, discussions with candidates• records of claims for certification• candidate portfolios• minutes of internal verification/standardisation
meetings• procedures for storage and retention of centre
records.
Creating and maintaining good information channels is particularly important following an external moderator/verifier visit. In some schools/organisations any action points are agreed with a programme co-ordinator or quality manager, often the internal verifier or Internal Verifying Co-ordinator may take this role.
Individual internal verifiers may well have the responsibility of ensuring that they communicate pertinent parts of the external verifier’s report to those assessors for whom they are responsible. It is important that action points raised by the external moderator verifier are acted upon within the timescales specified.
Of equal importance is the need to maintain security of learner/candidate files and their assessment records. Portfolios for candidates who have been certificated in between external verifier visits must be kept intact until the next visit so that they can be examined or produced for inspection purposes if needed.
Original assessor records must be retained for at least three years and be made available for external verification and to allow for any appeals or complaints to be progressed and resolved..
16
AnnEx 1 sChOOl iNTErNal VErifiCaTiON
Module/Qualification Achievement Learner RecordLearner/Candidate Name
Enrolment Number?
School Assessor Qualification Level Modules Achieved
Full Qualification Achieved
Qualification(s) Co-ordinator please enter the above information for the Internal Verification Records Signed...............................Assessor Date.................................
Signed.............................Internal Verifier Date.................................
17
SChOOL IntERnAL VERIFICAtIOniNTErNal VErifiEr samPliNG fOrm
For Period........................................... Qualification........................................... Assessment/Outcome Number...................................................................Internal Verifier.................................................................................Assessor Assessor
StatusAssess-ment site
Learners/ Candi-dates
Outcome to be Assessed: Out-comes planned to be sampled and any variation should be should be recorded below using an appropriate format e.g. bold, italics, or shaded box.
Method of assess-ment/evidence source checked
Part of assessment process reviewed.
Assessor status
T Trained
NT Not Trained
NST New Staff
Trained
NSNT New
Staff Not
Trained
Key
(Schools will need to devise their own key and enterbelow)
Method of assessment/evidence source checked
DO - Direct observa-tionsWT - Witness testi-monyAPCS - Assign-ment/Project/Case study/RPL - Recognition of PriorLearningQ - QuestionsWP - Work ProductEWT - Expert Wit-nessTestimonyS - Simulation
Assessmentprocess
P – PlanningR – ReviewF – FeedbackFC – Final check
18
ASSESSOR samPliNG PlaN
ASSESSOR iNduCTiON aGrEEmENT
Assessor Assessor Status
Assessment Site/Location
Candidate/Learner
Assessor name
Telephone number E-mail
The qualification title and level that I will be assessing is
School name and contact details
Address Telephone
School Accreditation Number
My countersigning assessor/IV is Telephone
19
IntERnAL VERIFIER ObSERVAtIOn And QuEsTiONiNG ChECklisT
My internal verifier is Telephone
The School Quality Manager is Telephone
The School External Moderator/verifier is
I have completed the induction/training programme and am now fully familiar with myresponsibilities and role as an Assessor.
I understand the procedures for checking the standard of my assessment practice and for seeking immediate help, should either I or my learners/candidates experi-ence difficulty.i.
Internal verifier name........................................ Module(s) reviewed............................... Assessor name................................................................
Learner /Candidate name(s)............................................
20
DID THE ASSESSOR Yes No EvidenceDevelop and agree an assessment plan with the learner/candidate(s) using appropriate assessment methods?Agree when assessment will take place with the learner/candidate(s) and other people involved?Agree arrangements with the learner/candidate(s)for reviewing progress against theassessment plan?Review and update assessment plans to take account of learner/candidate achievements?Give the learner/candidate(s) feedback at an appropriate time and pace?Give feedback in a constructive and encouraging way?Give feedback to the learner/candidate(s) which mettheir needs and was appropriate to their level of confidence?Clearly explain the assessment decisions?Can the assessor explainHow to access the centre appeals and access to fair assessment policy?How to identify and gain resources to support learner/candi-dates who have special assessment needs?Assessor Signature....................................................................... Date..........................................Internal Verifier Signature............................................. Date..........................................
21
IntERnAL VERIFIER RECORd – ObsErVaTiON Of assEssOr
Assessor: Internal verifier:Module/Level Date of ObservationProgramme: Site:Did the Assessor: CommentsProvide clear information, to the learner/candidate, on the assessment process? encourage the learner/candidate to identify and pre-sent evidence?assess the learner against only the specifieduse the standard assessment documentation?remain as unobtrusive as was practicable?use questions which are clear and do not lead the learner? use sufficient questions to cover the required knowl-edge and to be able to infer competence?communicate at a level appropriate to the learner?Involve the learner in the assessment process?Confirm success to the learner promptly and as soon as the evidence is sufficient?Encourage the learner to ask questions?Give feedback as soon as possible?Record evidence and assessment decisions promptly and in accordance with requirements?General Comments
Action Required: By When:Assessor Signature: Date:Internal verifier’s Signature: Date:
22
IntERnAL VERIFIER RECORd – fEEdbaCk TO assEssOr
Internal Verifier: Assessor:
Candidate: Module/Element
Please state whether Formative or Summative Sam-pling
Comments and Discussions
Action Points Date Actioned
Assessor Initials
Assessor’s Signature Date:
Internal Verifier Date:
Action Points Completed Internal Verifier’s Signature Date:
23
COuntERSIGnInG rEPOrT fOrm
Assessor name
Countersigning assessor/IV name
Learner/Candidate name(s)
Unit numbers/titles and level
Check and countersign assessment decisions made in each unit by the unqualified assessorensuring that all the assessment methods used are checked in rotation. The assessment process should also be checked.
Method(s) of assessment and evidence sources sampled (please tick)
Metoda (t) e vlerësimit dhe mostrat burimeve të dëshmive (ju lutemi vëni shenjën)Direct observation □ □ Expert witness testimony □ □ Witness testimony □Work product □ □ Simulation □ □ APL □Assignment/Projects/Case Study/Candidate/Reflective accounts □Questions/Professional discussion □
The evidence is (please tick or cross as appropriate):Valid □ □ Authentic □Current □ □ Sufficient □
Is there evidence of consistent practice?Yes □ □ No □
Part of assessment process checked (please tick or cross as appropriate):Planning □ □ Reviewing/feedback □Recording □ □ Judgement □
Evidence is correctly (please tick or cross as appropriate):Recorded □ □ Referenced □
Signed/dated by untrained assessor and the learner/candidate □
24
AnnEx 2 aPPEal PrOCEss aPPEal flOWCharT
Teacher judges learner’s assignment
Learner disagrees with assessment decision
Assessment decision recorded and results filed in the Programme file for the final evaluation and external verification
If acceptable by the learner If the candidate is still not
satisfied the evidence and all relevant documentation should be passed to an independent third party – ( a suitably qualified person other than the teacher or the internal verifier) within the school or from another school offering the same programme. The decision of the independent third party is final (STAGE 3)
The decision of the internal verifier will be recorded and communicated to the candidate within five working days
If the learner is still not happy he/she may appeal to the internal verifier (STAGE 2)
Learner disagrees with assessment decision
Learner discuss concerns with the teacher
Teacher explains the assess-ment decision to the learner
If the assessment decision is still not acceptable to the learner the recording should be made on the nature of the appeal
If the learner is still not happy he/she may appeal to the internal verifier
25
LEARnER/CAndIdAtE aPPEal rEPOrT fOrm
Learner/Candidate name Assessor’s Name Internal Verifier name
STAGE I – Assessor Decision
Assessment Details
Programme
Module / Learning Outcome
Assessment Method:
Original Assessment Decision:
Learner’s/Candidates Appeal:Learner’s/Candidates Signature:
Date:
This column to be completed by school staffAssessor’s Decision:
Signature:
Date:
Date Appeal received:
Date replied to Learner:
Signature:
Date:
This column to be completed by school staffLearner’s/Candidates Decision:
I accept the teacher’s decision □ □I wish to proceed to Internal Verification □ □(STAGE 2)Signature:
Date:
Date Reply received:
Date forwarded to Internal Verifier:
Signature:
Date:
26
STAGE 2-Internal Verifier Decision
Internal Verifier’ CommentsInternal Verifier Decision:
Signature:
Date:
Date Candidate Informed
Signature:
Date:
Candidate Decision
I accept the Int. Verif. Decision □
I wish to proceed to STAGE 3 □
Date reply received
Date forwarded to third party
Signature
STAGE 3 -Third Party Decision (MoE)
Third Party’s CommentsDate Appeal Decision Received:
Date Learner informed:
Name:
Signature:
Date:
Learner Response
I have received the decision of the third party
Signature:
Date:
Date reply received
Signature:
Date:
27
LEARnER aPPEals PrOCEdurE
28
29
