©Velg Training 2013

AIS NSWVET Network Day

15th July 2013

Tamara SimonVelg Training

Overview

• Regulation 1• Key areas of non-compliances2

• Role of the RTO3

• Continuous Improvement4

Overview

• Training and Assessment Strategy (TAS)5

• Industry Consultation6• Record Keeping7• Assessment8

1Regulation

What’s your biggest challenge in delivering

VET in your school?

RTO Compliance - VQF

VET Quality Framework

Standards

Fit and Proper Person

Financial Viability

Data Provision AQF

ASQA exercises its responsibilities based on:

• a robust framework of legislation and standards

• a risk-assessment focus

• active engagement with industry

• a rigorous audit methodology

• fairness and transparency

• promoting informed consumer choice

• accountability

• accessibility

ASQA’s approach to regulation

Key areas of non-compliance 2

• Generic strategies for training and assessment

• Lack of industry consultation to inform strategies

• Poor assessment tools and processes

• Lack of effective validation of assessment

• Currency of staff competencies

• Lack of systematic processes – data collective and

analysis

• Lack of data collection and analysis on client services

• Evidence of meaningful improvements

• ‘Torpid’ policies and procedures

Common issues at audit

School non-compliances

Qualification has been issued and school has phoned Board to issue different (incorrect) credentials

TAS outlines onsite and offsite access to facilities and resources but incorrect

or don’t have them

Old units of competency listed on marketing material issued to students

and parents

Changing teachers to deliver VET qualification and not notifying AIS NSW

– teacher doesn’t meet SNR 15.4

AIS NSW Findings

School non-compliances

School issuing their own qualifications/units of competency

to students

Assessment instruments not valid

Teachers not following policies and procedures

No evidence of validation and moderation

AIS NSW Findings

Role of the RTO

AIS NSW3

AIS NSW

• They are the RTO – not each school

• Risk is all with them

• Partnering agreements

• Notify of changes and gain approval

• 1 school can de-register the RTO

• Documented agreement

• Register of partnering arrangements

• Evidence of monitoring

Partnering arrangements

4Continuous Improvement

Training and Assessment

Client Services

Management of the RTO

Continuous Improvement

Is there a difference?

Follow Up

Regular Internal Reviews/Audits

Examples

Evident and Documented

Part of the culture

Continuous Improvement

5Training and Assessment

Strategy (TAS)

• Need to be documented – may be made

up of a number of documents

• Need to reflect what actually happens

• Need to reflect current training package

• Include evidence of industry

consultation

• Need to include details of

monitoring/improvements

Training and assessment strategies

Training and Assessment Strategy inclusions

Qualification and core and elective units

Clients/learner needs

Prerequisites/co-requisites Licensing/regulatory requirements

Entry requirements/course duration

Mode of delivery/delivery strategies

Assessment strategies/evidence gathering techniques (including RPL & reasonable adjustment)

Delivery and assessment staff requirements

Resources/infrastructure requirements

Assessment validation process

Pathways Industry consultation

6Industry Consultation

SNR 15.2

Strategies for training and assessment meet

the requirements of the relevant Training

Package or VET accredited course and have

been developed through effective

consultation with industry.

What the Standards say?

Hmmm….

What does ‘effective’ industry consultation

mean?

How do we do this?

How many industry

organisations?

How many ‘types’ of

organisations?

How many people

within each organisation

?

Whom should you consult?

If only there was a…..

1• Gain industry support

2• Ensure industry needs are met

(employability)

3• Provide updates on regulations or other

industry relevant legislation

4• Develop knowledge of broad range of

industry practices

5• Equipment and machinery standards

6• Assist with placing students in the

workplace

Industry consultation is needed:

Industry……

Employs the graduates

See the results in the workplace (ROI)

See what’s working and what’s not working

Can make suggestions for improvement

Can provide invaluable feedback

Industry Skills

Council

IndustryAssociatio

n

IndustryNetworks

Industry in your

local area

Employers

Who should you consult?

What’s next… document it!

• Telephone\Face to Face conversations• Emails• Questionnaire

• Organisation• Name• Position

• What was discussed?• What was confirmed?• What was suggested?• What was amended resulting from

feedback?

Training program is 12 months long

3 week intensive every 3 months

Training topics will be…

Test their skills

Knowledge test

Practice in the workplace

Workplace supervisor signs off

to say ready

Workplace Assessment

Skills/Tasks they will be able to do are

- ????- ????- ????

Do you think that sounds like the skills

you need in your workplace?

It’s about how we consult

AssessmentDuration

Content

Tasks

Skills

Knowledge

Industry Consultation

• Consultation with industry is about seeking

advice, not direction.

• The RTO is answerable only to the candidate,

the Training Package, the AQF and the VQF.

Remember…

7Record Keeping

Relevant Standards

SNR 23.3

The NVR registered training organisation must

retain client records of attainment of units of

competency and qualifications for a period of 30

years.

Records Management

Retention requirements for completed student

assessment items

An RTO is required to securely retain, and be able to produce in full

at audit, if requested to do so, all completed student assessment

items for each student, as per the definition, for a period of six

months from the date on which the judgement of competence for the

student was made.

ASQA General Direction

funding or Board of Studies may incur

further requirements over and above this

UPDATED20 February 2013

AIS NSW RTO

Documents

Compliance

Documents

VET Offerings

VET Course

Documents

VQF Publication

s

Overall Management of Documents

8Assessment

Relevant Standards

SNR 15.5

Assessment including Recognition of Prior Learning (RPL):

a) meets the requirements of the relevant Training Package

or VET accredited course; and

b) is conducted in accordance with the principles of

assessment and the rules of evidence; and

c) meets workplace and, where relevant, regulatory

requirements; and

d) is systematically validated.

Assessment

• Meets the requirements of the Training Package or

accredited course

• Is consistent with the training and assessment strategy

• Is valid, reliable, flexible and fair (principles of

assessment)

• Focuses on the application of knowledge and skill to the

standard of performance required in the workplace

• Involves the collection of sufficient, valid, authentic and

current (rules of evidence) to enable a judgement to be

made about whether competency has been attained

• Confirms that workplace and regulatory requirements

are met.

• Assessment is validated.

Guide to compliance - Assessment

OR NOT TO

That is the question!

Assessment

Assessment

Validation Moderation

10 Things to look for - Assessment!

1. Assessment of the whole unit2. Mapping of the assessment tools3. Assessing at the correct AQF (& ACSF) level4. Evidence of validation5. Assessment tools with clear instructions6. Assessment tools with clear marking

guides7. Appropriate observation tools8. Assessors using the guides9. Assessment tools that cater for learners

with special learning needs10.Evidence of transitioning

1. Assessment of the whole unit

Does the assessment evidence address all of the unit requirements?

The performance

criteria

This is a test of the (content) validity of

the assessment tools and the assessment

process.

The required skills and

knowledge

The critical aspects for assessment

• Are actual questions mapped?

• Are specified sections of tasks mapped?

• Are observable behaviours mapped?

2. Mapping of the assessment tools

Are components of the each of the assessment tools mapped to relevant unit

components?

This tests the (content) validity of the assessment tools

For example:

2. Mapping to the PCs

2. Mapping to the knowledge, skills and critical aspects

3. Assessing at the correct AQF level

Basic factual and procedural knowledge.

3. Assessing at the correct AQF level

e.g. AQF Level 2

NOT

Write a 1000 word essay.

Conduct research on this topic

Design next week’s menu for this restaurant

NOT:• analyse• distinguish• examine• compare• contrast• investigate• categorise• identify

3. AQF Level 2

Significance of Verbs – AQF Levels 1 and 2

This tests the (construct) validity of the assessment tools

VERBS such as:• tell• list• describe• relate• locate• write• find• name

3. Consider LL & N needs

What language, literacy and numeracy activities would be required in the

workplace?

• oral or written reports• forms, instructions,

labels, reports, etc.• measuring,

scheduling, etc.• etc., etc., etc.

Have we checked the LL&N skills:• required in the workplace?• of our students?• of our trainers / assessors?

Have we taken account of these in:• the teaching materials?• the assessment activities?

This tests the (face and content) validity and the fairness of the assessment tools.

It also alerts us to the significance of the Australian Core Skills Framework (ACSF).

4. Evidence of validation

Assessment tools must be validated with industryDo they make sense to people engaged in the industry?

Evidence such as:• Minutes of meetings• Diary records of discussions• Email confirming consultation• Etc., etc., etc

:

There should also be evidence that the assessment tools

have been validated internally.

This tests the (face) validity of the assessment tools

5. Assessment tools with clear instructions

There should be instructions for the student.

• WHAT has to be done• WHERE• WHY and HOW

There should also be instructions for the trainer.

• Under what conditions – duration; environment etc?

• What resources?• What support is

allowable?

This tests the reliability of the assessment tools

6. Assessment tools with clear marking guides

• What are the “expected” responses?• What constitutes a “satisfactory”

response?• Are “product” requirements spelled out?

This assists the reliability of the assessment tools and their fairness.

• Is each item directly observable?

• Is each item focussed on one behaviour?

• Are there descriptors of what constitutes satisfactory

behaviour?

• Is there advice for the assessor about what to do if all

items are not observed?

7. Appropriate observation tools

Do observation tools turn unit requirements into observable behaviours – not just a cut and paste from the PCs?

This tests the validity and reliability of the observation assessment.

8. Assessors using the marking guides

• Is there evidence of consistency among assessors?

• Is there evidence of rigour in the assessment process?

• Is there evidence of assessors making judgements and signing off on these?

• Is there evidence of the provision of feedback to students that takes account of the marking guides?

This tests the reliability and fairness of the

assessment process.

9. Assessment tools that cater for learners with special learning needs

Do the tools allow for some learners

to be assessed differently from

others in the group?

E.g.• Allowing a learner to

respond orally rather than in writing

• Allowing a learner more time or access to a particular resource.

Do the assessment tools define “reasonable adjustment”?

This demonstrates the flexibility and fairness of the assessment tools and processes.

10. Evidence of transitioning

When the Training Package has been updated or replaced, is there evidence that required transition processes are in place?

E.g. • Different Training and Assessment Strategies for

New and Continuing cohorts• Awareness of new ASQA “teach out”

requirements

Summarising – 10 Things to look for!

1. Assessment of the whole unit2. Mapping of the assessment tools3. Assessing at the correct AQF (& ACSF) level4. Evidence of validation5. Assessment tools with clear instructions6. Assessment tools with clear marking

guides7. Appropriate observation tools8. Assessors using the guides9. Assessment tools that cater for learners

with special learning needs10.Evidence of transitioning

How to avoid non-compliances

Strong Industry Relationship

Compliance

Quality Outcomes

It isn’t only about an audit…

Any questions?