Classification Manual Manual for formulating learning ... · taxonomy codes. Annexes 1 and 2 set out two taxonomies, namely, Bloom’s taxonomy and Romiszowski’s taxonomy. These

Classification Manual

Manual for formulating learning outcomes and verifying NLQF levels

Colophon

Title

Authors NCP NLQF: Tijs Pijls, Elle Langens

Version 1.2

Date August 2014

National Coordination Point NLQF

Tel: +31(0)73-6800784 www.nlqf.nl

© Nationaal Coördinatiepunt NLQF 2014

No part of this publication may be duplicated and/or published via printing, photocopying, or any other method without obtaining advanced, written permission from the publisher.

Contents

Introduction ........................................................................................................................ 1

1 Formulating learning outcomes ............................................................................. 3

1.1 What is a qualification? ....................................................................................... 3

1.2 Learning outcomes ............................................................................................... 4

1.3 Advantages of formulating learning outcomes .............................................. 4

1.4 Formulating learning outcomes and tips ......................................................... 4

2 Verifying NLQF levels ................................................................................................. 7

2.1 Verifying the NLQF level...................................................................................... 7

2.2 Suggestions for verifying NLQF levels ............................................................. 9

Annex 1: Bloom .................................................................. Fout! Bladwijzer niet gedefinieerd.

Annex 2: Romiszowski ...................................................... Fout! Bladwijzer niet gedefinieerd.

Annex 3 Other taxonomies......................................................................................................

Annex 4: Recommended and discouraged verbs .......... Fout! Bladwijzer niet gedefinieerd.

Handleiding Inschaling - www.nlqf.nl – [email protected] - Augustus 2014 1

Introduction

This Classification Manual has been created by the National Coordination Point NLQF

(hereafter referred to as: NCP NLQF) with the aim of supporting organisations in

formulating learning outcomes and verifying NLQF levels for the purposes of submitting

classification requests.

As a step towards classification, applicants must first perform a self-evaluation related to

the level of their qualification. The Leijnse Commission set out the basic principle that

qualifications must be transformed into learning outcomes as the basis for classification1.

The manual comprises two sections:

• Part 1: Formulating learning outcomes;

• Part 2: Verifying an NLQF level on the basis of the NLQF descriptors.

The theory is set out for each section, followed by examples and tips.

In the manual, the following terms are important:

Qualification The result of a validated assessment process conducted by a

competent body, which establishes that the learning

outcomes fulfil the previously determined standards.

Occupational profile A professional competency profile sets out the professional

tasks and knowledge / skills necessary to carry out a

particular job. The profile is drawn up on the basis of an

employee who has been carrying out the job for several

years.

Learning outcomes Definition of what somebody knows and can do after

completion of the learning process.

In the NLQF, learning outcomes are set out in terms of

knowledge, skills and independent responsibility, all within a

fixed context.

Taxonomy A taxonomy is a systematic classification in which rank plays

a role.

Descriptor Brief description of the minimum knowledge, insights and

skills that a learner must have acquired at the end of the

learning process.

Context Environment within which someone undergoes the learning

process or carries out the job. The context description of the

levels, together with the defined knowledge, are determining

for the difficulty level at which the skills must be

demonstrated.

1 Commission NLQF – EQF. Advice Commission NLQF – EQF. Introduction of the Dutch National Qualification framework NLQF in a national and European perspective. May 2011

http://www.nlqf.nl/

mailto:[email protected]


Knowledge Knowledge is the totality of facts, principles, theories and

working methods that are linked to a profession or area of

expertise.

Skills Cognitive skills (logistic, intuitive and creative thinking) and

practical capacities (psycho-motor skills in applying methods,

materials, resources and instruments) in order to:

reproduce, analyse, integrate, evaluate, combine and

apply knowledge in a professional or knowledge-based

setting;

recognise and resolve problems;

develop oneself, independently or with guidance;

obtain, gather, process, combine, analyse and assess

information;

communicate on the basis of the acceptable conventions

in the context;

all within a given context.

Autonomy and

responsibility

Proven ability to work with others and to take responsibility

for own work and study results and/or those of others.

http://www.nlqf.nl/



1. Formulating learning outcomes

1.1 What is a qualification?

A ‘qualification’ means a formal outcome of an assessment and validation process which is

obtained when a competent body determines that an individual has achieved learning

outcomes to given standards2. A qualification, thus, concerns the totality of learning

outcomes. Learning outcomes define what somebody knows and can do after completion

of the learning process.3

Qualifications form the basis of the curriculum for training, teaching and learning

methods, assessments, RPL (recognition of prior learning), assessments and examination

instruments.

The diagram below shows how job descriptions, occupational profiles, qualifications and

training should ideally relate to one another.

Functie-omschrijving

BCP KwalificatieOpleidenExamenEVC

Job Description Occupational Qualification Courses

Profile Exams

RPL

The above diagram assumes a model situation.

A job and the corresponding job description are established by a company, often on the

basis of collective labour agreements. The competencies that relate to a specific

profession can be represented in an occupational profile. An occupational profile often

consists of multiple roles. It depends on the sector whether a profile is set up or not. If a

profile is developed, then it is drawn up by the sector.

A qualification is a description of all of the learning outcomes that a person must have

achieved at the end of a course, learning process or assessments. It can be based on a

competency profile, but this is not always the case. The qualification can be developed by

a centre of expertise, sector, trainer or a company (often in consultation with social

partners). The qualification forms the departure point for curriculum development

(training), learning methods, assessments and assessment instruments.

2 &3 Commission NLQF – EQF. Advice Commission NLQF – EQF. Introduction of the Dutch National Qualification framework NLQF in a national and European perspective. May 2011

http://www.nlqf.nl/



1.2 Learning outcomes

For classification in the NLQF, a qualification must be formulated in terms of learning

outcomes.

A learning outcome clarifies what someone knows and can do on completion of a learning

process. This could be a learning process within education (formal and non-formal) or an

unstructured learning process at work or outside a work environment (informal learning).

The learning outcomes must therefore be formulated independently of the learning

pathway.

Learning outcomes can be created in various ways. One of these methods is the use of

taxonomy codes. Annexes 1 and 2 set out two taxonomies, namely, Bloom’s taxonomy

and Romiszowski’s taxonomy. These taxonomies can serve as tools for formulating

learning outcomes.

1.3 Advantages of formulating learning outcomes

The big advantage of (defining) learning outcomes is that it clearly sets out what the

learner has to know, do and prove in order to gain a qualification. Learning outcomes are

therefore more precise, easier to define and clearer than objectives.

Learning outcomes create a form of ‘common currency’ making qualifications more

transparent and easier to compare with one another (nationally and internationally).

In addition, learning materials and working methods can better connect into the context

in which the participant is working and applying his / her knowledge. What exactly does

the individual have to do with the knowledge? Know it by himself / herself (as

background), use it himself / herself, or pass it on / explain it to others?

1.4 Formulating learning outcomes and tips

A good learning outcome contains the following three components:

a. a behavioural

component

process verb (observable and assessable behaviour)

b. a content component with respect to which topic / subject the action / process

must be carried out (what)

c. the context concretisation of the behaviour that must take place (how)

An example of a learning outcome with these three components:

Treats burns victims according to First Aid guidelines

(Behaviour) (Content) (Context)

http://www.nlqf.nl/



When formulating learning outcomes, the following tips may be helpful:

Choose verbs that represent observable and assessable behaviour (see list of

recommended and discouraged verbs in Annex 3);

Define the desired learning benefit as precisely as possible (avoid terms such as:

only, correct, suitable);

Avoid vague terms such as ‘know', 'understand', 'learn', 'be familiar with' and 'be

aware of. These terms refer more to the topic of learning than to learning outcomes;

Avoid complex sentences. If necessary, use additional sentences to maintain clarity;

Choose a specification level that is clear to developers of courses and of assessments

but not so specific that the flexibility is compromised;

Introduce differentiations between learning outcomes for the various levels,

expressed in terms of:

- choice of subject

- content

- level of ability

- context

Example of a poorly formulated learning outcome

“Has an insight into the various pension systems and can justify choices for a pension

system.”

Behaviour: have insight into and provide justification

The use of the word ‘insight’ is intangible. When does a person have “insight”? And on which

level is s / he expected to have this insight? Does this just concern being able to provide

justification or must this insight lead to other things?

This learning goal does not refer directly (at least not the first part) to what the learner

must be able to show in terms of behaviour after completing the learning process. This will

be more concrete and tangible if it is formulated into a learning outcome.

This learning outcomes refer to components of the training programme and an intention of

the training programme. The intention is that the learner has an insight into the various

pension systems. How and at which level this can be expressed cannot be determined from

this learning goal. One of the consequences is that the expected insight must lead to being

able to provide justifications.

Contents: various pension systems

Context:

Example of a well formulated learning outcome

“Can compare and distinguish the various pension systems and can, on this basis, provide

justification (written and verbal) for the choice of a pension system within the financial

sector.”

Behaviour: Compare and distinguish and provide justification (written and verbal)

Contents: various pension systems

Context: Financial sector

Where is the difference? Behaviour component, content component and it is measurable

There is more focus on the concrete/visible behaviour of the learner. This is also easier to

measure during checking or when designing exams. In addition, this formulation provides

an indication of the level which the knowledge must reach, i.e. compare (understanding)

and distinguish (analysing).

http://www.nlqf.nl/



- extent of autonomy and / or pace;

Avoid explanations;

Ensure that the learning outcomes fit within a module of the overall result for a

qualification;

The learning outcomes must be visible and measurable;

Ensure that the learning outcomes can be tested and proven;

In defining a learning outcome, the period/timeline within which the outcome must be

achieved must be borne in mind. There is always a risk of exaggeration. The question

of whether it is realistic to achieve the learning result with the available time and

resources must always be answered;

The method for establishing whether the learning outcome has actually been achieved

must be borne in mind. Too broad could mean that it is difficult to prove the result.

Too narrow could mean that the list of learning outcomes is too long and detailed.

Check to ensure that the defined learning outcomes are recognisable (for example,

among colleagues, former students, and the field of work).

http://www.nlqf.nl/



2 Justifying NLQF levels

In order to classify a qualification on the NLQF, an applicant must verify, via self-

evaluation, that the qualification-based knowledge, skills and responsibility and autonomy

levels (set out in the learning outcomes) match the requested NLQF level.

The eight NLQF criteria together set out within which context and with which type of

knowledge a person, at a particular level, can apply knowledge, solve problems, develop

himself / herself, obtain information and communicate and how much responsibility the

individual can take on.

The diagram below indicates which criteria are applied.

Criteria

Context

Knowledge

Skills

Applying knowledge

Problem-solving skills

Learning and development skills

Information skills

Communication skills

Responsibility and autonomy

The criteria are elaborated into descriptors at every level. In the verification, a connection

must be made to the descriptors and an argument made as to why the qualification can

be classified at the requested level.

In this verification, reference should be made to relevant evidence and documentation in

this regard should be included in an annex to the application.

2.1 Verifying the NLQF level

Every criterion, for every level, has a definition, i.e. the descriptor. In the verification it is

necessary to demonstrate a link between the learning outcomes for the qualification and

the descriptors for the requested level.

When drawing up the substantiation, the following steps should be taken:

1. General: scan the qualification for key words from the NLQF descriptors. Which NLQF

level is appropriate, in the applicant’s opinion?

http://www.nlqf.nl/



2. Specific: verify the applicable NLQF level per criterion. In the verification of the

various NLQF criteria use (several) learning outcomes that connect into the level.

Justify why it is at this level rather than a higher or lower one. For example: NLQF

Level 4 is requested for a qualification, so it is necessary to justify why Level 4 rather

than Levels 3 or 5 has been requested in relation to each criterion. .

3. Use examples, e.g. from the qualification, the curriculum, the assessments, the

instrument mix, RPL procedure, etc. The examples used confirm the level indicated

during Step 2

4. Refer, in these examples, to concrete and numbered annexes in which the examples

are provided.

The box below contains an example of verification. There are, of course, other options for

formulating this verification.

Under the box, there is also an example of evidence which indicates which learning

results from the qualification correspond to specific descriptors.

Example of verification for NLQF Level 4, Applying Knowledge:

Apply descriptor for knowledge, Level 4:

Reproduces and analyses knowledge, evaluates and integrates data and develops

strategies for carrying out professional tasks. Analyses reasonably complex professional

tasks and carries them out.

Verification

During his work, XX is not only able to reproduce knowledge but also to analyse and

apply it. Even though there are only 5 key terms l under this descriptor for Level 4, they

are very broad terms within which the candidate is expected to use the entire spectrum of

knowledge, in every situation. To do so, it is vital that XX constantly analyses which

knowledge is most suitable and in which manner s / he can approach this.

For the learning goals and key terms that correspond to Level 3, a note must be inserted

by the level definition for mathematical skills (5 key terms). Specifically, for key terms

34.2 and 37.1, it is debatable whether or not these must be placed at Level 4 because a

certain degree of analysis is required in order to gain this result. Ultimately, we have

chosen to link these terms to Level 3 because reproducing and applying play a greater

role than analysing.

Even though the learning goal and key terms are generally found at Level 3, the skills

reproducing and applying might be necessary for skills that are found at Level 4. Without

the skills at Level 3, the skills at Level 4 are not achievable. It is technically and

practically unfeasible to cover everything when checking the key terms at Level 4. The

fact that they are, however, essential skills here means that the key terms are checked at

Level 3.

http://www.nlqf.nl/



Level 5 includes the addition that reproducing, analysing and applying must take place in

contexts other than the primary context. This is not the case for XX.

Reference to specific annexes:

C- Selecting learning goals and key terms in the context of Applying Knowledge, Level 4.

D- Selecting learning goals and key terms in the context of Applying Knowledge, Level 3.

Example Annex: selecting learning goals and key terms in the context of Applying

Knowledge, Level 4.

Name of qualification

Professional competency profile XX

Learning results, key terms, taxonomy, subjects, check-form NLQF descriptor and level

Professional competency PCP

Process PCP

Learning outcome

Check-terms Taxonomy Subjects Checks Descriptor Level

Second descriptor

41. The

candidate can use a calculator to carry out various practical

calculations

41.1 The candidate can convert metric

units to imperial units and vice versa

using a calculator.

RC MSV ES Applying knowledge

4

… … .. .. .. .. ..

2.2 Suggestions for verifying NLQF levels

In the verification, mention the learning outcome from the qualification to which the

descriptor relates;

In the verification, use terminology as indicated in the descriptors, such as signalling,

reproducing, evaluating, analysing, flexible, inventive, etc;

When drawing up the verification for specific levels, look for the points at which the

description of the descriptor differs from the upper and lower levels. Use the

descriptor overview for this purpose;

Use examples in the verification, such as specific processes that must be carried out;

When using examples, refer as specifically as possible to (pages in) annexes which

illustrate these examples, e.g. assessments, tests, curricula, etc. Ensure that

annexes are numbered.

Provide concrete, clear, brief and concise verification. Determining factors for the

content and level of the descriptor and the way in which this must be checked, must

be clear to the NCP NLQF from the description.

http://www.nlqf.nl/


Documents

Classification Manual Manual for formulating learning ... · taxonomy codes. Annexes 1 and 2 set out two taxonomies, namely, Bloom’s taxonomy and Romiszowski’s taxonomy. These