Module (7)* ALIGNING ASSESSMENT IN OBE-SCL Training Module … · · 2012-03-19Module (7)* Training Module for IHLs ... 15 Lesson Plan 16 Active Learning ... TEST 1 4 4 0.5 0.3

ALIGNING ASSESSMENT IN OBE-SCL

Module (7)*

Training Module for IHLs

Training Module for IHLs © UiTM 2010

Session’s Outcome

At the end of this session, participants will be able to

• conduct their own CDL for Courses and • indicate the direction of meeting the intended

PO hence LO of MOHE using manageable generic performance indicators.

PROGRAM for MODULE 7

Masa ITEM8:00 – 8:30 Registration Campus/faculty8:30 – 9:00 Revisiting Important Points UHEK

9:00 – 10:00 Learning & Assessment UHEK10:00 – 10:30 Break10:30 – 11:30 Assessment Blueprint UHEK11:30 – 1:00 Performance Indicators and

CDL TemplateUHEK

1:00 – 2:00 Lunch2:00 – 4:00 Group Workshop •Evaluate level of difficulty

(final exam / test Q)• Prepare a final exam or

continuous assessment blueprint of your course

• evaluate your course using CDL template

4:00 – 5:00 Presentation & Discussion Participants

PRE WORKSHOPRATE YOUR KNOWLEDGE ON THE FOLLOWING ITEMS (1: no knowledge, 5: very knowledgable)

No ITEM RATE1 Outcome based education2. Student Centred Learning3 Constructive Alignment of LO-PEO-PO-CO-TM-AM4 Measure of Breadth5 Measure of Depth6 Bloom’s Taxonomy 7 Assessment Types8 Table of Test Specification9 Continuous Assessment Blueprint

10 Final Assessment Blueprint11 Aligning Final Exam Questions to CO12 Aligning Test Questions to CO13 Aligning Projects to CO14 Assessment Evidence


No ITEM RATE15 Lesson Plan16 Active Learning17 Rubrics of (Soft-skills, Course work, OBE-SCL

Implementation)18 Assessing Alignment 19 Assessing level of difficulty of test/exam items20 Performance Indicators (generic/ specific for

curriculum)21 Closing the loop (generic/ OBE-SCL)

TOTAL

SOME IMPORTANT POINTS @ A GLANCE

• Outcome Based Education• Student Centred Learning• Student Learning Time• Designing Down • Constructive Alignment of LO-PEO-PO-CO-TM-AM• Lesson Plan• Active Learning• Rubrics of (Soft-skills, Course work, OBE-SCL Implementation)• Assessing Alignment • Assessing level of difficulty of test/exam items• Performance Indicators (generic/ specific for curriculum)• Closing the loop (generic/ OBE-SCL)

A necessary and sufficient condition to conduct an unbiased CDL :

LO, PO, CO are well documented and alignment of

assessment and learning activities to the CO PO LO are

assured

Indicating Learning At lesson level:

At course level:

At program level:

At university level:How do we indicate the breadth and the depth of a curriculum?

REVISIT BLOOM’S TAXONOMY


Example of depth in a course


Develop another example of progression of depth from your own course within a program

• Think and write it down


Recall Learning Outcomes

• LO1• LO2• LO3• LO4• LO5• LO6• LO7• LO8• LO9


How can we indicate learning?

• Evidence:


Indicating LO attainment through Capstone Course

Instrument Cognitive Learning

Affective Learning

Psychomotor Learning

Final Year Theses X XFinal Year Project X X XOral Presentation X X XPortfolio X X X

Assessment?


OUTCOME BASED EDUCATION

&STUDENT CENTRED

LEARNINGProgramme –Level Assessment

Course –Level Assessment

Institutional –Level Assessment

ASSESSMENT AS KEY COMPONENT OF OBE-SCL

Types of Assessment

• Continuous Assessment (formative)


Types of Assessment

• Final Examination (summative)


TYPES OF ASSESSMENT

Test Quiz Assignment Project Papers Case Study Presentation Experiments Seminars Community Project Final Exam

Lab report Physical model Practical Project Laboratory work Observation Peer assessment Coursework Survey report Site report

19

What type of assessment you are familiar with?

• List it down:


Why do we need to align assessment to learning outcomes?


Reflection on what you had done

• Take out past test Q or final exam Q• Take out your CO-PO/ Syllabus• For each question, determine the level of

knowledge assessed by comparing to the Bloom’s taxonomy

• Determine the % of low:medium:high level of taxonomy

• Rate the level of difficulty. • Reflect on your student performance.


Assessing the Level of Difficulty of the Item in a test or final exam

Taxonomy No of item % to total item RemarkKnowledgeComprehensionApplicationAnalysisSynthesisEvaluationTOTAL 100% Take Note of the

highest %, comment on the distribution

How best to align assessment back to the Course Outcomes?

Give Suggestion


Procedure

Build an assessment blueprint of

a program by year

Build an assessment blueprint for

a course

BuildContinuous Assessment

Blueprint

Test

Assignment

Project

Presentation

dissertation/ thesis

Soft-skills

BuildFinal Assessment

Blueprint

Written

Criterion referenced


LO/KIPOCO

How to apply the Blueprint?


Required Materials

Program Outcomes

Course Outcomes

Lesson Outcomes

Topics / Concepts

Levels of Blooms taxonomy (Cognitive,

Affective, Psychomotor)

SLT for assessment

Possible assessment methods

Course Information


Question to Ask in Designing a Blueprint

• What outcomes will(should) I be testing?• What types of items will be included in the

test?• How long will the test be in terms of time and

number of items?• How much will each outcomes be worth in

terms of weighting and number of item


Example of Test Specification Table: By distribution of questions

BLOOMS TAXONOMY

SUBJECT CONTENT KNOWLEDGE& comprehension

APPLICATION HIGHER MENTAL ACTIVITIES

EXAM EMPHASIS(%)

Art History 1,2,3,4,5,6,7,8, 9 10 10(50%)

Art Appreciation 13 12 11, 14-20 10(50%)

TOTAL # of Items 9(45%) 2(10%) 9(45%) 20 (100%)


Test Specification Table: Distribution of questions by % of Total Mark

COGNITIVE LEVEL

CONTENT/ CONCEPT

KNOWLEDGE COMPREHENSION & APPLICATION

HIGHER MENTAL ACTIVITIES

EXAM EMPHASIS(%)

Nature & behavior of light 6 12 5 23

Electric & magnetic fields 6 16 5 27

…….. 4 8 3 15

…….. 5 11 4 20

……. 4 8 3 15

TOTAL 25 55 20 100


Assessment Blueprint for Diploma Program


Final Assessment Blueprint for Diploma

YEAR 1

DISTRIBUTION

ASSESSMENT Knl Cmp App Anl Syn Eva % NoQ DURATION

FINAL (%) 0.2 0.4 0.4 0 0 0 1 15

3 HOURS

NO OF Q 3 6 6 0 0 0 15


Continuous Assessment Blueprint for Diploma Year 1

ASSESSMENT Knl Cmp App Anl Syn Eva NoQ DURATION

TEST 1 2 2 0 0 0 5 1 HOURNO OF Q 1 2 2 0 5

TEST 2 4 4 0 0 0 10 2 HOURNO OF Q 2 4 4 0 10

ASSIGNMENT 0.8 1.6 1.6 0 0 0 4 8 hoursNO OF Q 1 1 1 1 4

PROJECT 0 0 0 1 1 24 hours

QUIZ 3Q 1 1 1 0 3 30 min

QUIZ 2Q 1 1 0 2 20 min



YEAR 2

DISTRIBUTION


FINAL (%) 0.15 0.4 0.4 0.05 0 0 1 15

3 HOURS

NO OF Q 2 6 6 1 15




TEST 0.75 2 2 0.25 0 0 5 1 HOURNO OF Q 1 2 2 0 5TEST 1.5 4 4 0.5 0 0 10 2 HOURNO OF Q 1 4 4 1 10ASSIGNMENT 0.6 1.6 1.6 0.2 0 0 4 8 hoursNO OF Q 1 1 1 1 4PROJECT 0 0 0 1 1 24 hours

QUIZ 3Q 1 1 1 0 3 30 min

QUIZ 2Q 1 1 0 2 20 min



YEAR 3

DISTRIBUTION


FINAL (%) 0.1 0.4 0.4 0.05 0.03 0.02 1 15 3 HOURS

NO OF Q 1 6 6 1 1 15




TEST 0.5 2 2 0.25 0.15 0.1 5 1 HOURNO OF Q 1 2 2 0 5TEST 1 4 4 0.5 0.3 0.2 10 2 HOURNO OF Q 1 4 4 1 10ASSIGNMENT 0.4 1.6 1.6 0.2 0.12 0.08 4 8 hoursNO OF Q 1 1 1 1 4PROJECT 0 0 0 1 1 24 hours

QUIZ 3Q 1 1 1 0 3 30 min

QUIZ 2Q 1 1 0 2 20 min


Assessment Blueprint for Degree Program


Final Assessment Blueprint for Degree

YEAR 1

DISTRIBUTION


3 HOURS


Continuous Assessment Blueprint for Degree Year 1


TEST 5 1 HOURNO OF Q 5

TEST 10 2 HOURNO OF Q 10

ASSIGNMENT 4 8 hoursNO OF Q 4

PROJECT 1 24 hours

QUIZ 3Q 3 30 min

QUIZ 2Q 2 20 min



YEAR 2

DISTRIBUTION


1 15

3 HOURS

15




TEST 5 1 HOURNO OF Q 5TEST 10 2 HOURNO OF Q 10ASSIGNMENT 4 8 hoursNO OF Q 4PROJECT 1 24 hours

QUIZ 3Q 3 30 min

QUIZ 2Q 2 20 min



YEAR 3

DISTRIBUTION


FINAL (%) 1 15 3 HOURS

NO OF Q 15




TEST 0.1 5 1 HOURNO OF Q 5TEST 0.2 10 2 HOURNO OF Q 10ASSIGNMENT 0.08 4 8 hoursNO OF Q 4PROJECT 1 24 hours

QUIZ 3Q 3 30 min

QUIZ 2Q 2 20 min


Translating the Blueprint

Distribution of QuestionsDistribution of Marks


Course Code:Program Code:

TOPICCOGNITIVE

AFFECTIVE PSYCHOMOTOR CO POKC AA SE

CO Statements: PO mapped:


Example of Course in a Diploma Program

LO CO

Content Distribution of hours

% Knl Cmp App Anl Syn Eva TotalLo1 3 11%

CO1 27% Lo2 2.5 9%Lo3 3 11%

CO2 39% Lo4 2 7%Lo5 2 7%

CO3 14% Lo6 2 7%Lo7 4 14%

CO4 20% Lo8 3 11%Lo9 4 14%

Lo10 2.5 9%

100% Total 28 100% 0.1 0.4 0.4 0.05 0.03 0.02 1


Indicating Learning

• At lesson level:• At course level:• At program level:• At university level:• How do we indicate the breadth and the

depth of a curriculum?


WE NEED A SET OF GOOD PERFORMANCE INDICATORS AT

THOSE CRITICAL POINTS11:00 to 1:00


Measuring & Monitoring OBE-SCL Implementation

What to monitor?

Achievement of i) lesson outcomes ii) course outcomes iii) program outcomes iv) MOHE learning outcomes

Why monitor?

To manage the attainment of graduatesattributes as required by many quality assurance guideline

Measuring & Monitoring OBE-SCL Implementation

When to monitor?

Closing the Loop @ 4 levels: class/course/program /nationalLesson Outcomes: End of Class Course Outcomes: End of Semester Program Outcomes: End of Program Program Objectives: After 3 years of graduation

Who to monitor?

Lesson Outcomes: Lecturers Course Outcomes: Coordinator, Program Outcomes: Head of Program Overall LO: Dean of Academic/Academic Affairs

Measuring & Monitoring OBE-SCL ImplementationHow to monitor?

Direct measure: Student Performance Indirect measure: Entrance / Exit Survey

Performance indicators

• Lesson Outcomes: •Feedback count

• Course Outcomes: •% of student score > 50 • Gap between Exit & Entrance

• Program Outcomes: • weighted average of student passing the courses that mapped to each and every PO • gap between Exit & Entrance

• Learning Outcomes :• Average Actual SLT • Level of OBE-SCL implementation • % of graduates employed within 6 months of graduation

Measuring & Monitoring OBE-SCL Implementation•Lesson Outcomes:

•Feedback count > 80% of students

•Course Outcomes: • GPA where 80% of students are above it•Gap between Exit & Entrance

GPA >2.00 Non negative

•Program Outcomes: • CGPA where 80% of students are above it• Gap between Exit & Entrance CGPA >2.00

Non negative

•Learning Outcomes :• Average Actual SLT • Level of OBE-SCL implementation • % of graduates employed within 6 months of graduation

80% above Target SLT 4 80%

Indicating learning through indirect measure (survey score)

exit entrance

Possible gap

value score descriptor5 1 4 5 Excellent 4 2 3 4 Very good3 3 2 3 Good2 4 1 2 Fair1 5 0 1 Poor

-1-2-3-4

Determine the score (20th percentile) or where 80% of the students are above it for both exit and entrance. Determine the gap by getting the difference ( Exit- Entrance) scoreTranslate the score


Indicating learning through indirect measure (SUFO) of courses in a program

Determine the SUFO score for each courses, locate the 20% percentile or the score where 80% of the students are above it.Translate the score

20th Percentile SUFO SCORE Score Descriptor

4 5 Excellent 3.00-3.99 4 Very good 2.00-2.99 3 Good 1.00-1.99 2 Fair 0.00-0.99 1 Poor


Indicating learning through direct measure (GPA and CGPA) of a course and program

Determine the GPA (course) or CGPA (program)where 80% of the students are above itTranslate the score

GPA/CGPA Score Descriptor4 5 Excellent

3.50-3.99 4 Very good 3.00-3.45 3 Good 2.00-2.99 2 Fair 0.00-1.99 1 Poor


Process

Conduct entrance survey for

program and for course to determine entry level skills and

competency at early stage,

Conduct course delivery

effectively, monitor lesson outcomes and

SLT

Conduct assessments (continuous

and final)

Conduct exit survey after

the completion of

course and completion of

program

Consolidate measurement

at every lower level

before consolidating

at higher level

Measurement MechanismLEVEL OF OUTCOMES

MEASUREMENT MECHANISM Source

Lesson: Immediate response Level of interactionIndication of understanding (Observation)

Students

Course Direct measure: GPA student result Indirect: Entrance and Exit Survey:

Examination unitFaculty/ Campusi-learn

Program Direct measure: student overall CGPA Consolidating average pct passing of courses mapped to each PO for all semestersIn direct: Entrance and Exit Survey: first semester and before exitOBE-SCL Rubrics of OBE-SCL ImplementationGap on My3S Online Survey

Examination unitFaculty/Campus

Program

Faculty/ CampusMy3S Online

PEO 3-5 year alumni tracer study PPS

CQI of OBE-SCL

• Revision of the learning-teaching activities• Area for review should cover:

– Teaching schedule, lecture outlines, lesson plan,Lecturers Preparation Time

– Student attendance, class cancellation orpostponement

– Course Delivery– Student Learning Time– Assessment of students– Student performance

• Future development and improvement (CQI)


Activity

CONTINUAL QUALITY IMPROVEMENT (CQI)

MOCK-CQI


Mock CQI: Lesson Level

If 80% of the student still not able to indicate a change in behaviour, what action need to be taken?


Mock CQI: Course Level If 80% of the students taking the course score below 2.00 (COs are not met), what are the potential factors AND what actions to be taken?


Mock CQI: Program Level If 80% of the students from that program only achieve CGPA of less than 2.00 , what are the potential factors and what need to be done?


PROGRAM EVALUATION:

• Faculty:• Campus:• Program Description:…….• Evaluation for Semester:…….• Number of students:• Number of academic staff:

Mesyuarat JK OBE Induk dan Kampus Cawangan UiTM: Closing the OBE Loop (Penilaian Pencapaian

First Semester’s Structure

• Courses


CO-PO-LO-KI (first semester)

• Mapping


PERFORM THE ANALYSIS


INTERPRETATION


Program:……………….

CODE AlignmentLevel

CO-PO-LO-KI

Descriptor(GPA)

Descriptor (Exit-

Entrance)

OBE –SCL Level

SUFOScore level

Code 1 3 3 4 2 5

Code 2 3 2 4 3 4

Code 3 3 5 5 3 5

Code 4 4 2 3 2 3

Code 5 3 2 3 2 2

Code 6 3 4 5 3 2

Average 3.2 3 4 2.5 3.5


The attainment of the program outcome stands at level 3 : Good

Identify Strength/Weaknesses

STRENGTH WEAKNESSES


Discuss each item thoroughly

Decision:

Endorsement:


Report Brief:

• To be consolidated and submitted to UHEK at the end of every semester for the second level CDL in Dec 2013

• A copy must be kept at the site (to be used as and when needed). Good to be treated as evidence of good practice during any audit exercise

Mesyuarat JK OBE Induk dan KampusCawangan UiTM: Closing the OBE Loop

(Penilaian Pencapaian

What have we learnt?

Any more points not understood?



No ITEM RATE1 Outcome based education2. Student Centred Learning3 Constructive Alignment of LO-PEO-PO-CO-TM-AM4 Measure of Breadth5 Measure of Depth6 Bloom’s Taxonomy 7 Assessment Types8 Table of Test Specification9 Continuous Assessment Blueprint

10 Final Assessment Blueprint11 Aligning Final Exam Questions to CO12 Aligning Test Questions to CO13 Aligning Projects to CO14 Assessment Evidence


No ITEM RATE15 Lesson Plan16 Active Learning17 Rubrics of (Soft-skills, Course work, OBE-SCL

Implementation)18 Assessing Alignment 19 Assessing level of difficulty of test/exam items20 Performance Indicators (generic/ specific for

curriculum)21 Closing the loop (generic/ OBE-SCL)

TOTAL

Thank You

Associate Prof Dr Roziah Mohd JanorDirector of Curriculum Affairs

Academic Affairs DivisionUniversiti Teknologi MARA


Copyright © 2011 Universiti Teknologi MARA

All rights reserved. No part of this publication may be reproduced, stored in a retrieval systemor transmitted in any form or by any means, electronic, mechanical, photocopying, recording,or otherwise, without the prior written permission of Universiti Teknologi MARA.

Direct all inquiries to:

ASSOC. PROF. DR. ROZIAH MOHD [email protected]


Documents

Module (7)* ALIGNING ASSESSMENT IN OBE-SCL Training Module … · · 2012-03-19Module (7)* Training Module for IHLs ... 15 Lesson Plan 16 Active Learning ... TEST 1 4 4 0.5 0.3