TRAINING ON

OUTCOME-BASED EDUCATION

Year 2011

INTRODUCTION TOOUTCOME-BASED ASSESSMENT

Year 2011

At the end of this session, participants are should be able to:• describe the basic concept of outcome based

assessment and its application (C3,A4) • explain assessment terms used in outcome

based education. (C3,A3)• describe various methods of assessment in

accordance with the course and learning outcome.(C3, A3)

• design rubric for assessing learning outcomes (C3,P2)

LEARNING OUTCOME

3

ASSESSMENT IS…

4

OUTCOME BASED ASSESSMENT

Focuses on providing students with multiple opportunities to practice what they need to do, and the provision of feedback on that practice (formative). Eventually, students must demonstrate achievement (summative).

5

6

PURPOSE OF ASSESSMENT

7

PRINCIPLES OF ASSESSMENT

PRINCIPLE OF

ASSESSMENT

8

ASSESSMENT LEVELS

Program AIMS (PAI)

Programme Learning Outcomes (PLO)

Course Learning Outcomes (CLO)

Upongraduation

Upon subject completion

Few years after Graduation – 3 to 5 years

Topic Learning Outcomes Upon topic completion

9

REFLECTION 1

• What do you do when your class performs poorly on quiz/test/practical etc?

Adjust the scores upward Grade on a curve Adjust the difficulty of the next assessment Give advice on study strategies Provide more practice problems, review Spend more time on the materials in the future Change your teaching materials or approach

10

Assessment Task

Assessment task refers to summative and formative assessment or criterion referred assessment. Examples are case study, critique, debate, demonstration, drawing, sketch, essay, exhibition, interview, journal, laboratory/practical, literature review, logbook, model, oral examination, peer review, presentation, portfolio, practicum, problem solving, projects, reflection, reports, self assessment, research paper, thesis and workshop

11

ASSESSMENT TERMS

Assessment Criteria

• A list of requirements that describe how well a student has to be able to achieve the learning outcome in order to be awarded a particular grade. Read more: http://www.answers.com/topic/assessment-criteria#ixzz1DjoBm8oF

Continuous Assessment

Continuous Assessment is assessments that takes place over a period of time. In other words you will be assessed right through your learning process and not only after the learning process. By doing continuous assessment you can track the improvement (if any) of the learner, you will be able to give more support and guidance, and the learner will have more opportunities to improve. http://wiki.answers.com/Q/What_is_continuous_Assessment#ixzz1DjnPBHWy

12

ASSESSMENT TERMS

Evaluation

A process that enables us to measure, compare, and judge the quality of student work, schools, or specific educational programs.

Measurement

The process by which attributes or dimensions of some physical object, process or opinion are determined, conducted using questionnaires, standardized test etc.

13

DOMAINS OF EDUCATIONAL GOALS

Psychomotor

Affective

Cognitive

The Heart

The Hand

The Head

3H14

Adapted from DrJJ. UiTM.

Learning outcomesLearning outcomes

CognitiveCognitive AffectiveAffective Psychomotor / skillsPsychomotor / skillsDOMAINS

Evaluation (C6)

Synthesis (C5)

Analysis (C4)

Application (C3)

Comprehension (C2)

Knowledge (C1)

Internalizing (A5)

Organisation (A4)

Valuing (A3)

Responding (A2)

Receiving (A1)

Naturalisation (P5)

Articulation (P4)

Precision (P3)

Manipulation (P2)

Imitation (P1)

Higher Higher orderorder

Lower orderLower order

BLOOM’S TAXONOMY

15

LEARNING DOMAINS

Values, Attitudes &

Professionalism

DOMAIN OF LEARNING OUTCOMES (MQF)

CRITICAL THINKING & PROBLEM SOLVING

SKILLS

COMMUNICATION SKILLS

ENTREPRENEURSHIP SKILLS

INFORMATION MANAGEMENT SKILLS

& CONTINUOUS SKILLS

LEADERSHIP SKILLS

MORAL & PROFESSIONAL SKILLS

TEAMWORK SKILLS

GENERIC STUDENT ATTRIBUTES (MOHE)

Social Skills &Responsibilities

Practical Skills

Knowledge of

Discipline Areas

Communication Leadership & Team SkillsProblem

Solving & Scientific

Skills

Managerial & Entrepreneu

rial Skills

Information Management

& Lifelong Learning Skills

16

STAK

EHO

LDER

SPR

OR

GR

AM

E AIM

S

Management support and commitment

PAI-1

Regulatory /Professional

Bodies

KPT/MQA IHL requirements Industrires

students…

….. etc

PAI-2 PAI-3 PAI-4 PAI-5 ….. etc

PLO-1 PLO-2 PLO-3 PLO-4 PLO-5 ….. etc

PRO

GR

AM

ME

LEAR

NIN

G

OU

TCO

MES

Subject CLO-1

CO

UR

SE LEA

RN

ING

O

UTC

OM

ES

Subject CLO-3

SubjectCLO-4

Subject CLO-5

Subject CLO-2

RELATIONSHIPS BETWEEN CLO’S, PLO’S, PAI’S and COMPLIANCE TO THE STAKEHOLDERS

17

FORMATIVE ASSESSMENT

Formative assessment is generally carried out throughout a course or project, in which the feedback from the learning activities are used to improve the process of learning. Formative assessment is also referred to as "educative assessment,“ to aid learning. In an educational setting, formative assessment might be a lecturer (or peer) or the learner, providing feedback on a student's work, and would not necessarily be used for grading purposes. Formative assessments are diagnostic.

18

SUMMATIVE ASSESSMENT

Summative assessment is generally carried out at the end of a learning programme. Summative assessments are typically used to assign students a course grade. Summative assessments are evaluative and often in the form of a report for a third party e.g. the Head of Department or parent.

“when the cook tastes the soup, that’s formative, when the guests taste the soup, that’s summative.”

Albert Stake

19

DIAGNOSTIC ASSESSMENT

Diagnostic Assessment occurs at the beginning of the teaching/learning cycle. It involves making judgements as to how a student is performing against a predetermined set of criteria. Diagnostic assessment is also one strategy to prepare students for the course requirements. It can be used as a form of orientation or induction.

20

DIRECT ASSESSMENT

21

DIRECT VS INDIRECT ASSESSMENT

22

TYPES OF ASSESSMENT

23

NORM REFERENCED ASSESSMENT

• Norm-referenced assessment compares students to that or those of a larger group. or "norm group" which is a national sample representing a wide and diverse cross-section of students. Students are assigned grades according to their standing relative to other students and normally follow the normal curve (grading on the curve). NRA is usually used to sort students and not to measure achievement towards some criterion of performance For this reason, there are strong moves internationally to move towards CRA.

24

CRITERION-REFERENCED ASSESSMENT

In criterion-referenced assessment, students' performances are compared to a specific learning objective or performance standard and not to the performance of other students For CRA to be beneficial to learning, explicit criteria are essential which provide clear learning goals for students.

25

REFLECTION 2

The folllowing are some of the issues in assessment. Discuss in pair about some of these issues.

1. Overload of students and staff

2. Too many assignments with the same deadline

3. Insufficient time for students to do the assignments

4. Insufficient time for staff to mark the assignments

5. Inadequate or superficial feedback provided to students.

6. Overuse of one mode of assessment such as written

examinations, essays, or closed problems

26

LearningLearning AssessmentAssessment

Feedback is the bridge between learning and assessment

FEEDBACKFEEDBACK

FEEDBACK

27

FEEDBACK

28

RUBRICS

29

RUBRIC

A rubric is a set of criteria and standards for assessing student work or performance. . For each criteria and standards, a rubric defines the specific attributes that will be used to score or judge the student’s performance and to differentiate between different levels of performance. Scoring rubrics include one or more dimensions on which performance is rated, definitions and examples that illustrate the attribute(s) being measured and a rating scale for each dimension.

Dimensions are generally referred to as criteria, the rating scale as levels, and definitions as descriptors.

30

TYPES OF RUBRIC

31

HOLISTIC RUBRIC

Description Score

Demonstrates complete understanding of the problem. All requirements of task are included in response.

5

Demonstrates considerable understanding of the problem. All requirements of task are included

4

Demonstrates partial understanding of the problem. Most requirements of task are included.

3

Demonstrates little understanding of the problem. Many requirements of task are missing

2

Demonstrates no understanding of the problem 1

No response/task not attempted 0

32

ANALYTIC RUBRIC

Needs Improvement

(1)

Developing (2)

Sufficient (3)

Above Average (4)

Clarity (Thesis supported by relevant information and ideas.)

The purpose of the student work is not well-defined. Central ideas are not focused to support the thesis. Thoughts appear disconnected.

The central purpose of the student work is identified. Ideas are generally focused in a way that supports the thesis.

The central purpose of the student work is clear and ideas are almost always focused in a way that supports the thesis. Relevant details illustrate the author’s ideas.

The central purpose of the student work is clear and supporting ideas always are always well-focused. Details are relevant, enrich the work.

Organization (Sequencing of elements/ideas)

Information and ideas are poorly sequenced (the author jumps around). The audience has difficulty following the thread of thought.

Information and ideas are presented in an order that the audience can follow with minimum difficulty.

Information and ideas are presented in a logical sequence which is followed by the reader with little or no difficulty.

Information and ideas are presented in a logical sequence which flows naturally and is engaging to the audience.

33

ANALYTIC RUBRIC

• Determine your peformance levels. Examples of performance levels. iptor-

• Excellent/ Good/Moderate/Poor/ Very Poor or Excellent/Good/ Satisfactory/Needs Improvement

• Numbers/numerical scores – 4/3/2/1

34

STEPS TO CREATE A RUBRIC

35

ASSESSMENT TASK

36

SELECT ASSESSMENT METHODS

No. Category of Learning Outcome Sample of Assessment Methods

1 Thinking critically and making judgements

essay, report, journal , present a case for an interest group ,Book review (or article) for a particular journal and write a newspaper article

2. Solving problems and developing plans

group work, work-based problem, analyse a case and conference paper (or notes for a conference paper plus annotated bibliography)

3. Performing procedures and demonstrating techniques

Demonstration, Role Play, Make a video (write script and produce/make a video), Produce a Poster, Lab report

4. Managing and developing oneself

Journal , Portfolio, Learning Contract and Group work

37

SELECT ASSESSMENT METHODS

No. Category of Learning Outcome Sample of Assessment Methods

5 Accessing and managing information

Annotated bibliography , Project, Applied task ,Applied problem

6. Demonstrating knowledge and understanding

Written examination, Oral examination, Essay , Report Short answer questions: True/False/ Multiple Choice Questions (paper-based or computer-aided-assessment)

7. Designing, creating, performing

Portfolio, Performance , Presentation, Projects

8. Communicating Written presentation (essay, report, reflective paper etc.), Oral presentation. Group work, discussion/debate/role play, observation of real or simulated professional practice

38

ASSESSMENT TASKS

Objective Tests measure the learners’ ability to remember facts and figures as well as their comprehension of course materials. Common variations include multiple-choice (MCQ), true-false, and matching items.

39

ASSESSMENT TASKS

Case Studies describes typical issues or problems. It gives students the opportunity to place themselves in the position of the decision maker in a specific simulated situation. The scenarios are usually taken from real life and presented from the standpoint of the decision maker. A successful case is one which offers debate on alternative courses of action, rather than offering a single settled outcome or solution

40

ASSESSMENT TASKS

Essay Questions are the most commonly-used assessment method in polytechnics. There are two main varieties:

• Unstructured questions, also known as free-response questions. Students have maximum freedom for discussion.

• Structured or restricted-response questions. The student is given far less freedom to determine the nature and scope of the response. Often, the questions guide both the structure and the content of the expected response.

41

ASSESSMENT TASKS

• Capstone Assessment are often final courses or internships in a program. They provide an excellent opportunity to identify whether students have acquired the central knowledge and skills that are the carefully selected outcomes expected of the program. Capstone assessment may also want to assess broader skills important to our graduates such as problem solving, computer literacy, team work, communication skills, and even the reading of professional material. (www.indstate.edu/ academicaffairs /assessment/capstone_assessment-intro.pp/)

42

ASSESSMENT TASKS

• End-of-chapter problems are used to reinforce and apply some concepts and skills learned in the classroom. As an example when a mathematical procedure is presented in the classroom, students are given numerical practice problems to solve, in order, after class. These numerical problems are usually over-simplified real-life problems calling for direct substitution of values into formulae.

43

ASSESSMENT TASKS

• Reflective journal is a piece of writing which allows students to record thoughts and insights about their own learning experience. This can be writing about what and how they have learned and understood a topic. It can also be a review of their learning process, self-evaluation of their performance and planning for future learning based on past learning experience, etc

44

ASSESSMENT TASKS

• Seminar Presentation. Students work individually, or in teams, to investigate a topic relevant to the course and present their findings in the form of a seminar. There are other variations of the seminar presentation.

45

ASSESSMENT TASKS

• A portfolio is a collection of course-related work performed by the student. It also a written reflections in which students evaluate their own learning.. Traditionally, portfolios are used to assess learning in the arts and humanities. However, they can be equally useful in just about any discipline.

46

ASSESSMENT TASKS

• Projects are popular methods of assessment in Polytechnics today. Projects can be based on literature and/or empirical research on a relevant problem. They can also be applied projects – solving real-life or simulated problems, producing technical prototypes, or preparing a business plan.

47

ASSESSMENT TASKS

• Examination are used primarily for grading or selection purpose. We call this activity, in educational terms, summative assessment. It can use many of the methods we discussed above. Since examinations can take on many variations, it is important to understand the range of possibilities in order to design examinations that are compatible both with outcome-based and student-centred education.

Adapted from http://www.polyu.edu.hk/obe/

Check also Enhanced Learning and Teaching, Hong Kong University of Science and Technology at http://celt.ust.hk/ideas/afl/MExam/index.html#see

48

CONCLUSION

• The OBE implementation required a lot of coordination in planning which involved a lot of assessment. A well designed quality management system is needed to ensure that our academic programme , delivery system, assessing methods and our graduates are at high quality. A framework of continuous improvement with the involvement of the academic staff, external examiner, students participation and other stakeholders will probably contribute to the successful implementation of outcome-based assessment in polytechnics.

49

ACTIVITIES

50

ACTIVITY 1

51

• Designing Test Items.

This exercise offer you some experience in constructing test items.

By referring the JSP and JSI, outline the CLO to be covered by the test.

Use the Exam Mapping Table as a guide to construct the test items.

EXAM MAPPING TABLE

Exam Content

Type (Essay/Calc/Sketch)

CLO PLO Bloom Taxonomy

Question 1a.b.c.

Question 2a.b.c.

52

ACTIVITY 2

Think of your favourite assessment when you were an undergraduate. In groups of three or four, discuss:

What made it a good assessment from your point of view?

What does it reveal about the tasks that you like doing? Solving problems? Learning, discovering, finding out, researching? Reading? Writing?

53

ACTIVITY 3

Please refer to the handout given to you.

54

ACTIVITY 4

The first step when creating a rubric is to determine what you want students to learn from the assignment.

In a group of 4 , create A Task Specific Rubric for student assessment.

55

56