Outcomes-based Teaching & Learning (OBTL)

Findings of the FGDs and surveys

Department of Asian & International Studies

Purpose of the study is to find out:

where students are in their learningwhat they think of OBTLwhether their program’s intended outcomes

are being metwhether the teaching/learning activities and

assessment tasks are helping them achieve those intended outcomes

Made use of Quantitative and Qualitative Research methods

MethodsQuantitative AIS Student experience survey - general

survey for all AIS studentsYr3 Exit Surveys - survey for graduating

students

QualitativeFocus group discussions for BSSESAS

studentsFocus group discussions for BSSIS students

AIS Student Experience survey

The purpose of the general survey is to collect data on the extent of students’understanding of OBTL & the impacts of OBTL to their learning

Method: used a mixed of Likert-scale responses and open-ended questions

Available to all AIS undergraduate students including graduating students

Participants 98 out of 295 students (33% of the total

population)92 completely answered the survey 6 did not finish the survey

By program:54% from BSSIS program46% from BSSESAS program

By year (cohort):1% from the 2005 cohort 16% from the 2006 cohort 41% from the 2007 cohort42% from the 2008 cohort

67% did not answer the survey (198 students)

Results1. Where did students learn of OBTL?

63% have heard of OBTL through their professors or in class

56% from university-related sites and events/activities

9% through their peers 12% have never heard of OBTL

2. Rating the impacts and effectiveness of OBTL

6 out of 9 statements were rated a 3 (Neither agree nor disagree)

These statements referred to: Difference to non-OBTL courses Usefulness of OBTL OBTL’s application-oriented Acquired transferable skills Relevance of OBTL to their studies How OBTL actively motivates them

Favorable ratings on the remaining 3 statements:Through OBTL, I understand more about what

is expected of me in a course. The intended learning outcomes of my

program (ESAS / IS) identify what I am expected to learn and help me plan my learning.The assessment tasks are relevant and

measure my learning performance.

More than half (63%) gave an acceptable rating on its effectiveness in enhancing teaching & learning

3. Skills & capacities developed from their learningApproximately 40% students developedIndependent and critical thinking/reasoning

skillsMore informed about contemporary issuesKnowledge and specific expertise in your field

of studyOther skills & capacities also received

favorable ratingsOnly ‘leadership skills’ & ‘succeeding on

their own’ received a neutral answer (Neither agree nor disagree)

4. Student comments & feedback

Students gave examples of useful classroom activities

Comments about OBTL’s usefulness included:

DiscussionNo strong opinion of OBTLOBTL not a major factor that affects their studies.

Students strategically use their learning approach based on how they’ll be assessed.

What and how we want students to learn does not always matchMisalignment of intended outcomes &

teaching/learning activities or assessment tasksStudents need to be fully aware and understand why

OBTL is implemented.Best source for learning about OBTL: instructors

and in classroom settings

Exit surveys

The purpose is to gain information on their learning experiences in their respective programs

Method: used a mixed of Likert-scale responses and open-ended questions

Available only for graduating students

ParticipantsBSSESAS (expected graduates = 38)11 completed the survey3 did not finish the surveyAll in all, 34% of the total population answered

the survey BSSIS (expected graduates = 92)20 completed the survey4 did not finish the survey26% of the total population answered the survey

Results1. Rating of intended outcomesBSSIS students had given favorable ratings

to all statements pertaining to their learning experience

BSSESAS students gave favorable ratings to the majority of statements with the exception on transferable skills, ability to critically evaluate and courses reflecting major priorities of their program

Quality of the program: Both programs believed that over-all, it was good.

2. Skills & capacities developed from their learning

Majority of BSSIS students, strongly agree: Teamwork Skills More informed of contemporary issues Expand their knowledge of cultures

Majority of BSSESAS students, agree: Ability to get along and work with different people

Majority of both groups are doubtful they attained: BSSESAS: sense of maturity BSSIS: leadership skills

Student comments & feedback

Discussion

Worry about career prospectsAttain knowledge, including skills and

experiences for future careers or jobs

Quality of teaching and program => students can succeed in fulfilling intended outcomes

Increase student awareness, transforms student’s understanding of their surroundings => make more informed choices & decisions

Limitations of the surveys

Survey function of Blackboard is simple and generates aggregated results only.

Low student response rates Email errors (i.e. delivery failure, over

quota) No other contact information of students

was available

Focus group discussions

Purpose: to obtain in-depth information from a smaller group to understand the context behind their answers on the surveys.

Limitations: Conflicting schedulesLow turnout of studentsEmail errors ((i.e. delivery failure, over quota) No other contact information of students was

available

Participants

Full-time2008M19BSSESAS112 - April 20, 2009

Full-time2008F21BSSESAS102 - April 20, 2009

Full-time2008M21BSSESAS92 - April 20, 2009

Full-time2008F23BSSESAS82 - April 20, 2009

Full-time2007F23BSSIS71 - April 6, 2009

Full-time2006Mn/aBSSIS61 - April 6, 2009

Part-time2007M23BSSIS51 - April 6, 2009

Full-time2007F23BSSIS41 - April 6, 2009

Full-time2007M22BSSIS31 - April 6, 2009

Part-time2007Fn/aBSSIS21 - April 6, 2009

Full-time2007M25BSSIS11 - April 6, 2009

Mode of Study

Year (Cohort)

GenderAgeProgramStudentFGD Session

Key points of discussion:1. Approaches & strategies to learning Students motivated to get good gradesAppears to come first before learningMost prefer to use readings and follow guidelines

already found in their course syllabus

Student strategy uses both deep and surface approach. Whatever fits to achieve a good grade.

Students’ suggestion in handling heavy workloadBSSIS prefer reduce workloadBSSESAS prefer redistribute workload

2. Relevance of OBTL Vague or little understanding of OBTL Difficult to measureBSSIS studentsUnable to align the ILOs to the TLAs and AtsNobody had emphasized the importance of OBTL View the ILOs as an introduction to the course (as

course objectives)Sometimes, strategy over substance

BSSESAS studentsOBTL becomes more relevant to them when they start

doing their assessment tasksApply ILOs to the ATs in effect fulfilling the ILOs of

the course

Not much difference to non-OBTL coursesMain difference: engaging students

Preference for more diverse classroom activities (similar to examples given in the general survey)Motivates students to actively participateActivities are useful if it is appropriate to the

courseManaging the class and class size affect the

effectiveness of OBTL Feedback leads to more student success and

good learningMore summative & formative feedback

3. Impact & effectiveness of OBTL to their learning experiences

Conclusions

Primarily motivated by good grades to perform well in their programs

Majority of students have a vague understanding of OBTL and are not aware that some of the activities they are doing are outcomes-basedBSSESAS students applying ILOs to fulfill

their ATsStudents have an active role in class