A Study on Cost-Efficiency and Quality of OER Integrated Course Materials

3/9/173/9/17 1

A Study on Cost-Efficiency and Quality of OER Integrated Course Materials

Mohan MenonDisha Global Trust, Kochi, India

[email protected]. Phalachandra

Wawasan Open [email protected]

OE Global Conference 2017 Cape Town

3/9/173/9/17 2

A Study on the Processes of OER Integration

for Course Development

3/9/17 33/9/17

This is one of the sub-projects under a larger researchproject called Research on Open Educational Resourcesfor Development (ROER4D), coordinated by the Universityof Cape Town and undertaken as part of the grant receivedfrom IDRC of Canada by WOU.

This paper is based on the findings of Sub-Project 7 – Action research study of teacher

educators – Malaysia and India

Research Team

Members of the Sub-Project Research Team: l Mohan Menon

l Phalachandra B. l Jasmine Emmanuel

l Chng Lay Kee

3/9/173/9/17 4

3/9/17 53/9/17

PROJECT IMPLEMENTED WITH COOPERATION OF SEVERAL INSTITUTIONS AND ACADEMICS IN MALAYSIA, INDIA AND SRI LANKA

3/9/173/9/17 6

Present Study

l Most work being done in the OER movement involvescreation of OER with less focus on reuse of OER

l The study probes the operational processes involvedin course development using available online multi-media OER materials for a 5-credit (200 hours study)course on ‘Research Methodology in Education’involving course developers from Malaysia and India.

l Findings of this study have provided some tentativeanswers regarding the reuse of OER materials fordeveloping quality assured self-instructional materials.

Research QuestionsThis Paper presents only the part of the study focussing on the following two Research Questions:

1. Will course materials developed by integrating OER be more cost-efficient?

2. Will the use of OER integrated course materials expected to be of high quality from learners’ perspective?

3/9/173/9/17 7

3/9/173/9/17 8

Definition of variables

1. ‘Cost/Time Efficiency’: relates to a comparison ofwhat is actually produced or performed (OER-integrated course development) with what can beachieved with the same consumption of resourcesin terms of money and time.2. ‘Quality of learning materials’: Quality can bedefined as ‘appropriately meeting the stakeholders’objectives and needs which is the result of atransparent, participatory negotiation process withinan organization’ (Pawlowski, 2007).

3/9/173/9/17 9

OERPractice

EngagementofOERsearchteam,coursedevelopersandlearners

ReuseandIntegrationprocess

OUTCOMES&OUTPUTS

w Re-usabilityDigitisedOERMaterials

CoursePackage

ToolonOERQuality

QUALITY

PROCESS Findability&ReusabilityFactors

IMPACT

COSTEFFICIENCY

CONCEPTUAL FRAMEWORK

Methodology

l In this project the exploratory case study approach was adopted to understand and critically reflect on the factors and processes involved in the integration of OER material in developing a new set of course materials.

l MIXED METHODS APPROACH- Qualitative and Quantitative

l A variety of Research Tools used

3/9/173/9/17 10

Nature of Data and

Data Sourcel Both Qualitative and Quantitative data

3/9/173/9/17 11

VARIABLE DATA SOURCE AND COLLECTION PROCESS

COST/TIME EFFICIENCY

From the course developers before, during and after their engagement in Course writing

Quality of Learning materials

From Course users after learning through sample units of the OER integrated course

3/9/173/9/17 12

Research Tools

Variable Tools

Cost/Time efficiency

v Search logv Course development LOGv Written Format for individual reflection on

OER usev Narratives by course developersv Focus group discussionsv Individual Reflection Questionsv Semi-structured interviews

Quality of Learning Materials

v Course development LOGv Individual Reflection Questions

Data Analysis and Intrepretation

l On the part of COURSE DEVELOPERSl A triangulation of findings from the content

analysis and coding of the reflections and observations of the OER search team and course developers in the focus group discussions, interviews as well as diary recordings and their written log records and findability ratings was carried out.

3/9/173/9/17 13

COST/TIME EFFICIENCY

3/9/173/9/17 14

FINDABILITY

Time taken

3/9/173/9/17 AAOU Coference,2015, Kuala Lumpur 15

FINDABILITYTIME

No of Resources

Percentage

Less than 10 minutes

75 31.65

10 minutes 58 24.4715 minutes 38 16.0320 minutes 35 14.7725 minutes 6 2.5330 minutes 24 10.13More than 30 minutes

1 0.42

TOTAL 237

Cost-Efficiencyl Course Developers had expressed their thinking

about cost-efficiency based on their experience of finding and reusing and integrating OER for course development.

l When a large chunk of OER text materials (such as an open book or an entire course) are available in advance it became easier and less time consuming to use these materials and contextulise/localise them.

l Using a large number of smaller reusable learning objects (RLOs) took more time.

l Expertise in 7 competencies including 5 core competencies and 2 supportive competencies emerged in the study enhanced the cost efficiency of OER based course development

3/9/173/9/17 16

Cost-Efficiencyl When OER materials are collected in advance

for each concept or section of the modules by experienced and competent persons, the course writers took less time to integrate

l The search team reported that over 80% of relevant OER materials could be searched, discovered and down loaded within 20 minutes in each case, of which over 55 % took only less than 10 minutes.

l There is a general agreement among course developers that using OER enhanced the quality of the course produced in several ways

3/9/173/9/17 17

QUALITY OF COURSE MATERIALS


QualityAssessmentTool

• The quality of the course materials was assessed through a fivepoint (Strongly Agree to Strongly Disagree )

• Rating scale consisted of 7 dimensions and 38 items .• The seven dimensions are : Format and presentation, Language

and communication, Self-learning Interactivity, Inclusiveness ,Resources ,Assessment.

• The scale was administered after the participants went throughfew units of Research Methodology Modules in a seriesworkshops held in Sri Lanka, India and Malaysia


OER Quality Assessment Scale (OER-QAS)



Component No of Items

Total Score possible

Mean SD

Format and Presentation 20 4 14.12 4.54Language and Communication 30 6 23.22 3.18Self-Learning 45 9 34.65 5.01Interactivity 25 5 17.45 4.72Inclusiveness 20 4 14.89 3.20Resources 20 4 15.14 3.60Assessment 30 6 21.79 4.93Total 190 38 141.11 25.83

Findings

ttestresults(forentirescale)N Mean Standard

deviationt p

Female 64 140.61 26.66 0.27 0.78

Male 28 142.25 24.77

ttestresults(forentirescale)N Mean Standard

deviationt p

Educators 49 135.57 29.72 2.42* 0.02

Students 38 148.37 19.31

Comparisonofresponsesacrossgender

Comparisonofresponsesacrossparticipants

*Significantat0.05level

Findings• Theoverallqualityratingsofcoursematerialsishigh.• Allparticipantsfoundthecoursematerialtobeofhighquality.Thisisalsoindicatedwithrespecttodifferentindimensionsofthescale.

• Whilebothfemaleandmaleparticipantsshowedsimilarqualityrating,thetraineesgroupshowedhigherqualityratingascomparedtotheeducators.

• Itislogicaltothinkthattheeducatorswouldhavehigherexpectationsofqualitythantheteachertrainees.

CONCLUSIONl Reuse processes are being studied for the first

time in Asia and dimensions included in the study are being studied for the first time.

l The findings are tentative as this is an exploratory case study

l Open up a number of studies-Several Hypothesis emerged for further study

l Impact on Learning to be studiedl Second part of the Study – presented in another

paper

3/9/173/9/17 24

Contact details of Authors

Mohan MenonDISHA Global Trust, Kochi, [email protected]

Phalachandra BhandigadiWawasan Open University, Penang, MALAYSIA

[email protected]

3/9/173/9/17 25

THANKS

3/9/17 263/9/17

Authors would like to thank the ROER4D Team and IDRC for the support and funding

Authors also would like to thank the Research Team members of Sub-Project -7.

This presentation is released under

CC-BY-SAopen licence