VC’s L&T Forum 2013

Integration of Research Literacy and Skills in the Undergraduate Curricula

using the RSD Framework: Implementation across Four Different

Disciplines

Dr Jito Vanualailai (Director of Research, Research Office)

Ms Naolah Pitia (MG101 Coordinator, FBE)

Ms Jackie-Anne I Ete (UU204 Assistant Lecturer, FALE)

Dr Sereima Naisilisili (ED153 Coordinator, FALE)

Dr Shaneel Chandra (CH101 Coordinator, FSTE)

Ms Heena Lal (RSD Coordinator, Research Office)

Copyright © Jito Vanualailai, Heena Lal, Naolah Pitia, Jackie-Anne I Ete, Sereima Naisilisili, Shaneel Chandra, The University

of the South Pacific, 2013. Available under Creative Commons Attribution NonCommercial-ShareAlike 3.0 Unported License

Implementation of RSD Framework at USP

Purpose of RSD

To embed research literacy and skills in an explicit, coherent and systematic manner from 100-level to 300-level/ final year courses in undergraduate programmes and also at the postgraduate level.

The RSD Framework

• Five Facets: Refer to research literacy and skills

• Six Levels of Student Autonomy: Autonomy in research

• Research literacy and skills is now the 8th Graduate Attribute.

Link to RSD Framework – simplified version

Timeline for RSD

Implementation at USP

Year 2013 Year 2014 Year 2015

Embed RSD in UU courses

Introduce RSD in 100-

level discipline based

courses

Continue to embed RSD

in selected 100-level

courses

Introduce RSD in 200-

level discipline based

courses

Continue to embed RSD

in selected 200-level

courses

Implement and embed

RSD in core 300-level

courses

Introduce RSD in core

postgraduate courses

Continue to introduce

and embed RSD in core

postgraduate courses

Continue to embed RSD

in core postgraduate

courses

• RSD Framework rolled out at USP in 2012 - ‘UU100 Communication and Information Literacy’ and ‘UU114 English for Academic Purposes’, UU200 Ethics and Governance’. RSD implemented in UU204 in 2013.

What does RSD Implementation in courses

entail?

RSD is integrated in courses using as assessment based approach. This entails revision or modification of assignments and development of marking rubrics underpinned by the RSD framework.

• In this symposium, the RSD marking rubrics developed in three 100-level discipline based courses and one 200-level generic course during Semester 1 2013 and experiences of RSD implementation will be presented by course coordinators.

Courses: MG101 ‘Introduction to Management’, UU204 ‘Pacific Worlds’, ED153 ‘School and Society’, CH101 ‘Chemical Principles’.

RSD Implementation in MG101 Introduction to

Management MG101 Research Assignment Choose only one of the specific area given below to do your research assignment on:

1. Use the research sources you have in your disposal (e.g. literatures, articles, journals, and

book etc.; survey questionnaire, structured or semi structured interviews, case studies

etc.) to collect and compile information/data you require to write a report on the current

developments in terms of progress for minority groups and women in the workplace;

using the following questions as your guidelines:

a. What are some facts on the current developments in terms of progress for minority

groups and women in the workplace? Gather some informational facts/data from

countries like Australia, New Zealand, Fiji and other Pacific Island countries

b. What are the benefits and challenges of diversity at workplace?

c. What are “best” managers doing , in relation to diversity at workplace?

d. What lessons of diversity have you learned from this study?

OR

2. Analyze survey reports (or conduct a survey yourself) on how glass ceiling may affect the

careers of women and minority groups in various organization setting, and draw out

implications of such on the progress of women and other minority groups’ professionally and

personally.

MG101 RSD Assignment Marking Rubric

MG101 Experiences: Benefits and Challenges

Benefits: • Integrated the

assessment of more than one learning outcome.

• Allows for a more systematic marking (marker-friendly, less student complaints).

• Provide students with more understanding of the scope and standard of quality expected of their research assignments.

Challenges:

• Understanding the use of the RSD framework.

• Developing RSD assessment rubric that is simple and relevant to the development of the target group.

• Effective integration of the assessment of other learning outcomes.

• Applying the assessment rubrics to other assignments.

MG101 Recommendations

1. Revision of marking rubrics for simplification to both students and teaching staff.

2. Workshop training at School level:

• Learning outcomes at different levels as per year of study.

• Training on how to design an effective integrated assessment rubric based on RSD framework.

• Understanding the application of the assessment in other assignments or assessment work.

• Monitoring student development and remedial effort needed.

UU204 Letter to the Navigator & Vaka Crew

Assignment 1: Essentially a student’s introduction of him/herself to their online crew/ vaka-mates, mapping themselves, their ‘identities’ in relation to the readings, research and writings they encounter under Unit 1 themes - ‘Naming and Mapping Oceania’.

UU204 RSD Assignment Marking Rubric

Fig 2 & 3: Navigator

Bera Cama and youth

aboard Camakau,

Korova, Suva.

Picture courtesy of

Leota-Ete

UU204 Experiences with RSD

Prior to RSD

• NO additional research was required. Students were restricted to the use of UU204 course material which we provided as well as knowledge of own oral histories/stories.

• NO marking rubrics were available to students. (Except Steven Dutch’s ‘What is an ‘A’?’ guidelines for one assessment)

After RSD

• Oral histories/stories supported by scholarly literature.

• Students learn about source credibility and reliability (Lots of Wikipedia/tourist sites referenced at the beginning of semester).

• Transparent marking processes. Precise, concise, visually appealing rubrics.

UU204: Challenges in RSD Implementation

• Linear

• Phrasing/wording

• Preparation? (Defrosting)

• Connectivity in the region to online library databases.

RSD Implementation in ED153 Education and

Society Assignment 3: Research

Weighting: 15%

Due date: 24th May, 2013

• Carry out a research to find out people’s attitude about integrating Pacific culture in education. Your study may focus on specific aspects of your culture (e.g. cultural values, gendering process, traditional parenting styles, traditional models of discipline, traditional food preservation methods, traditional music/dance, etc.). Are there ways of merging culture and education?

• The skills section in your ED153 Course Book clearly provides the details required for this assignment. Follow the instructions in order to complete the research and your report.

ED153 Background: FALE Student Learning

Support Help in unpacking the question and process into researchable

components

Example

• Q: What are you writing about? Ans: attitudes

• Q: What attitudes?

• Ans: attitudes about integrating culture in education

• Q: Whose attitudes?

• Ans: youths/ parents/ women/ men

• Q: What/Which youths/ parents/ women/ men?

• Ans: Youths etc. in my/your community

• Q. Which aspect of culture?

• Ans: music/dance/values etc.

• Why?

ED153: My experience with RSD

• March symposium at the Holiday Inn

• Research assignment was analyzed against the RSD framework

• 207 students for Face-to-face and Print modes

• Research assignment and rubrics were already given to students

• I had no prior experience with RSD

• So focused on marking rubrics

ED153 Original Marking Rubric for Assignment 3

(15 marks) Aspect Criteria /15m

Introduction No introduction, aim not stated or wrong 0

Vague, introduction fails to describe the background/ outline of work undertaken 1

Aim is accurate and clearly stated adequate description of background/ outlines

of work undertaken

2

Exceptional/clear comprehensive description of aims, background and outline of

work

3

Methodology/

materials

No mention of methods/materials used 0

Some mention of methods/materials used 1

Methods and materials are almost complete and generally accurate 2

Accurate and complete account of methods/materials used 3

Results & Discussion No analysis and results given or totally incorrect 0

Incorrect analysis of data, results are incomplete and vaguely presented 1

Adequate analysis of data, results are accurate, some discussion of results

attempted

2

Analysis of data, results, and discussions are accurate and complete 3

Analysis of data, results and discussions are accurate and complete with

suitable illustrations

6

Conclusion No conclusion or wrong conclusion 0

Conclusion is partly correct but incomplete 2

Conclusion is correct, complete, related to the aim of the research and

suggestions for improvement of research are given

3

ED153 Marking Rubric Reorganised using RSD

Research

feature

Elements Excellent - 4 Very good - 3 Good -2 Unsatisfactory - 1

to 0

MARK

Embark and

clarify

Coverage

Introduction

All terms of reference

met

Detailed analysis

Purpose and

background of study

exceptionally clear ,

supported by literature

Terms of reference

met.

Details and analysis

adequate

Purpose is generally

clear and meaningful

with some literature

Most terms met.

Some

background and

purpose of study

mentioned,

weakly supported

Below standard

Weak with several

missing elements of

background and

purpose

Find and

generate

Methodology

and data

collection

Clearly discussed and

detailed with

appropriate data

Clear with some

appropriate data.

Generally

described lacking

details.

Some appropriate

data

Ineffective

methodology.

Unclear and

inappropriate

Organise and

manage

Appropriate

format

presentation of

results

Well planned, logical

sequence. All major

sections included.

Thorough with clear,

and properly

sequenced

presentation of data &

subheadings

Planned report,

logical sequence ;

major sections

included, data

correctly integrated

in to texts –

meaningful and easy

to follow

Adequate, mostly

clear discussions,

data

interpretations are

quite weak

Data not integrated.

Little organisation

Research

feature

Elements Excellent - 4 Very good - 3 Good -2 Unsatisfactory - 1

to 0

MARK

Analyse and

synthesise

Discussion of

results

Full, clear and

sophisticated.

Accurate

interpretations

well supported

Clear and detailed

discussion of data

with sound

interpretation

Adequate, mostly

clear discussion of

data

Discussion is weak.

Thin interpretations

of data

Evaluate and

reflect

Conclusions Well considered

and strong. Highly

relevant. Well

supported by data

analysis.

Relevant and

mostly clear.

Supported by data

and analysis

Relevant but lacks

depth. Supporting

evidence unclear

Questionable

conclusion.

Lacking support from

data analysis

Communicate

and apply

Grammar

Accurate writing

convention

Sentence fluency

and word choices

of high level

Strong & specific

words used to

convey meaning

Generally uses

appropriate words.

Sentences are

grammatical but

are sometimes stiff

or choppy.

Contains

occasional spelling

errors

Some errors in

grammar and word

choices. Some

distracting spelling

errors that need

proof reading

spelling errors,

punctuation,

typographical errors

–overly distracting

Presentation* +1 mark : assignments must be typed with proper font etc

TOTAL out of 25

ED153 Marking Rubric Reorganised using RSD

ED153: Results

• Gained a better & more holistic view of the features of research

• Redistributed weighting to reflect specific phases & process of research as reflected in the report

• Important features of research were brought to the forefront of the rubric. E.g. communication

• Marks carried more meaning at a glance

• Complemented the comments within the assignments

• Students were generally satisfied, judging from the almost zero rate of complaints about marking

RSD Implementation in CH101 Chemical Principles

CH101 RSD Assignment

The objective of the experiment was to determine the amount of calcium carbonate in eggshells, a routine but very relevant analysis. The goal of this experiment was to, among the learning of the method:

• To provide a "feel" of real time analysis of common food/items to students

• To also impart a sense of multiple analysis, which is a core component of scientific research

• CH101 RSD Marking Rubric for Lab Report

CH101: Student Feedback

Strongly

Agree

Somewhat

Agree

Somewhat

Diasgree

Strongly

Disagree Undecided

1 2 3 4 0

QUESTION Score Rationale

I have a complete understanding of the Rubrics 2 General "feel"

The Rubrics are too demanding for us as first year students 2 Issues such as adjustment

The Rubrics make writing labs easier 2 Are the rubrics helping?

I generally have difficulty in writing scientific reports 2 Baseline for lab writing skills

The Rubrics tell me what the Instructors expect of me in write-ups 1 Are the rubrics guiding?

I have never referred to the Rubrics at all in my Practical report writing 3 Disinclined?

Having Rubrics in labs has resulted in higher marks for me 2 Improvement noted?

I have difficulty in identifying/sourcing the information required by the Rubrics 2 Are the rubrics too complex?

The Rubrics have helped me improve my research and report writing skills to some extent 2 Is any perceived progress evident?

I prefer no Rubrics as my marks used to be higher without Rubrics 3 Failed experiment by instructors?

I can easily satisfy the requirements for the Above Benchmark category in my lab reports 2 A self-review

I do not like Rubrics as all I want is a pass in my practical reports 3 Generally content with pass

I get more valuable feedback when using Rubrics 2 A desire to improve

I can only manage a lab report that satisfies the quality of Benchmark and/or below it 2 Rubrics setting the bar too high?

I love the University style of laboratory classes 2 General enthusiasm - bias?

I need more time to adjust in writing up lab reports using the Rubrics 2 I can do it, but give me time!

The timeframe given (2 weeks) is not sufficient enough for me to closely follow the Rubrics 2 Time constraints?

CH101: Observations and Possibilities

• Previously marks in practical reports ranges between 14-18.5.

• Students appeared to have understood and followed the rubrics quite well and scored between 15-17

– (though there was scope for improvement particularly in the discussion of their results), there were a handful of students who wrote quite brilliantly and the amount of effort they had put in was evident.

• Key places where students still need to build on is critical thinking and discussion of results (justification of their findings with scientific reasoning)

Conclusions and Way Forward

• It can be seen that the RSD framework has been

contextualized to develop marking rubrics for assignments in different disciplines, providing a common platform to assess research literacy and skills across 4 different disciplines.

MG101 ‘Introduction to Management’, UU204 ‘Pacific Worlds’, ED153 ‘School and Society’, CH101 ‘Chemical Principles’.

Conclusions and Way Forward

Benefits/ Advantages reported:

• RSD framework allows for explicit articulation of research components and assessment of research literacy and skills that were not previously assessed in the course. (MG101, UU204 – credibility of sources, ED153 – communication, CH101- objectives and research questions).

• RSD based rubrics allowed for explicit communication of assignment expectations to students. Less student queries noted on marked assignments – MG101.

• More consistent marking across assignments reported when using the RSD based marking rubrics in MG101.

• Use of RSD based marking rubric allowed for a more systematic marking and subsequent reduction in marking time – MG101, possibly in ED153.

• Allowed for assessment of other USP Graduate Outcomes - Communication, Critical Thinking, Creativity. (Refer to Matrix)

Conclusions and Way Forward

Challenges:

• ‘Defrosting’ the research facets to students.

Challenge for students to understand the ‘facet’ expectations, and to relate facets to marking criteria.

• Wording used in rubrics (UU204, MG101).

Conclusions and Way Forward

Recommendations/ Way Forward

• Gradual introduction and assessment of RSD facets from the beginning of the semester (e.g CH101). Allow students time to get familiar with the expectations of RSD rubrics.

• Work on ‘defrosting’ of RSD facets and expectations of marking criteria, for example in tutorials.

• Simplification and fine tuning of RSD marking rubrics (UU204, MG101).

• To plan for further workshops on integration of RSD in rubrics.

• Expand on monitoring and evaluation as implementation progresses.

• Moving forward, use assessments at the 100-level as baseline for developing assessments at 200-level and 300-level to enhance student level of autonomy in research.

THANK-YOU!

Questions and Comments