RSD Framework Recent Progress in the Electrical ...€¦ · RSD Symposium, Melbourne, Nov 27 2012 Details: RSD for Masters (Defence) Minor Thesis • Duration of the project – minor

RSD Symposium 2012

Nov 27, Melbourne

Monash University, Caulfield Campus

Brian Ng

Said Al-Sarawi

RSD Framework – Recent Progress in the Electrical & Electronic Engineering Context

Life Impact The University of Adelaide

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Outline of Presentation

• History of RSD Framework in EEE

• Why RSD made sense for EEE

• A Way Forward

• Wrap up and discussions



History of RSD Framework in EEE

• Timeline

• 2006: Applied to supervision and assessment of research project

component inside Masters coursework

• 2007: Second cohort of Masters coursework student; assessment

framework for Defence Masters thesis

• 2009: Trial of using RSD for Honours capstone project assessment

• 2010-current: School-wide use of RSD for Honours project assessment

• Findings:

• Moderately positive outcomes identified, e.g. reduced plagiarism

• School wide adoption is difficult, and required (significant) annual

tweaking

• Enjoys a good level of backing/support

• Various initiatives to “upscale downwards”

• RSD workshops, pilots

• School level engagement



Why RSD made* sense for EEE?

• Provides clear and explicit assessment criteria

• Highly transparent assessment process

• Verbose rubrics allows for communication to a range of assessors

• Assessment process emphasises development and assessment of

research skills

• Valuable information for aspiring students

• Descriptors for skills which need be developed (and assessed

against!)

• BUT each of these also had its corresponding downsides

*and still makes!



Brave New World – 2014 and Beyond

• Student interest EEE has declined over the past 10 years

• In favour of other disciplines such as Mechanical

• New educational environment

• Teaching modes and emphases change with technology

• Employment landscape also much different

• Need to produce newer breed of graduates

• External drivers loom large on the horizon, e.g. AQF8, international

market

• Curriculum refresh cycle is an opportunity to address these

• Much soul searching as a School?



EEE Workshop 2012

• School of E&E Engineering convened for a series of 4 workshops in

semester 2

• Key stakeholders were in attendance

• Aim for some “grand design” curriculum to address raised issues

• Discussions

• High level graduate attributes

• Pathway to developing them over 4 years

• Practical constraints

• Pedagogical framework



Characteristics of Our B.E.

• Multiple facets: core science, technologies & applications,

professional practice, research



Pedagogical Model

• Emphases on Engineering Science and the System Engineering

methodology

• Some subtle difference to the scientific method

• Latter is highly relevant to workplace practices for

contemporary engineering in Australia



Consequences and Challenges

• Fundamental shift in thinking for our School’s teaching

• Current Levels 1 and 4 can remain with minor tweaking

• Revamp Levels 2 and 3

• “construct a bridge to connect the two banks”

• Significant investments in the coming years

• Delivery and assessments will need to be revised

• Material may not need much modification beyond repackaging

• No longer be obsessed with coverage

• Pace of change will need careful management

• Where to find time to do all this?



Role of RSD in All This …

• Critical to get the “language” right – RSD is ready to inform this

process

• Much discussions on how students evolve over 4 years

• or

• Development framework

• Scaffolding for group work is novel (for us!)

• Assessment framework – rubrics



Wrapping Up and Discussions

• Work in progress at Program level • Many factors driving this process: high complexiy

• Process needs completion by 2013, deployment 2014

• Contact us! [email protected] [email protected]

Many thanks to generous support from library services at

Adelaide, and John, Lyn, Leanne, Cindy, Irene and all

other organisers of this Symposium

mailto:[email protected]

mailto:[email protected]



Research Skills Development Framework

• Framework developed by Willison and O’Regan (2007)

– “Commonly known, commonly not known, totally unknown: a framework for

students becoming researchers”

• Makes explicit and coherent the incremental attainment of research

skills

– Emphasis on degree of autonomy in inquiry

– Context sensitive

– Useful for generating marking rubrics to drive learning activities towards

objectives

• RSD framework used in various disciplines

– Diverse range of disciplines at University of Adelaide

– Participation at other institutions, e.g. Monash, Macquarie, Melbourne, South

Australia



Details: Use in Honours Project

• Traditional “cap-stone” project level 4 (Honours)

• 6 units; multiple objectives

– Professional practice skills, project management

– Technical outcomes are vital

– Research skills development embedded

• Major revamp in 2010

– RSD is identified as a suitable framework for research facet (guidelines)

– New assessment process for all project components (policy)

• Assessment matrices were developed using the RSD framework

– Make the once implicit explicit

– Serve to communicate feedback to students

– Also excellent documents to base conversations on



2010/11 Honours project assessment components

Activity Type Group / Individual Weight Due Assessor(s)

Proposal seminar Formative Group 5% Sem A week 3 Supervisor, co-supervisor

Stage 1 progress report

Formative Group 10% Sem A week 5 Supervisor, co-supervisor

Stage 2 progress report

Formative Individual 10% Sem A week 12 Supervisor, co-supervisor

Semester A performance

Summative Individual 10% Sem A week 12 Supervisor, co-supervisor

Final seminar Summative Group 10% Sem B break Independent assessor

Final report Summative Individual 20% Sem B week 11 Supervisor, independent

assessor

Project exhibition Summative Group 5% Sem B week 12 Independent assessor

Final performance Summative Individual 20% Sem B week 12 Supervisor, independent

assessor

Outcomes Summative Group 10% Sem B week 12 Independent assessor



2012 Honours project assessment components

Activity Type Group / Individual Weight Due Assessor(s)

Proposal seminar Formative Individual 10% Sem A week 3 Supervisor, co-supervisor

Stage A progress report & performance

Summative Individual 25% Sem A week 12 Supervisor, co-supervisor

Final seminar Summative Individual 15% Sem B break Panel of > 3 assessors

Final report Summative Individual 20% Sem B week 11 Supervisor, co-supervisor

Project exhibition Summative Group 10% Sem B week 12 Panel of > 3 assessors

Final performance Summative Individual 20% Sem B week 12 Supervisor, co-supervisor



RSD for Honours: Issues and Lessons

• Significant investment in effort for School staff

– Especially “core” personnel e.g. course coordinators, champion

• Transition arrangements must be communicated to students and

staff members alike

– Staff training on assessment principles would particularly be useful

• New for many students

– Increases quantity, and subsequently, the demand for feedback

– Interpretation issues for descriptions

• Acceptance issues can arise

– Among staff members: considered a waste of time

– Inhomogeneous application across the course

– Impact with regards to workload, supervision, feedback, … etc

• Evaluation should be integrated

– Embedded throughout the year

– Student interviews? Other reflections?



Details: RSD for Masters (Defence) Minor Thesis

• Duration of the project – minor thesis : – 0.5 full time equivalent workload (12 units)

• Aims: – To train students in research and development methodology and

techniques

– Develop critical evaluation skills appropriate to their project topic

– Develop project management experience

• Outcomes: students are expected to gain skills in

– Engineering design, research and verification

– Critical evaluation of findings

– Project and time management



Past Minor Thesis Assessments

• Students shall be assessed on: – the performance of research, planning, execution and management of

the project

– the quality of the project deliverables

– Written thesis as a standalone document

• Shortcomings of existing approach: – Clear assessment criteria do not exist

– It was hard to describe the assessment to non-academics

– It does not allow for direct assessment of research skill development

– Requirements to achieve various grades are not clear

– No explicit information on which skills need to be developed



Minor Thesis Assessment Example

• Overall comments on the thesis and research outcome

• General feedback

• Technical feedback

• Pedagogical Assessment of the thesis from RSD

Framework point of view



Pedagogical Assessment of Minor Thesis using RSD Framework

A. Candidate embarks on an inquiry and so determine a need for

knowledge/understanding – demonstrated level is 4 – 5/10

B. Candidate finds/generates needed information/data using

appropriate methodology – demonstrated level is 4 – 7/10

C. Candidate critically evaluate information data and the process of

find/generate this information data – demonstrated level 4 – 7/10.

D. Candidate organizes information collected/generated –

demonstrated level is 4 – 7/10.

E. Candidate synthesis, analyse and apply new knowledge –

demonstrated level 4 – 7/10.

F. Candidate communicates knowledge and the processes used to

generate it, with an awareness of ethical, social and cultural issues

– demonstrated level 4 – 5/10.



Final Assessment Matrix

Documents

RSD Framework Recent Progress in the Electrical ...€¦ · RSD Symposium, Melbourne, Nov 27 2012 Details: RSD for Masters (Defence) Minor Thesis • Duration of the project – minor