Equitable assessment of the practical component of scientific modules

Equitable assessment of the practical component

of Scientific Modules

Darvree Downey and Maeve ScottScience Department

Institute of Technology Tallaght

1

2

Students value laboratory

practicals as much for their role in

illuminating and reinforcing

theory as for the opportunity to

learn the ‘tools of the trade’

(Willmott, 2005).

• Background• Objectives

Study

• Descriptions• Delivery of practical element

Module

• Statistical methodsMethod

Results

Conclusions

Contents

3

Laboratory Assessment ~ 35% of module

marks

Laboratory assessment aims to assess

achievement of the practical-component

learning outcomes.

Changes have been introduced in laboratory

assessment practices in ITT in recent years.

Educational Situation

4

Since the last programmatic review (in 2006) there

is no failed element in any of the laboratory

subjects in ITT’S Science Department.

Anecdotal belief that students are getting sufficient

marks to pass without achieving core learning

outcomes

Educational Situation

5

How does laboratory assessment equate with

students achievements in other summative

assessments within the same module?

Is correlation with other forms of assessment

indicative that the laboratory assessment is

equitable?

Study Background

6

Practical Component Learning Outcomes can include:

Interpretation of legal requirements

or guidelines

Use of analytical equipment

Application of calculations and

statistics

Analysis, interpretation and

application of published resources

Critical evaluation of results Organisational skills

Time management Meeting industry needs

7

The practical component of laboratory

exams are designed to assess laboratory

technique and equipment usage skills.

Learning outcomes common to both theory

(final module exam) and laboratory exams

include organisational and time

management skills and data analysis skills.

8

Learning Outcomes

To determine if current laboratory

assessment methods used for a range of

scientific modules in ITT Dublin are

1. Reliable indicators of a student’s ability to

meet the overall module learning

outcomes.

2. Equitable for all learning styles

Objectives

9

Students have diverse learning styles so

assessment methods should vary for an

equitable approach.

Summative assessment methods should not

disadvantage any group of students.

Inclusive and Diverse Assessment

10

The practical element of a course can be

assessed a number of ways to determine

the same learning outcomes. All

assessment methods have benefits and

limitations.

Assessment methods

11

Suits students with good language and planning skills

Can disadvantage some learners, in particular

Students with Dyslexia

Students for whom English is a second language

Auditory learners

Students with organisational difficulties

Too much time spent writing reports at the expense

of getting to grips with the subject matter (Race,

2006)

Laboratory Reports

12

1. Originality?

◦ Anti-plagiarism software

2. High marks not necessarily reflective of

laboratory competence

◦ Revision of marking schemes

◦ Introduction of practical lab exams

◦ In-class questions & observation

Laboratory reports

13

Helping the students to think about the

quality of their own work (Rust, 2005)

Generation of internal feedback

Self Assessment of Laboratory Reports

14

Formal oral laboratory assessment

exams can disadvantage students with

poor auditory and language skills

Challenging for students who suffer

from anxiety

Preferred by auditory learners and

students who have poor written skills.

Oral Assessment

15

Questioning of students during laboratory

sessions can assess the preparation,

depth of learning and organisational

skills.

difficult to assess through other methods?

Oral Questions in Laboratory Practicals

16

◦ Students with poor language skills can have

difficulties with providing articulate responses

◦ Students with auditory problems can have

comprehension difficulties.

◦ Questions during laboratory work can add

additional stress to students with poor

organisational skills such as those with specific

learning difficulties

Oral Questions in Laboratory Practicals: Disadvantages

17

A chance to assess students’ laboratory technique and data

analysis skills

Should suit sensing learners

Reflective learners can find practical skills assessment

stressful

Dyslexic students and students with other learning difficulties

may not be given extra time/special consideration

18

Laboratory Exams

Method

19

4 modules:

◦ Systems Validation (Pharmaceutical Science, Semester 7)

◦ Scientific Analysis (Pharmaceutical Science, Semester 4)

◦ Biochemistry (Bioanalysis & DNA&Forensic Science, Semester 4)

◦ Principles of Biochemistry (Pharmaceutical Science, Semester 3)

Final exam as marker of module competence

Lab assessments methods as predictors

Statistical analysis: Regression

Method

20

Analysis

21

Laboratory Report Marks as a predictor of Final Exam Marks: Systems Validation Semester 5

22

23

Comparison of Laboratory Report Marks and Laboratory Exam Marks as a predictor of Final Exam Marks: Principles of Biochemistry, Semester 3

Suggests need for failed element?

24

Laboratory Report Marks as a predictor of Final Exam Marks: Biochemistry, Semester 4

25

Laboratory Exam Marks as a predictor of Final Exam Marks: Biochemistry, Semester 4

26

Why did the lab assessments in

2011 correlate with the theory

exam to a greater extent than in

previous years?

More in-class time spent on laboratory preparation & feedback? YES

Academically more capable students? NO

Suggestion of a ‘better teaching’ effect?

Expectations vs. Actual

27

28

Laboratory assessment and module exams measure different learning outcomes but there is crossover

Very high correlation not expected

Where there is very little correlation, it does suggest a disconnect

Intuitive learners struggling with practical skills?

Sensing learners struggling to relate the theory to the practical?

Are global learners losing out in the laboratory?

29

RecommendationsFor ITT

Re-instatement of failed element where applicable to

ensure theory and practical learning outcomes are met

Use of this type of analysis in all laboratory-based modules

to highlight inconsistencies and measure the effects of

year-on-year assessment changes

General

Use the wealth of data available to highlight potential areas

for improvement of summative assessment practices

Measure how learning style affects summative assessment outcomes for laboratory-based modules

Monitoring effects of new laboratory interventions on summative assessment outcomes

What next?: Future analysis

30

Broaden the range of lab assessment

methods…..

VLEs; Mindmaps; Reflective journals;

Posters,

Broaden use of questions in labs & oral

assessments

What next?: Changes to Summative Assessment Practices

31

32

Acknowledgement:Many thanks to James Reilly, ITT Dublin for his help and advice on statistical methodologies