Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Denise Clarkson, Willem Fourie, Dobrila
Lopez,
Mike Lopez, Kirsten Marais
Manukau
Institute of Technology
Paper presented at the Symposium on Tertiary Assessment and Higher Education Student Outcomes: Policy, Practice, and Research, Rutherford House, Wellington, New Zealand, 17-19 November 2008.
ContextResearch questionsCourses and sampleMethodResultsConclusionLecturer perspectiveQuestions
HistoricalModernPractitioner
Temple of Apollo at Delphi from belowPhotograph from the Wikimedia
Commons
Gnothi seautonKnow yourself
Socrates?Heraclitus?Pythagoras?From heaven?
Temple of Apollo at Delphi
Photograph from the Wikimedia
Commons
“The maxim “Know thyself” comes down to us from the heavens; it should be imprinted in the heart, and stored in the memory, whether you are looking for a wife, or wishing for a seat in the sacred Senate.”
Juvenal (Decimus lunius luvenalis) - circa 100AD The Satires of Juvenal, Satire 11, Dinner and a moral.
Gnothi seautonKnow yourself
Socrates?Heraclitus?Pythagoras?From heaven?
Everyone knows
Hi, my name is Mike and I’m an Alcoholic
Hi, my name is Mike and I’m a MathematicianAcknowledging where we are is the start of any journey of learningWe need to avoid judging the person
Issues for summative assessment?
We believe that self-assessment is a foundation capability for lifelong learning.This capability should be built progressively throughout a learner’s study.Self-assessment should be used for its believed contribution to learning, not as a means of generating marks.Marks (if any) should be awarded for the quality and perceptiveness of judgementsmade.
Do students value self-assessment?Do students believe professionals need to be able to judge their own work?
Does it help learning?Is it valid?Do students have the capability to assess?
sufficient subject knowledge and understanding of the marking criteria
Are students concerned about the workload involved?
We looked at self-assessment in three domains:Foundation studiesNursingComputing
218 participants66% female53% English as first language26% in first semester of study
Non-experimental, post test only, designQuantitative studyConversion of ordinal data to interval level measurement.Use of scales to reduce uncertaintyCompare scale means by t-testsNon parametric tests of raw responsesRegression used for background variable associations
Do students value self-assessment?t(217)=14.3691; p<.0001CI.99 = (6.5670 ≤ Valued ≤ 7.2557).
D01 The ability to assess my own performance is important 98%D03 The idea of self-assessment is a good one 93%
D16 A similar (but improved) scheme should be used in other courses
82%
D23 We should have more opportunities for self assessment 76%
Do students believe professionals need to be able to judge their own work?Supported (p<.0001)
D24 Most professionals need the ability to judge their own work.
90%
Does self-assessment help learning?t(217)=13.2929; p<.0001CI.99 = (6.6432 ≤ Learning ≤ 7.4380)
D07 I found assessing my own work to be valuable 90%D09 This exercise helped me make a realistic assessment of my own abilities 95%D10 I had to consider my own learning progress more closely than usual 96%D11 Having to justify my mark helped me focus on the value of the work
done96%
D12 This exercise made me more aware of what I need to know in this subject 96%
D02 Self-assessment is more demanding than having someone else assess me 81%
Is self-assessment seen as valid?t(217)=7.0193; p<.0001CI.99 = (5.5611 ≤ Valid ≤ 6.2167)
Negative codingD14 I found it difficult to follow the marking scheme 26%D18 The assessment scheme was unfair 13%D19 The whole exercise of self and peer marking was a waste of time 10%D21 I did not feel comfortable with the responsibility of marking my
own work
30%
Negative codingD22 It’s the lecturer’s job to mark my work. 53%D25 I don’t know enough to be able to judge whether work is good. 51%
Do students have sufficient subject knowledge and understanding of the marking criteria?t(217)=3.1983; p=0.0018CI.99 = (5.1159 ≤ Pknow ≤ 6.1004)
Negative coding (sub-scale of valid)D14 I found it difficult to follow the marking scheme 26%D25 I don’t know enough to be able to judge whether work is good. 51%
Are students concerned about the workload involved?Not significant (t(217)=1.7325; p=0.0807)
Negative codingD04 Self-assessment would be a good idea if we didn’t have such a high
workload76%
D15 I would like to see some changes in the procedure 46%D17 I don’t think the rewards were sufficient for the amount of time I spent 49%
Is self-assessment valued? Yes - strongProfessionals need to judge? Yes - strongDoes it help learning? Yes - strongIs it valid? Yes - strongDo students have capability? Yes - weakConcerned about workload? Not significant
But …
12% of participants agreed (or strongly agreed) that self-assessment was either:
A waste of time ORUnfair
Those who were working in the six months prior to the course were less likely to be opposed. (Beta = 0.2358; R2 = 6%; p< .01)Those who expressed a preference for a surface approach to studying were more likely to be opposed. (Beta = 0.1967; R2 = 4%; p< .01)
Others, although not strongly opposed, had some reservations about self-assessment.
Will this change over time if self-assessment is used progressively in more courses?
D22 It’s the lecturer’s job to mark my work. 53%D25 I don’t know enough to be able to judge whether work is good. 51%D14 I found it difficult to follow the marking scheme 26%D17 I don’t think the rewards were sufficient for the amount of time I spent. 49%
Criterion Predictor(s) Beta R2 Sig.
Opposed Surface approach 0.200 4% .0035
Capability Surface approach ‐0.299 9% <.0001
Valid Surface approach ‐0.187 3% .0057
Workload Surface approach 0.295 9% <.0001
Demanding Deep approach 0.230 5% .0008
Valued Strategic approach 0.323 10% <.0001
Learning Strategic approach 0.501 25% <.0001
Needed Strategic approach 0.209 4% 0.0021
Criterion Predictor(s) Key contrasts Beta R2 Sig.
Valid Gender Female = more valid 0.287 8% p<.0001
Capability Domain Computing = less 0.251 6% p=.0003
Valued Level Foundation = more 0.193 4% p=.0043
Learning Level Foundation = more 0.326 11% p<.0001
Workload Level Foundation = more 0.291 8% p<.0001
Demanding LevelAge
Marginally significant even with 2 predictors
0.1520.149 4% p=.0097
Most demanding:Level 5 Over 25
Students strongly endorsed the value and validity of self-assessment and the learning that resulted from it.They endorsed the belief that they had the capacity to do it, but this was less strong.They were neutral on workload concernsSome (12%) were strongly opposed to its use.
Denise ClarksonSchool of Foundation Studies
Engagement
Students became their own teachers
They could see purpose and that effort counted
Test conditions
They wrote their own feedback
Fostered discussion
Evaluation
Students practised critical thinking
Exam TechniqueBecame intimate with the marking schedule
Exposure Repeated exposure created improvement
Questions