An experience using degrees of certitude

Excel spreadsheet to suppor t veterinar y student’s

metacognition

Jean-Loup CastaigneIFRES — LabSET

University of Liege, Belgium

EARLI — SIG 16 Metacognition, 2nd Biennial Conference — Friday 21 July 2006

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Liege, the Vet facultyGeographical situation

Who am I?

What teaching am I involved with?

Reproduction of the cow and the mare

5th year out of 6 — students’ mean age of 24

390 students — 75% of girls — 10% of students with interest in cattle and horses.

2

Excel spreadsheet to support veterinary

student’s metacognition

3

Outline

• Theoretical background

• Presentation of the tool

• Survey

• Conclusions


“Regulative” metacognitionBernadette Noël (1991)

Metacognitive judgment

Metacognitive process: being aware

Metacognitive decision: decide to or not to regulate

4


student’s metacognition Outline



• Survey

• Conclusions


Teaching

Study

formative test 1

September

Methodology (1 year)

Teaching Studyformative

test 2December

certification test 1 (January)

TeachingStudyformative test 2February

StudyOralexam(June)

Teaching Study certification test 2 (March)

6from Edgar P. Jacobs 1956 - The Yellow "M"

ISBN 2-87097-010-2

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006 7

Weight of each testFormative test #1

Formative test #2

Certification test #1

Formative test #3

Certification test #2

Final oral examination

100%


Pay-off for MCQ“cognitive” score

Correct Answer (CA): +1Wrong Answer (WA): -0,5No answer (NoA): -0,25

Freedom to use, or not toconfidence marking for MCQself-estimation for open questions

8


Hypothesis

If confidence marking does NOT influence the results of the test, students will be more sincere giving the confidence they associate with the chosen answer.


Are the students sincere?

10

Certitude

0-25% 25-50%

50-70%

70-85%

85-95%

95-100%Référen

ce12,5 % 37,5 % 60 % 77,5 % 90 % 98,5 %

cochez 0 1 2 3 4 5RC +13 +16 +17 +18 +19 +20RI +4 +3 +2 +0 -6 -20


In certification tests including confidence marking, I choose maximum 60% and less if I doubt because for an incorrect answer with a confidence of 80% ou 100% penalize my score.

Survey (n=369)

11

Confidence

0 % 20 % 40 % 60 % 80 % 100 %Thick 0 1 2 3 4 5

CA +13 +16 +17 +18 +19 +20

IA +4 +3 +2 +0 -6 -20


5 metacognitive measures

Individual test with comments

Global view of a# 5 testsHelp


Computing and drawing


Spectral distribution

no a

nsw

er

Incorrect answers

Confidence : from -100 to -0

Correct answers

Confidence : from 0 to +100

Num

ber

of a

nsw

ers

Jans (1999)

14


Three types of knowledge situation

misinformed

uninformed

informed


15

Hunt (1993)



no a

nsw

er

misinformed


informed


Num

ber

of a

nsw

ers

16

uninformed

Hunt (1993) Jans (1999)



no a

nsw

er

misinformed


informed


Num

ber

of a

nsw

ers

Hunt (1993)

17

uninformed

Jans (1999)

DangerousKonwledge

UsableKnowledge

unusable knowledge

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006Confidence : from -100 to -0Confidence : from 0 to +100

Num

ber

of a

nsw

ers

Four zonesUnawareness

Usableknowledge

Leclercq 2003

18

Mid knowledge

Dangerousknowledge


Drawing of graphs

19


adapted "om Leclercq & Gi$es 1995Chosen confidence

Perc

enta

ge o

f CA

Realism graphRegarding all MCQ for which you choose a confidence of 40%

For those MCQ you’ve reached a score of 63% of correct answers

You’ve succeeded more (63%) than you estimate (40%): thus in the class of 40% you underestimate your level

20


Objective judgments & analysis

Comments: evaluated fieldsCognitive field

Metacognitive field

Subjective judgments & analysis

Metacognitive regulation

21





• Survey

• Conclusions


I’ve progressed in metacognition only because I progressed in knowledge in reproduction

24

Cognition vs. Metacognition


Filling the spreadsheet pushed me to study more reproduction

Regulation “product”

25


Regulation “process”

26

Because of what I’ve learned filling the spreadsheet I’ve change my method of working


Because of what I’ve learned filling the spreadsheet I’ve changed the way I use confidence marking

27

Regulation “process”





• Survey

• Conclusions


Conclusion

29

ProductsProductsProducts ProcessessProcessessProcessesspre per post pre per post

Judgment ✔

Analysis ✔ ✔

Regulation ✔

Noël (1991)

“Regulative” metacognition

✔

✔

✔

✔

✔

✔

✔✔


Conclusion“ I’m not sure that this system helps us to understand better success or failure. It is very complex and it doesn’t matter commenting numbers or graphs, it’s always theoretical…”

“I was really dubious regarding the efficiency of this system but my own progresses proves me that it was hugely profitable.”

30

Thank you for your attention.

Education

An experience using degrees of certitude