An experience using degrees of certitude

Excel spreadsheet to suppor t veterinar y student’s

metacognition

Jean-Loup CastaigneIFRES — LabSET

University of Liege, Belgium

EARLI — SIG 16 Metacognition, 2nd Biennial Conference — Friday 21 July 2006

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Liege, the Vet facultyGeographical situation

Who am I?

What teaching am I involved with?

Reproduction of the cow and the mare

5th year out of 6 — students’ mean age of 24

390 students — 75% of girls — 10% of students with interest in cattle and horses.

2

Excel spreadsheet to support veterinary

student’s metacognition

3

Outline

• Theoretical background

• Presentation of the tool

• Survey

• Conclusions

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

“Regulative” metacognitionBernadette Noël (1991)

Metacognitive judgment

Metacognitive process: being aware

Metacognitive decision: decide to or not to regulate

4

Excel spreadsheet to support veterinary

student’s metacognition Outline

• Theoretical background

• Presentation of the tool

• Survey

• Conclusions

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Teaching

Study

formative test 1

September

Methodology (1 year)

Teaching Studyformative

test 2December

certification test 1 (January)

TeachingStudyformative test 2February

StudyOralexam(June)

Teaching Study certification test 2 (March)

6from Edgar P. Jacobs 1956 - The Yellow "M"

ISBN 2-87097-010-2

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006 7

Weight of each testFormative test #1

Formative test #2

Certification test #1

Formative test #3

Certification test #2

Final oral examination

100%

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Pay-off for MCQ“cognitive” score

Correct Answer (CA): +1Wrong Answer (WA): -0,5No answer (NoA): -0,25

Freedom to use, or not toconfidence marking for MCQself-estimation for open questions

8

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006 9

Hypothesis

If confidence marking does NOT influence the results of the test, students will be more sincere giving the confidence they associate with the chosen answer.

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Are the students sincere?

10

Certitude

0-25% 25-50%

50-70%

70-85%

85-95%

95-100%Référen

ce12,5 % 37,5 % 60 % 77,5 % 90 % 98,5 %

cochez 0 1 2 3 4 5RC +13 +16 +17 +18 +19 +20RI +4 +3 +2 +0 -6 -20

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

In certification tests including confidence marking, I choose maximum 60% and less if I doubt because for an incorrect answer with a confidence of 80% ou 100% penalize my score.

Survey (n=369)

11

Confidence

0 % 20 % 40 % 60 % 80 % 100 %Thick 0 1 2 3 4 5

CA +13 +16 +17 +18 +19 +20

IA +4 +3 +2 +0 -6 -20

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006 12

5 metacognitive measures

Individual test with comments

Global view of a# 5 testsHelp

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006 13

Computing and drawing

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Spectral distribution

no a

nsw

er

Incorrect answers

Confidence : from -100 to -0

Correct answers

Confidence : from 0 to +100

Num

ber

of a

nsw

ers

Jans (1999)

14

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Three types of knowledge situation

misinformed

uninformed

informed

Spectral distribution

15

Hunt (1993)

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Spectral distribution

no a

nsw

er

misinformed

Confidence : from -100 to -0

informed

Confidence : from 0 to +100

Num

ber

of a

nsw

ers

16

uninformed

Hunt (1993) Jans (1999)

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Spectral distribution

no a

nsw

er

misinformed

Confidence : from -100 to -0

informed

Confidence : from 0 to +100

Num

ber

of a

nsw

ers

Hunt (1993)

17

uninformed

Jans (1999)

DangerousKonwledge

UsableKnowledge

unusable knowledge

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006Confidence : from -100 to -0Confidence : from 0 to +100

Num

ber

of a

nsw

ers

Four zonesUnawareness

Usableknowledge

Leclercq 2003

18

Mid knowledge

Dangerousknowledge

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Drawing of graphs

19

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

adapted "om Leclercq & Gi$es 1995Chosen confidence

Perc

enta

ge o

f CA

Realism graphRegarding all MCQ for which you choose a confidence of 40%

For those MCQ you’ve reached a score of 63% of correct answers

You’ve succeeded more (63%) than you estimate (40%): thus in the class of 40% you underestimate your level

20

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Objective judgments & analysis

Comments: evaluated fieldsCognitive field

Metacognitive field

Subjective judgments & analysis

Metacognitive regulation

21

Excel spreadsheet to support veterinary

student’s metacognition Outline

• Theoretical background

• Presentation of the tool

• Survey

• Conclusions

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

I’ve progressed in metacognition only because I progressed in knowledge in reproduction

24

Cognition vs. Metacognition

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Filling the spreadsheet pushed me to study more reproduction

Regulation “product”

25

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Regulation “process”

26

Because of what I’ve learned filling the spreadsheet I’ve change my method of working

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Because of what I’ve learned filling the spreadsheet I’ve changed the way I use confidence marking

27

Regulation “process”

Excel spreadsheet to support veterinary

student’s metacognition Outline

• Theoretical background

• Presentation of the tool

• Survey

• Conclusions

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Conclusion

29

ProductsProductsProducts ProcessessProcessessProcessesspre per post pre per post

Judgment ✔

Analysis ✔ ✔

Regulation ✔

Noël (1991)

“Regulative” metacognition

✔

✔

✔

✔

✔

✔

✔✔

Jean-Loup Castaigne, EARLI-SIG Metacognition, Cambridge 2006

Conclusion“ I’m not sure that this system helps us to understand better success or failure. It is very complex and it doesn’t matter commenting numbers or graphs, it’s always theoretical…”

“I was really dubious regarding the efficiency of this system but my own progresses proves me that it was hugely profitable.”

30

Thank you for your attention.