Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Institute ofChemistry Education
Challenging student intuition with item designMICER III Zoomposium, 18.06.2020, Prof. Nicole Graulich
#nik_kola82 Graulich CER Group: www.uni-giessen.de/dc
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
How do people make decisions and what does this mean for item design in CER?
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
PRE-READING:Graulich, N., Hedtrich, S. and Harzenetter, R. Explicit
versus implicit similarity – exploring relational conceptual understanding in organic chemistry, Chem.
Educ. Res. Pract., 2019, 20, 924-936.
René Harzenetter Sebastian Hedtrich
Roadmap
1. Intuitive reasoning impacts students’ decision-making processes
2. An exploratory study to challenge student surface reliance – approaches and struggles (pre-reading)
3. Where to go from there? - Ideas for continuing research
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
How do people make decisions?
System II Thinking• analytical• slow• not effortlessly
System I Thinking• fast• intuitive• effortlessly• unconcious
Evans, J. S. B., In two minds: dual-process accounts of reasoning. 2003 Trends in Cognitive Sciences, 7(10), 454-459.
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
Which of these two cities is larger?
Essen Frankfurt
A B
Goldstein, D. G., & Gigerenzer, G. 1999. The recognition heuristic: how ignorance makes us smart. InG. Gigerenzer, & P. Todd (Eds.), Simple heuristics that make us smart. New York: Oxford University Press.
Knowing less can, depending on the question, be more effective than knowing more.
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
Which person would you consider more friendly?
A B
Monin, B., The Warm Glow Heuristic: When Liking Leads to Familiarity, 2003, Journal of Personality and Social Psychology, 85, 1035-1048.
We tend to substitute a difficult (or even impossible) question by a simpler one.
attribute substitution effect
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
Which are the appropriate reagents for this reaction?Attribute substitution effect
Graulich N., Intuitive Judgments Govern Students’ Answering Patterns in Multiple-Choice Exercises in Organic Chemistry, 2015, J. Chem. Educ., 92, 205-211.
Hanna: “I would go with d.) There are two chlorines in the product, so with Cl2 and CH2Cl2, I guess, you have enough chlorines to make it.“
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
„Students often apply rules memorized from particular examples to new questions which they deem to be similar based upon surface features of the molecules. […],
which lead them to incorrect answers.“
Cl+ CH3OH methanol
+ NaI acetoneBr
de Arellano, D. C. R. and Towns, M., Students understanding of alkyl halide reactions in undergraduateorganic chemistry, 2014, Chem. Educ. Res. Pract. 15, 501-515.
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
„Students are engaged in rote memorization of what features are related to nucleophilic and electrophilic behavior, rather than try to more deeply comprehend the
relationships between those features and functionality.“
Anzovino, M. E. and Bretz, S. L. Organic chemistry students' ideas about nucleophiles andelectrophiles: the role of charges and mechanisms, 2015, Chem. Educ. Res. Pract., 16, 797–810.
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
„The student focused most of the attention on those explicit structural cues that differentiated the three chemical compounds, implicitly discarding (i.e., benzene ring) or explicitly eliminating (i.e., –OH group) features that were actually relevant for the
successful prediction of acid strength.”
McClary, L.; Talanquer, V., Heuristic Reasoning in Chemistry: Making decisions about acid strength. Int. J. Sci. Educ. 2011, 33 (10), 1433-1454.
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
Acid-Base theory
Stereochemistry
Nucleophiles-Electrophiles
Polarity
Stability
Driving force
Kinetics
Electron movement
Reaction types
Electronegativity
Steric hindrance
Thermodynamics
π- and σ-bonds
Ring strain
Geometry
Resonance
Hydrogen bonds
Hyperconjugation
Review: Graulich, 2015, Chem. Educ.
Res. Pract., 16, 9-21.
students’ success in OC depends on the ability to:
(1) establish connections between the chemical properties and the structural representation
(2) consider implicit properties of molecules(3) determine which implicit property is relevant
in a problem context, despite explicit cues.
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
Acid-Base theory
Stereochemistry
Nucleophiles-Electrophiles
Polarity
Stability
Driving force
Kinetics
Electron movement
Reaction types
Electronegativity
Steric hindrance
Thermodynamics
π- and σ-bonds
Ring strain
Geometry
Resonance
Hydrogen bonds
Hyperconjugation
What type of items would capture this relational conceptual
understanding?
Review: Graulich, 2015, Chem. Educ.
Res. Pract., 16, 9-21.
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
A B C
Item Design Background
Inspired by:
• Variation theory of learning (Bussey, T. J. et al. 2012)
• Learning through contrasts (Schwartz, D. L. et al. 2011)
Explicit versus implicit similarity
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
Which two food items contain more sugar?
A B C
Item Design Background
Inspired by:
• Variation theory of learning (Bussey, T. J. et al. 2012)
• Learning through contrasts (Schwartz, D. L. et al. 2011)
Explicit versus implicit similarity
Poll #2
A: A and B
B: B and C
C: C and A
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
Which two nucleophiles would react similar with this bromalkane in a substitution reaction?
Item Design
A B C
Br ?
O
O
O
C N
A
B
C
Br ?
O
O
O
C N
A
B
C
Br ?
O
O
O
C N
A
B
C
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry EducationItem Design (SN1/SN2 reactions)
General Prompt: Indicate the two structures that would react similar in a substitution reaction.
nucleophilicity
Br ?
O
O
O
C N
A
B
C
Br BrBr
CBA
hyperconjugative effects
Cl
C N?
SN2
OH
A B C
CCl4
Br OCH3Cl
A B C
solvent effects
leaving group
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
LEADING QUESTION
How do students judge a comparable reactivity in items that are explicitly distracting?
How does students’ level of elaboration, when asked to provide a reason for their choice, relate to their performance?
Quantitative study, 156 students (chemistry majors / student teachers), 1st year OC
Paper-Pencil:• Part 1 – Multiple Choice• Part 2 – Multiple Choice and elaboration• Instrument: substitution reactions, 68 items in total• Concepts: Leaving group, hyperconjugative effects, solvent effects, nucleophilicity
Analysis• statistical analysis• rubric for the coding student elaboration
Study Design
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry EducationHow to categorize students' elaboration?
A Both are secondary halogen alkanesB Both react similar as both are good leaving groupsC Halogens are able to better stabilize a negative charge when leaving the moleculeD A and B have a high electronegativity.
Indicate the two structures that would react similar in a substitution reaction.
Br OCH3Cl
A B C
leaving group
Students’ answers for choosing A and C:
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry EducationRubric for students’ elaboration
Student Examples:explicit implicit
descriptive
functional
Both are secondary halogen alkanes
Both react similar as both are good leaving groups
Both halogens are able to better stabilize a negative charge when leaving the molecule
A and B have a high electronegativity
E1
E2
E3
E4
Deductive coding
a. Both are secondary halogen alkanesb. Both react similar as both are good leaving
groupsc. Halogens are able to better stabilize a
negative charge when leaving the moleculed. A and B have a high electronegativity.
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry EducationResults
This is what we got from a first analysis!
These items were not all answered in the same way
dist
ribut
ion
of st
uden
ts’ m
ean
scor
es
item type 1
item type 2
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry EducationResults
Br OCH3Cl
A B C
leaving group
Item type 1 - supporting Item type 2 - distractingExplicit features support item solution Explicit features do not support item solution
nucleophilicity
Br ?
O
O
O
C N
A
B
C
Reorganization of items
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry EducationResults
Box Plots after reorganization
• large effect size for the item types (Cohen‘s d=1.4)
• small effect size for the elaboration prompt in both item types (type 2: d=0.31, p=0001 and type 1: d=0.26, p= 0.009)
Part 1without elaboration
Part 2with elaboration
type 1supporting items
type 2distracting items
dist
ribut
ion
of st
uden
ts’ m
ean
scor
es
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry EducationStudents’ elaboration
• Students, who elaborate on the lower levels have a higher chance of solving the item of type 2 incorrectly.
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry EducationWhere to go from here?
• Assessing students’ understanding can be misleading, when the correct solution is possible based on surface features.
• Combining assessment with students’ deeper reasoning, e.g. two-tiers MCQs (Treagust, D. F. 1988).
• Supporting students to activate deep conceptual reasoning does not come deliberately and already starts during instruction
What are you doing while teaching to support students to become more reflective about their intuitive approaches?
THINK
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry Education
Get to know each other and exchange your experiences.
THINK What are you doing while teaching to support students to become more reflective about their intuitive approaches?
BREAKOUT
CHAT Decide which idea/question you want to share as a group via the chat box.
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry EducationMaking the unconscious conscious
Students’ 1st attempt to a prompt
Analyzing worked-example, expert- or peer-solutions (e.g. work by E. Yuriev; Finkenstaedt-Quinn et al., 2019)
PROMPT Revision of 1st
attempt
Prompt to reflect on possible erroneous solutions (e.g. Talanquer, 2017)
Learning with material (e.g. conflicting animations, Kelly et al., 2017)
….
metacognitive „loop“
• How can I determine changes, what can be taken as evidences?
• Do changes last? How I can observe this?• How are learned abilities used in other
contexts?
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry EducationHuge Thanks to !
... Michael & Aishling for just being fabulous
... an open and respectful CER community
... every one of you for being here today!
... a fantastic CER research group in Gießen
Reach out to us!!
Graulich CER Group: www.uni-giessen.de/dc
Institute ofChemistry EducationResources
• Evans, J. S. B., In two minds: dual-process accounts of reasoning. Trends in Cognitive Sciences 2003, 7(10), 454-459.• Talanquer, V., Concept Inventories: Predicting the Wrong Answer May Boost Performance. J. Chem. Educ. 2017, 94 (12), 1805-1810.• Posner, G. J.; Strike, K. A.; Hewson, P. W.; Gertzog, W. A., Accomodation of a scientific conception: Towards a theory of conceptual change. Sci. Educ. 1982, 66, 211-227.• Goldstein, D. G., & Gigerenzer, G. 1999. The recognition heuristic: how ignorance makes us smart. In G. Gigerenzer, & P. Todd (Eds.), Simple heuristics that make us smart. New York:
Oxford University Press.• Monin, B., The Warm Glow Heuristic: When Liking Leads to Familiarity, Journal of Personality and Social Psychology 2003, 85, 1035-1048.• De Arellano, D. C. R.; Towns, M., Students understanding of alkyl halide reactions in undergraduate organic chemistry. Chem. Educ. Res. Pract. 2014, 15, 501-515.• Anzovino, M. E. and Bretz, S. L., Organic chemistry students' ideas about nucleophiles and electrophiles: the role of charges and mechanisms, Chem. Educ. Res. Pract. 2015 16, 797–810.• Graulich, N., Hedtrich, S. and Harzenetter, R., Explicit versus implicit similarity – exploring relational conceptual understanding in organic chemistry, Chem. Educ. Res. Pract. 2019, 20, 924-
936.• Graulich, N.; Hopf, H.; Schreiner, P. R., Heuristic thinking makes a chemist smart. Chem. Soc. Rev. 2010, 39 (5), 1503-12.• Graulich, N. The tip of the iceberg in organic chemistry classes: how do students deal with the invisible? Chem. Educ. Res. Pract. 2015, 16, 9-21.• Graulich, N. Intuitive Judgments Govern Students‘ Answering Patterns in Multiple-Choice Exercises in Organic Chemistry. J. Chem. Educ. 2015, 92, 205-211.• Graulich, N.; Bhattacharyya, G. Investigating Students’ Similarity Judgments in Organic Chemistry, Chem. Educ. Res. Pract., 2017, 18, 774-784.• Bussey, T. J.; Orgill, M.; Crippen, K. J., Variation theory: A theory of learning and a useful theoretical framework for chemical education research. Chem. Educ. Res. Pract. 2013, 14 (1), 9-22.• Schwartz, D. L.; Chase, C. C.; Oppezzo, M. A.; Chin, D. B., Practicing versus inventing with contrasting cases: The effects of telling first on learning and transfer. J. Educ. Psychol. 2011, 103
(4), 759.• Kelly, R. M.; Akaygun, S.; Hansen, S. J. R.; Villalta-Cerdas, A., The effect that comparing molecular animations of varying accuracy has on students' submicroscopic explanations. Chem.
Educ. Res. Pract. 2017, 18 (4), 582-600.• Finkenstaedt-Quinn, S. A.; Snyder-White, E. P.; Connor, M. C.; Gere, A. R.; Shultz, G. V., Characterizing Peer Review Comments and Revision from a Writing-to-Learn Assignment Focused
on Lewis Structures. J. Chem. Ed. 2019, 96 (2), 227-237.• Treagust, D. F. Development and use of diagnostic tests to evaluate students’ misconceptions in science. Int. J. Sci. Educ. 1988, 10, 159–170.
Online resources:• Intuitive concept inventory: https://www.surveygizmo.com/s3/4364595/ICI• V. Talanquers Webinar: https://rsccerg.wordpress.com/2018/04/30/vicente-talanquer-webinar-23rd-may/• Stacey Bretz Webinar: https://www.youtube.com/watch?v=WfSh4r3wtR0