Upload
saad-chahine
View
381
Download
0
Embed Size (px)
DESCRIPTION
This presentation, was given at ICSEI 2012, in Malmo on Defining Data Habit of Mind.
Citation preview
DefiningData Habit of Mind
Saad Chahine Mount Saint Vincent University
Push to Use DataExternal pressures for educators to use data
from large- scale assessments
Very little is known about the process by which educators interpret and make judgments based on reports from large-scale testing
Potential compromise in validity when educators interpret and make judgments about students from large-scale assessment reports
π
Data Habit of Mind π
Methods Designed instruments to measure Statistical Literacy
and Score Report Interpretation (2 mock score reports)
Conducted pilot test with 12 pre-service students and refined instruments and interview procedure
Recruited educators through advertising
Conducted cognitive interviews to examine 20 educators’ data habit of mind
Used protocol analysis procedure to examine cognitive interview data (i.e., educators’ cognitive processing when interacting with score reports)
π
Sample Snap Shot All elementary (Gr. 1-8) teaching experience
with provincial testing
13 Female, 7 male
1-21+ years of experience
17 had undergrad focus in Humanities or Social Science and 3 had undergraduate focus in Sciences
Majority had B.Ed. (13); others had higher degrees (1 Ph.D., 6 M.Ed.)
π
Statistical Literacy
π
π
Results: Statistical Literacy
Min Max Mean SD
Statistical Literacy
Describing 3.00 4.00 3.77 0.24
Organizing & Reducing 2.00 4.00 2.89 0.65
Representing 1.67 4.00 3.33 0.65
Analyzing and Interpreting 2.00 4.00 3.38 0.53
π
Score Report Interpretation
π
π
Results: Score Report Interpretation
Cooper’s Fitness Test
Min Max Mean SD
Describing 4.00 4.00 4.00 0.00
Summarizing 3.00 4.00 3.75 0.44
Questioning 1.00 4.00 2.10 0.91
Proposed Application 1.00 4.00 2.45 1.10
Jabberwocky Test
Describing 4.00 4.00 4.00 0.00
Summarizing 3.00 4.00 3.75 0.44
Questioning 1.00 4.00 1.95 0.98
Proposed Application 1.00 4.00 2.00 0.97
π
Data Habit of Mind
π
π
Categories Based on Data Habit of Mind Skills π
Example: Questioning
Questioning Patterns for Cooper’s Fitness Test Score Report
π
Example: Questioning
Questioning Patterns for Jabberwocky Test Score Report
π
Influences of Data Habit of Mind
π
Results: Educational History & Comfort Levels
Expected that educational history and comfort levels (information, math, and stats) would be meaningfully related to Data Habit of Mind
However, found no meaningful relationships…
Examination of interview transcripts gives us glimpse of insight into what other factors may be meaningfully related to Data Habit of Mind…
π
Educator Beliefs About the Use of Assessment Data
Participant E18: “Assessment is probably, for me, has always been in my annual learning goal...my life-long learning goal in my formal years of education. It has been a constant pursuit of mine to understand, advance and implement assessment. We are a very data-driven school and I didn’t have to come to a data-driven school to be hungry for data. It already was in anything that I did...You keep yourself alive by having data. It just goes hand in hand. We are a very…we are a school that is constantly pushing for data.”
Educator with Higher Statistical Literacy and Higher Score Report Interpretation
π
Educator Beliefs About the Use of Assessment DataParticipant E13: “I’ve done a few workshops on DRA and reading, in math, like effective instruction. I’ve done a lot of reading myself there and we have had a lot of in-service training. That being said, it is an area that is still a little bit hazy because we are going along in our lessons and I don’t necessarily assess them. We do the evaluation side to see what we can get in marks. That’s why I’m interested in the whole literacy thing because we assess the kids in the reading and see where we need to go from there.”
Educator with Higher Statistical Literacy and Lower Score Report Interpretation.
π
Educator Beliefs About the Use of Assessment DataParticipant E2: “Ha ha, very high … Well, I think it’s kind of the package, frankly. I have my reading specialist, I have my religion specialist, which is essential for the Catholic school board. I do have a Master’s degree. I also think the fact that I’ve taught all three divisions helps a lot. Um, I am, in fact, I was an adjunct professor, and you have to mentor the young teachers... I’ve gone anywhere from basal readers to whole language to comprehensive literacy. I’ve kind of seen everything. That goes for math education, as well.
Educator with Lower Statistical Literacy and Higher Score Report Interpretation.
π
Educator Beliefs About the Use of Assessment Data
Participant E11: “Assessment to me ... it’s not always like paper and a test, a hard assessment that way. Sometimes I think you can get enough of an assessment by watching, observing and talking to the students. Some might not be able to articulate on paper and pencil what they do by talking.”
Educator with Lower Statistical Literacy and Lower Score Report Interpretation.
π
Future Research More research needed to examine categories
Double coding or multiple raters
Make links with medical diagnostic methods
Include more interactive reporting methods
Make links with cognitive human-computer Interface work (e.g., Ware’s work)
Conduct large-scale study of reporting systems and improvements
π