Upload
katherine-sauer
View
218
Download
0
Embed Size (px)
Citation preview
7/31/2019 2012 ABE Test Assessment
1/29
The Test Assessment Questionnaire
Katherine M. Sauer William MertensMetropolitan State College of Denver University of Colorado at Boulder
[email protected] [email protected]
March 29, 2012 MBAA ABE Annual Conference Chicago, IL
7/31/2019 2012 ABE Test Assessment
2/29
7/31/2019 2012 ABE Test Assessment
3/29
Students are Overconfident
Walstad (2001) - suggests concepts from behavioral
economics (e.g. overconfidence) could be used to explainstudent behavior
Falchikov and Boud (1989) - students have grade
expectations that are higher than the typical distributionfor the course
Grimes (2002) - has principles students predict examscores and finds overconfidence in their understanding ofmaterial
Nowell and Alston (2007) - instructor grading practicescan influence the degree of overconfidence
7/31/2019 2012 ABE Test Assessment
4/29
What Happens When Students are Overconfident?
Students may start toblame the instructorfor thedisconnect between (perceived) effort and desired
grade.
Students may label the instructor as unfair or thecourse simply too hard.
7/31/2019 2012 ABE Test Assessment
5/29
Grimes, Millea, and Woodruff (2004) - degree to whichstudents accept personal responsibility for performanceaffects their evaluation of teaching effectiveness and
course satisfaction
Carrell and West (2010) - in course evaluations, studentsreward professors who increase achievement in the
contemporaneous course, not those who facilitated deeplearning for subsequent courses
Millea and Grimes (2002) - instructors need not water-
down courses in order to receive favorable courseevaluations they can positively influence evaluations byaddressing negative student attitudes about forthcomingcoursework
7/31/2019 2012 ABE Test Assessment
6/29
How can I help my students become
more self-aware in their learning?
Get them to focus on the type of mistake they are
making instead of the number of mistakes they aremaking.
Get them thinking about the link between their study
activities and their exam performance.
7/31/2019 2012 ABE Test Assessment
7/29
The Test Assessment Questionnaire
This instrument guides students through an analysis oftheir midterm exam mistakes.
questions about exam preparation activities
studying activities for the course in general
formulate a study strategy for the rest of the course
7/31/2019 2012 ABE Test Assessment
8/29
The Version Im Currently Using
7/31/2019 2012 ABE Test Assessment
9/29
7/31/2019 2012 ABE Test Assessment
10/29
7/31/2019 2012 ABE Test Assessment
11/29
7/31/2019 2012 ABE Test Assessment
12/29
Instructors can tailorthe first set of questions to theircourse.
7/31/2019 2012 ABE Test Assessment
13/29
7/31/2019 2012 ABE Test Assessment
14/29
Three Benefits
1) it guides students toward more self-awareness in theirstudies
2) it can be used as a part ofassessment and assurance oflearning efforts
3) may improve students final exam scores over themidterm
7/31/2019 2012 ABE Test Assessment
15/29
1) self-awareness in studies
On the course evaluations: Please comment on the
experience of completing the Test Assessment and/ormeeting with me to discuss it.
- It made me improve my studying skills.
-Good, it gave me one-on-one time with you to figure outhow I could improve.
-Helpful in learning how I make mistakes and how to fixthem better.
7/31/2019 2012 ABE Test Assessment
16/29
- It helped me to personally analyze my test taking andstudy habits. Very helpful! I have never had a teacher whocared enough to take the time to help me improve in thisaspect.
-Your tips helped me in other classes and got me runningagain and I dont feel as stressed. My mom thanks you.
There was no negative feedback on this question.
7/31/2019 2012 ABE Test Assessment
17/29
7/31/2019 2012 ABE Test Assessment
18/29
7/31/2019 2012 ABE Test Assessment
19/29
3) improve students final exam scores
Experiment:
Fall 2011 University of Colorado
All Students in Principles of Microeconomics10 course sections, 1713 students
62 recitation sections
7/31/2019 2012 ABE Test Assessment
20/29
Design:
Course instructors do not modify their class time orteaching activities in any way.
Each TA that teaches two or more recitation sections has
a control and experiment section assigned to them.
The experiment will take place during the recitationclass in which the midterm is handed back.
7/31/2019 2012 ABE Test Assessment
21/29
Control Sections: TAs spend the class time going overthe exam as they normally would and will offer somegeneral study strategies.
Experiment Sections: Students will be given the correct
answers and will spend the class time going over theexam by themselves while completing the TAQ.
After the TAQs are completed, the TA discusses study
strategies that complement each of the common mistakecategories.
7/31/2019 2012 ABE Test Assessment
22/29
All students complete Duttweilers(1984) InternalControl Index.
- many such instruments
- argue this one is most reliable and validone for adults
- normed on college students
- 28 questions, Likert-type scale
7/31/2019 2012 ABE Test Assessment
23/29
7/31/2019 2012 ABE Test Assessment
24/29
Half of the items are worded so that high internallyoriented respondents are expected to answer at the
"usually" end of the scale and the other half at the"rarely" end.The "rarely" response is scored as 5 points on items
1,2,8,11,14,17,19,22,23,24,26,27.
This produces a possible range of scores from 28 to 140with higher scores reflecting higher internal locus ofcontrol.
For college students: MEAN: 99.3- 120.8
7/31/2019 2012 ABE Test Assessment
25/29
Data:
Two course instructors opted out of the experiment.n = 1337
27 treatment recitations, n = 66727 control recitations, n = 670
But not all sections were full.
7/31/2019 2012 ABE Test Assessment
26/29
We received 522 first-round ICI surveys from 28different recitations.
We received 488 second-round ICI surveys also from 28different recitations (but not exactly the same recitationsas those for the first-round surveys).
We received 234 TAQs from 13 different recitations.
7/31/2019 2012 ABE Test Assessment
27/29
Treatment groupsWe received data that included both surveys and the
TAQ for six recitations involving a maximum of88students having all 3 sheets (if they match).
We received data that included one survey and a TAQ
from seven recitations involving a maximum of104students having both sheets (if they match).Control groups
We received data that includedboth surveys for tenrecitations involving a maximum of175 students having
both sheets (if they match).
7/31/2019 2012 ABE Test Assessment
28/29
The students TAQs and ICIs are being scanned and thedata entered into a spreadsheet.
Control data is being obtained from InstitutionalResearch.
7/31/2019 2012 ABE Test Assessment
29/29
Comments? Questions?
Katherine M. SauerMetropolitan State College of Denver
March 29, 2012 MBAA ABE Annual Conference Chicago, IL