Upload
horatio-norman
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Using Demographic Data to Predict Students’ Achievement in
DSPM Courses
Daryl Stephens, ETSUDaryl Stephens, ETSU
[email protected]@etsu.edu
TNADE TNADE
October 27, 2005October 27, 2005
Students take developmental studies Students take developmental studies program (DSP) courses for many reasons:program (DSP) courses for many reasons:
Forgot material learned in high school by Forgot material learned in high school by the time they took entrance teststhe time they took entrance tests
Long gap between high school and Long gap between high school and matriculationmatriculation
Didn’t take high school seriouslyDidn’t take high school seriously
Didn’t consider going to college until laterDidn’t consider going to college until later
First generation college studentFirst generation college student
(Salter & Noblett, 1994)(Salter & Noblett, 1994)
Most students (~90%) at Most students (~90%) at ETSU take MATH 1530, ETSU take MATH 1530, Probability and Statistics, as Probability and Statistics, as their mathematics course to their mathematics course to satisfy core curriculum satisfy core curriculum requirements.requirements.
In previous years, this course In previous years, this course had a high failure rate.had a high failure rate.
The ProblemThe Problem
Placement procedures for TBR schools Placement procedures for TBR schools have changed.have changed.
Pre-2000:Pre-2000:ACT < 19 or age > 22 → take AAPPACT < 19 or age > 22 → take AAPP
2000-2002: COMPASS replaces AAPP2000-2002: COMPASS replaces AAPP
2002: ACT score alone determines 2002: ACT score alone determines placement for students with score < 3 placement for students with score < 3 years old; COMPASS alone for othersyears old; COMPASS alone for others
The ProblemThe Problem
State funding is static. State funding is static.
Placement decisions based on one test.Placement decisions based on one test.
Enrollment caps may happen in the future.Enrollment caps may happen in the future.
Is there a way to augment the placement Is there a way to augment the placement process by predicting a student’s chance process by predicting a student’s chance of success or failure in developmental or of success or failure in developmental or core math classes?core math classes?
PurposePurpose
Develop models to predict success of Develop models to predict success of students instudents in– DSPM 0800 (Elementary Algebra)DSPM 0800 (Elementary Algebra)– DSPM 0850 (Intermediate Algebra)DSPM 0850 (Intermediate Algebra)– MATH 1530 (Probability and Statistics)MATH 1530 (Probability and Statistics)
Use multiple regression to develop the Use multiple regression to develop the models using readily obtainable models using readily obtainable informationinformation
ImportanceImportance
MATH 1530 traditionally had high failure MATH 1530 traditionally had high failure raterate
Developmental math students are at a Developmental math students are at a greater risk of failure and dropping outgreater risk of failure and dropping out
Specific developmental studies program Specific developmental studies program advisors done away with in budget crunch advisors done away with in budget crunch of 2003of 2003
ImportanceImportance
Previous similar studies on developmental Previous similar studies on developmental students used additional instruments students used additional instruments which cost money.which cost money.
Very few universities require probability Very few universities require probability and statistics for math credit for and statistics for math credit for graduation. Most similar studies on core graduation. Most similar studies on core math deal with college algebra, math deal with college algebra, precalculus, or math survey courses.precalculus, or math survey courses.
AssumptionsAssumptions
Prediction is possiblePrediction is possible
Self-reported data are correctSelf-reported data are correct
Information in SIS is correctInformation in SIS is correct
LimitationsLimitations
Different demographics and Different demographics and course requirements from course requirements from other institutions, so work only other institutions, so work only applies to ETSUapplies to ETSU
Data only collected for fall; Data only collected for fall; spring and summer grades are spring and summer grades are probably differentprobably different
DefinitionsDefinitions
DSPM 0800: Elementary algebra DSPM 0800: Elementary algebra – Arithmetic review, algebraic representations, Arithmetic review, algebraic representations,
linear equations in one and two variableslinear equations in one and two variables
DSPM 0850: Intermediate algebraDSPM 0850: Intermediate algebra– Exponents, polynomials, factoring, Exponents, polynomials, factoring,
MATH 1530: Probability and statisticsMATH 1530: Probability and statistics– ““Stat mansion” and “stat cave”Stat mansion” and “stat cave”
DefinitionsDefinitions
COMPASS (Computerized Adaptive COMPASS (Computerized Adaptive Placement Assessment and Support Placement Assessment and Support SystemSystem– Prealgebra and algebra sectionsPrealgebra and algebra sections– Reading and writing sectionsReading and writing sections– Replaced AAPPReplaced AAPP
Previous ResearchPrevious Research
Developmental Studies HistoryDevelopmental Studies History
Special programs at Harvard in 17Special programs at Harvard in 17thth centurycentury
Preparatory department at University of Preparatory department at University of Wisconsin, 1849Wisconsin, 1849
Morrill Acts, late 19Morrill Acts, late 19thth century, establish century, establish land grant colleges and extend access to land grant colleges and extend access to higher education to more peoplehigher education to more people
Developmental Studies HistoryDevelopmental Studies History
Preparatory departments widespread in Preparatory departments widespread in early 20early 20thth century (350 in 1915) century (350 in 1915)
GI Bill of Rights brings in veterans with GI Bill of Rights brings in veterans with needs for auxiliary servicesneeds for auxiliary services
1960s-70s: Increase of women, students 1960s-70s: Increase of women, students of color, first-generation students, students of color, first-generation students, students with learning disabilities; open-access with learning disabilities; open-access community colleges establishedcommunity colleges established
Tennessee DSP HistoryTennessee DSP History
TBR establishes formal developmental TBR establishes formal developmental studies program in 1984.studies program in 1984.
Defining Our FutureDefining Our Future (2001): “operate more (2001): “operate more efficiently and with more limited resources”efficiently and with more limited resources”
Move 0700-level courses to community Move 0700-level courses to community collegescolleges
Related ResearchRelated Research
Developmental MathDevelopmental Math
Core Math (almost nothing on P&S)Core Math (almost nothing on P&S)
What factors are related to success in the What factors are related to success in the courses? Three broad categories:courses? Three broad categories:– AcademicAcademic– DemographicDemographic– AffectiveAffective
What variables predict course What variables predict course success in dev. math?success in dev. math?
High school GPA, at least for traditional High school GPA, at least for traditional studentsstudents
Scores on the ACT or SAT — sometimesScores on the ACT or SAT — sometimes
Placement tests (e.g. COMPASS, ASSET, Placement tests (e.g. COMPASS, ASSET, Accuplacer, CPT, PTT, CLAST) — Accuplacer, CPT, PTT, CLAST) — sometimessometimes
GED math scores — as adjunct to other GED math scores — as adjunct to other placement scoresplacement scores
Course success in dev. mathCourse success in dev. math
Number of high school mathematics Number of high school mathematics courses taken — sometimescourses taken — sometimes
High school math GPA – some coursesHigh school math GPA – some courses
College GPACollege GPA
AttendanceAttendance
Study habitsStudy habits
Age or length of time since last math classAge or length of time since last math class
Course success in dev. math (cont’d)Course success in dev. math (cont’d)
Gender (some studies)Gender (some studies)
Race Race
Math anxiety level Math anxiety level notnot related to grade! related to grade!
AttitudeAttitude
Perception of success, engagement in Perception of success, engagement in classclass
Paying attention and interacting with Paying attention and interacting with instructorinstructor
Success in core classesSuccess in core classes
Very little research on success in classes Very little research on success in classes like MATH 1530, so other courses like MATH 1530, so other courses examinedexamined
ACT/SAT math scores (usually)ACT/SAT math scores (usually)
COMPASS, TASP, local placement testsCOMPASS, TASP, local placement tests
High school GPA for college algebra High school GPA for college algebra (multiple studies) and calculus (but not (multiple studies) and calculus (but not precalculus)precalculus)
Success in core classes (contd.)Success in core classes (contd.)
HS math GPA – mixed resultsHS math GPA – mixed results
HS percentile rankHS percentile rank
Number and difficulty of HS math classes Number and difficulty of HS math classes takentaken
Whether a math class taken senior yearWhether a math class taken senior year
Students who didn’t take intermediate Students who didn’t take intermediate algebra scored sig. higher in college alg.algebra scored sig. higher in college alg.
Success in core classes (contd.)Success in core classes (contd.)
Age in some casesAge in some cases
Time since last math courseTime since last math course
Gender? Yes in 3, no in 5 studiesGender? Yes in 3, no in 5 studies
Full-time vs. part-time (1 study)Full-time vs. part-time (1 study)
Class meeting time (1 study)Class meeting time (1 study)
No difference in resident vs. commuter, No difference in resident vs. commuter, campus activitycampus activity
Success in core classes (contd.)Success in core classes (contd.)
AttitudeAttitude
Learning stylesLearning styles
Self-conceptSelf-concept
Research QuestionsResearch Questions
Relationship QuestionsRelationship QuestionsIs there a relation between course grade and Is there a relation between course grade and
……
ACT (DSPP) math scores?ACT (DSPP) math scores?
ACT (DSPP) reading scores?ACT (DSPP) reading scores?
COMPASS intermediate algebra scores?COMPASS intermediate algebra scores?
COMPASS reading scores?COMPASS reading scores?
Number of college preparatory math classes Number of college preparatory math classes taken in high school?taken in high school?
High school GPA?High school GPA?
Regression QuestionsRegression QuestionsCan a regression equation be found to predict final Can a regression equation be found to predict final
course grade based on these items?course grade based on these items?
From SIS:From SIS:ACT compositeACT compositeACT mathACT mathACT readingACT readingACT EnglishACT EnglishHigh school GPAHigh school GPAAge on first day of Age on first day of classclass
From Survey:From Survey:Number of high Number of high school math classes school math classes takentakenNumber of years Number of years since last HS math since last HS math classclass
Regression QuestionsRegression QuestionsCan a regression equation be found to predict final Can a regression equation be found to predict final
course grade based on these items?course grade based on these items?
From SIS:From SIS:COMPASS writingCOMPASS writingCOMPASS readingCOMPASS readingCOMPASS COMPASS prealgebraprealgebraCOMPASS COMPASS intermediate algebraintermediate algebraAge on first day of Age on first day of classclass
From Survey:From Survey:Number of high Number of high school math classes school math classes takentakenNumber of years Number of years since last HS math since last HS math classclass
MethodMethod
Collect information about courses taken in Collect information about courses taken in high school and year of last high school high school and year of last high school math class from students via surveymath class from students via survey
Obtain other information from SISObtain other information from SIS
Use Pearson correlation for relationship Use Pearson correlation for relationship questionsquestions
Use stepwise multiple regression for grade Use stepwise multiple regression for grade prediction questionsprediction questions
Initial PlacementInitial PlacementACT (DSPP) Math sectionACT (DSPP) Math section– << 17 (SAT 17 (SAT << 440): DSPM 0800 440): DSPM 0800– 18 (SAT 450): DSPM 085018 (SAT 450): DSPM 0850– >>19 (SAT > 450): college level math19 (SAT > 450): college level math
COMPASSCOMPASS– Prealgebra score Prealgebra score << 29: DSPM 0700 29: DSPM 0700– Prealgebra 30-99 and algebra 20-27: DSPM Prealgebra 30-99 and algebra 20-27: DSPM
08000800– Algebra 28-49: DSPM 0850Algebra 28-49: DSPM 0850– Algebra 50-99: college level mathAlgebra 50-99: college level math
Data considerationsData considerations
Not counted: grades of W, WF, INot counted: grades of W, WF, I
FN grade not counted to be consistent FN grade not counted to be consistent with some other ETSU studieswith some other ETSU studies
Online sections, RODP sections not Online sections, RODP sections not includedincluded
Surveys returnedSurveys returned
DSPM 0800: 149 / 304 (49%) DSPM 0800: 149 / 304 (49%) – No night, off-campus, or ITV sectionsNo night, off-campus, or ITV sections
DSPM 0850: 214 / 455 (47%)DSPM 0850: 214 / 455 (47%)– Included Kingsport, night, ITVIncluded Kingsport, night, ITV
MATH 1530: 631 / 1074 (59%)MATH 1530: 631 / 1074 (59%)
ResultsResults
Grade Grade DistributionDistribution
Table 3
* Grades of C-, D+, and D are not allowed in developmental studies courses.
**Not used in answering the research questions.
GradeDSPM 0800
DSPM 0850
MATH 1530
A 35 75 70
A- 17 18 48
B+ 14 17 35
B 21 21 98
B- 8 11 36
C+ 9 13 34
C 13 25 89
C- * * 35
D+ * * 42
D * * 28
F 19 19 49
FN** 10 8 58
I** 0 1 1
W** 3 6 5
WF** 0 0 3
Total 149 214 631
Mean 2.785 2.991 2.419
SD 1.2993 1.2075 1.1526
ACT Math vs. GradeACT Math vs. Grade
ACT Math
DSPM 0800 DSPM 0850 MATH 1530
r 0.025 .322** .481**
GPTS p 0.802 0 0
N 102 169 499
ACT Reading vs. GradeACT Reading vs. Grade
ACT Reading
DSPM 0800 DSPM
0850MATH
1530
r 0.047 0.164* 0.164*
GPTS p 0.642 0.033 0
N 102 169 497
COMPASS Int. Alg. vs. GradeCOMPASS Int. Alg. vs. Grade
COMPASS Intermediate Algebra Score
DSPM 0800
DSPM 0850
MATH 1530
r 0.107 0.326* 0.133
GPTS p 0.448 0.029 0.348
N 52 45 52
COMPASS Reading vs. GradeCOMPASS Reading vs. Grade
COMPASS Reading Score
DSPM 0800
DSPM 0850
MATH 1530
r 0.222 0.206 0.118
GPTS p 0.194 0.323 0.513
N 36 25 33
High High School School Math Math
CoursesCourses(Most (Most
Common)Common)
CourseCourse 08000800 08500850 15301530
PrealgebraPrealgebra 4747 3737 6767
Foundations 2Foundations 2 1616 2626 2828
Algebra 1Algebra 1 123123 181181 518518
Algebra 2Algebra 2 111111 182182 556556
GeometryGeometry 102102 176176 544544
PrecalculusPrecalculus 66 2727 211211
Trig/Adv. Alg.Trig/Adv. Alg. 44 2727 133133
Prob. & Stat.Prob. & Stat. 44 66 7272
CalculusCalculus 22 44 8888
NN 149149 214214 633633
Course Grade vs. Number of Course Grade vs. Number of College Prep Classes TakenCollege Prep Classes Taken
HSMATH (All students)
DSPM 0800
DSPM 0850
MATH 1530
r -0.068 0.191** 0.265**
GPTS p 0.434 0.007 .000
N 135 198 564
Mean # taken 2.52 3.11 3.73
SD taken 1.034 0.955 1.115
Course Grade vs. Number of Course Grade vs. Number of College Prep Classes TakenCollege Prep Classes Taken
HSMATH (Students with ACT Scores)
DSPM 0800
DSPM 0850
MATH 1530
r -0.237* 0.193* 0.298**
GPTS p 0.017 0.012 .000
N 102 168 499
Mean # taken 2.84 3.28 3.85
SD taken 1.2333 1.2 1.034
Course Grade vs. Number of Course Grade vs. Number of College Prep Classes TakenCollege Prep Classes Taken
HSMATH (Students with COMPASS scores)
DSPM 0800
DSPM 0850
MATH 1530
r -0.061 0.131 0.171
GPTS p 0.671 0.392 0.226
N 51 45 52
Mean # taken 1.88 2.53 2.85
SD taken 1.103 1.254 1.036
HSGPA vs. GradeHSGPA vs. Grade
HS Overall GPA
DSPM 0800 DSPM 0850 MATH 1530
r .389** .333** .445**
GPTS p .000 .000 .000
N 105 176 500
Regression Using ACTRegression Using ACT
DSPM 0800 DSPM 0800 (95 students)(95 students)
Model 1:Model 1:ŷ = –.589(HSMATH) + 4.599 (p = .001)
Model 2:ŷ = –.765(HSMATH) + 1.009(HSOGPA)
+ 2.298 (p < .001)
Regression Using ACTRegression Using ACT
DSPM 0850DSPM 0850 (160 students) (160 students)
Model 1:Model 1:ŷ = .364(ACTM) – 3.238 (p < .001)
Model 2:ŷ = .301(ACTM) + .662(HSOGPA) – 4.111
(p < .001)
Regression Using ACTRegression Using ACT
MATH 1530MATH 1530 (475 students) (475 students)Four models Four models ((p p < .001 in each case; < .001 in each case; rr22 ranging ranging
from .233 to .353):from .233 to .353):ŷ = .134(ACTM) – .521ŷ = .092(ACTM) + .771(HSOGPA) – 2.181ŷ = .103(ACTM) + .910(HSOGPA)
+ .108(AGE) – 4.953ŷ = .089(ACTM) + .855(HSOGPA)
+ .105(AGE) + .025(ACTE) – 4.9
Regression Using COMPASSRegression Using COMPASS
DSPM 0800DSPM 0800 (33 students) – no equation (33 students) – no equation possiblepossible
DSPM 0850DSPM 0850 (23 students with all sections) (23 students with all sections)
ŷ = .020 (COMPASS writing) + 1.374
(r2 = .221, p = .024)
DSPM 0850 (44 students with math scores)
ŷ = .023 (COMPASS arithmetic) + 1.730
(r2 = .190, p = .033)
Regression Using COMPASSRegression Using COMPASS
MATH 1530MATH 1530
22 students with all COMPASS sections: 22 students with all COMPASS sections: no equation possibleno equation possible
51 students with just math sections:51 students with just math sections:
ŷ = .027(COMPASS arithmetic) + .671
(r2 = .276, p < .001)
Observations, Conclusions, Observations, Conclusions, and Recommendationsand Recommendations
Observations: 0800Observations: 0800
Elementary algebra: only high school GPA Elementary algebra: only high school GPA showed a significant correlation with showed a significant correlation with course grade; math preparation in high course grade; math preparation in high school showed a little bit of predictive school showed a little bit of predictive value.value.
Finding agrees with Long (2003) – ACT Finding agrees with Long (2003) – ACT math score not significantly correlated with math score not significantly correlated with E.A. course gradeE.A. course grade
Observations: 0800 (cont’d)Observations: 0800 (cont’d)
Odd finding: Why a negative correlation Odd finding: Why a negative correlation between HS math prep and grade in between HS math prep and grade in 0800?0800?
ObservationsObservations
Intermediate algebra: course grade Intermediate algebra: course grade correlated with ACT math and reading, correlated with ACT math and reading, COMPASS intermediate algebra and COMPASS intermediate algebra and reading, high school math preparation, reading, high school math preparation, high school GPA. high school GPA.
ACT math correlation agrees with Lott ACT math correlation agrees with Lott (1990)(1990)
HS math preparation agrees with 4 studiesHS math preparation agrees with 4 studies
ObservationsObservations
Probability and statistics grade was Probability and statistics grade was significantly correlated with ACT math and significantly correlated with ACT math and reading and HSGPA, but regression used reading and HSGPA, but regression used ACT math and English, age, and HSGPA. ACT math and English, age, and HSGPA. Only COMPASS arithmetic score was in Only COMPASS arithmetic score was in regression for students without ACT regression for students without ACT scores.scores.Findings in line with 7 other studies of Findings in line with 7 other studies of other core college math courses.other core college math courses.
RecommendationsRecommendations
For students with borderline placement For students with borderline placement scores, use the regression equation to scores, use the regression equation to fine-tune the placement.fine-tune the placement.
If student’s regression equation predicts a If student’s regression equation predicts a failing grade, give extra attention to that failing grade, give extra attention to that student (watch attendance, recommend student (watch attendance, recommend tutoring, etc.), or consider placement in a tutoring, etc.), or consider placement in a lower level class if possible.lower level class if possible.
Future ResearchFuture Research
Repeat study with data from spring and Repeat study with data from spring and summer. Historically, grades in summer. Historically, grades in developmental math classes are developmental math classes are significantly higher in summer and lower in significantly higher in summer and lower in spring. (Unknown if true for 1530)spring. (Unknown if true for 1530)
Repeat treating repeaters separately, or Repeat treating repeaters separately, or include number of previous attempts as a include number of previous attempts as a variablevariable
Future ResearchFuture Research
Is grade in 1530 affected by other college-level Is grade in 1530 affected by other college-level or developmental math experience?or developmental math experience?
Characteristics of repeating students (mostly Characteristics of repeating students (mostly qualitative research)qualitative research)
What if core curriculum at ETSU changes What if core curriculum at ETSU changes allowing other math courses to count for allowing other math courses to count for graduation?graduation?
How would raising cutoff scores affect grades? How would raising cutoff scores affect grades? (TBR uses 19, ACT recommends 22)(TBR uses 19, ACT recommends 22)
Future ResearchFuture Research
Qualitative study (suggested Qualitative study (suggested by a student): What kind of by a student): What kind of guidance were students given guidance were students given in selecting high school math in selecting high school math classes?classes?
Why a negative correlation Why a negative correlation between HS math prep and between HS math prep and 0800 grade?0800 grade?
For more informationFor more information
Entire dissertation is available online:Entire dissertation is available online:http://faculty.etsu.edu/stephen/http://faculty.etsu.edu/stephen/
Will be online from UTK in Jan. or Feb., Will be online from UTK in Jan. or Feb., but will stay on my faculty web page but will stay on my faculty web page unless there’s a space crunch.unless there’s a space crunch.
This slide show is also on my faculty web This slide show is also on my faculty web page. Look for the “conference page. Look for the “conference presentations” link.presentations” link.