Tracking Pathways to Success

Identifying Learning Success Factors Across Course Delivery Formats

Peter Usinger, State Assessment Meeting 2013

Student Success in College Environment

High W/F

As high as 50% attrition

of FTIC cohort

45% of Withdrawal Reasons are

Course Related

Student Attributes

Financially Limited

Non-traditional

Academically Underprepared

Study Context

Reviewing High F/W

Engagement in College Prep

F2F vs. Online Instruction

Nursing OrientationSLS Course

Support

Research Questions

Q1

• Delivery format based differences between self-directed behaviors and learning strategies.

Q2

• Similarities and Differences across subject domains/disciplines and their sensitivity to course delivery formats.

Q3

• Relationships among MSLQ constructs, and with student characteristics, course characteristics, and delivery formats.

Q4

• “Predictive” model to explain the relationships for developing support mechanisms and change curricular design for greater success/completion

(Online) Course Completion

Student Characteristics

Demographic variables

Previous e-learning experience

Self-Directed Learning Habits

Metacognitive skills

Motivation

Self-discipline

Autonomy

Self-regulated behaviors

Sample drawn from…

Mixed withdrawal/failure rates136 Courses• Humanities, English/Letters, Mathematics and Statistics,

Sciences, Social Sciences, Nursing; Hybrids and Workforce Dev. excluded from analysis due to low frequencies.

Delivery• Online (216 Sections)• Face to Face (931 Sections)

>20,000 StudentsPopulation• Invited via email to complete MSLQ online (15 min.)

1,179 Course sections (total)

Sample

by

Academic A

rea

2,200 Participants:11% Response Rate

Discipline F2F Online

Humanities 43 29

Letters 314 84

Math 957 82

Nursing 136 33

Sciences 207 56

Social Sciences 85 64

MSLQ Scales and Reliability

Motivated Strategies for Learning Questionnaire (MSLQ)

Original MSLQ Applied in Study

Items α Items α

A. Motivational Constructs1. Intrinsic Goal Orientation 4 0.74 4 0.752. Extrinsic Goal Orientation 4 0.62 2 0.633. Task Value 6 0.90 2 0.904. Control of Learning Beliefs 4 0.68 3 0.765. Self-Efficacy for Learning & Performance 8 0.93 8 0.966. Test Anxiety 5 0.80 4 0.75 B. Learning Strategies Constructs 1. Rehearsal 4 0.69 2 0.692. Elaboration 6 0.75 2 0.583. Organization 4 0.64 2 0.544. Critical Thinking 5 0.80 5 0.805. Metacognitive Self-Regulation 12 0.79 10 0.846. Time/Study Environmental Management 8 0.76 7 0.827. Effort Regulation 4 0.69 4 0.718. Peer Learning 3 0.76 3 0.779. Help Seeking 4 0.52 3 0.72Total Items in Questionnaire 81 61

Motivational Correlations

Pearson Correlation Coefficients (N=2,200)

 …between Grades Passing Online+

…and Motivational Constructs      

1. Intrinsic Goal Orientation 0.19*** 0.16*** 0.04

2. Extrinsic Goal Orientation 0.03 0.08*** -0.04

3. Task Value 0.24*** 0.19*** 0.04*

4. Control of Learning Beliefs 0.30*** 0.25*** 0.10***

5. Self-Efficacy for Learning & Performance 0.46*** 0.37*** 0.07**

6. Test Anxiety -0.23*** -0.15*** -0.01

Note 1: *p < .05. **p < .01. ***p < .001. Note 2: +Method: F2F=0, Online=1.

Learning Strategies

Pearson Correlation Coefficients (N=2,200)

 …between Grades Passing Online

…and Learning Strategies Constructs      

1. Rehearsal 0.00 0.02 -0.06**

2. Elaboration 0.09*** 0.07** 0.02

3. Organization 0.04 0.03 -0.06*

4. Critical Thinking 0.04 0.01 0.06*

5. Metacognitive Self-Regulation 0.04 0.04* 0.00

6. Time/Study Environmental Management 0.19*** 0.13*** 0.07**

7. Effort Regulation 0.27*** 0.21*** 0.06*

8. Peer Learning -0.04 -0.02 -0.19***

9. Help Seeking -0.04 0.00 -0.32***

(where *p < .05. **p < .01. ***p < .001.)

Due to the very minor statistical differences in the results for grades vs. course success, and the fact that a significantly higher proportion of C and D students across most discipline areas are subsequently failing, we’ve created a Student Performance aggregate variable that combines student’s Course Performance into three levels:

• Level 0: Failure or Withdrawal from Course• Level 1: Course Grade Equals C or D• Level 2: Course Grade Equals A or B

This Performance variable is used in the student success analysis following.

Data Aggregation Rules

Comparison by Delivery Type

MSLQ Correlation w/ Course Performance

Online Face to Face

Mean Correlation Mean Correlation

Intrinsic Goal Or. 5.11 0.18*** 4.99 0.19***

Extrinsic Goal Or. 5.82 0.01 5.94 0.06*

Task Value 4.99 0.23*** 4.79 0.23***

Control Beliefs 5.83 0.31*** 5.49 0.28***

Self-Efficacy 5.54 0.43*** 5.20 0.44***

Test Anxiety 4.42 -0.21*** 4.47 -0.21***

Rehearsal 4.79 -0.04 5.04 0.03

Elaboration 5.28 0.12* 5.20 0.06*

Organization 5.00 0.04 5.21 0.03

Critical Thinking 4.33 0.07 4.13 0.01

Metacog. S-Reg. 4.86 0.03 4.84 0.04

Time/Study Mgmt. 5.48 0.22*** 5.26 0.15***

Effort Regulation 5.86 0.25*** 5.69 0.24***

Peer Learning 2.58 -0.07 3.41 0.00

Help Seeking 3.10 -0.02 4.52 0.01

N= 366 1,751

Comparison by Course Level

MSLQ Correlation w/ Course Performance

Developmental College Level

Mean R Coeff. Mean R Coeff.

Intrinsic Goal Or. 4.97 0.21*** 5.03 0.17***

Extrinsic Goal Or. 6.04 0.07 5.84 0.05*

Task Value 4.59 0.24*** 4.94 0.21***

Control Beliefs 5.48 0.31*** 5.57 0.28***

Self-Efficacy 5.15 0.43*** 5.32 0.44***

Test Anxiety 4.56 -0.23*** 4.40 -0.19***

Rehearsal 5.13 0.07 4.92 0.00

Elaboration 5.14 0.06 5.26 0.09***

Organization 5.33 0.05 5.10 0.04

Critical Thinking 4.13 -0.02 4.21 0.04

Metacog. S-Reg. 4.94 0.10* 4.80 0.03

Time/Study Mgmt. 5.42 0.23*** 5.24 0.17***

Effort Regulation 5.79 0.27*** 5.69 0.25***

Peer Learning 3.33 0.00 3.22 -0.04

Help Seeking 4.45 0.00 4.19 -0.01

N= 551 962

Important Note by Author:Please note that the following slide has been corrected since it contained a transfer error due to a truncated spreadsheet field. Affected is one of the interesting findings with regard to the role of test anxiety in the course performance of Nursing students. The original presentation slides showed a positive correlation between Test Anxiety and Course Performance for the whole Nursing group in this sample. However, this is only true for the 2nd year cohort of the RN program involved. Pre-admission, 1st year, and BSN students show patterns similar to other academic areas.

This is an extremely valuable research result since this correlation also goes along with a positive correlation between Test Anxiety and Rehearsal Strategies and Peer Learning. This indicates that our 2nd year Nursing students were not negatively overwhelmed with anxiety, but had learned to offset their anxiety levels successfully by rehearsing the subject matter at hand, and working with their peers to master the course content.

We will follow-up on these findings with an appropriate discipline-specific pathway model in the next phase of this longitudinal research project.

Correlations by Academic Area

Pearson Correlation with Course Performance

Humanities (N=75)

Letters (N=416)

Math (N=1043)

Nursing (N=174)

Sciences (N=276)

Soc. Sci. (N=156)

Motivational Constructs

1. Intrinsic Goal Orientation 0.14 0.07 0.22 0.22 -0.04 0.24

2. Extrinsic Goal Orientation -0.01 0.11 0.06 0.06 0.00 0.12

3. Task Value 0.25 0.10 0.21 0.27 -0.03 0.32

4. Control of Learning Beliefs 0.30 0.16 0.31 0.48 0.15 0.27

5. Self-Efficacy Learn. & Perf. 0.32 0.29 0.46 0.55 0.34 0.33

6. Test Anxiety -0.15 -0.04 -0.27 -0.20 -0.16 -0.09

Learning Strategies

1. Rehearsal -0.06 0.03 0.01 0.06 -0.08 0.12

2. Elaboration 0.23 0.11 0.04 0.13 -0.04 0.12

3. Organization -0.05 0.10 0.02 0.02 -0.04 0.11

4. Critical Thinking 0.09 0.00 -0.02 0.07 -0.13 0.00

5. Metacognitive Self-Reg. 0.02 0.03 0.07 0.11 -0.12 0.11

6. Time/Study Management 0.20 0.25 0.15 0.28 0.09 0.31

7. Effort Regulation 0.19 0.22 0.27 0.39 0.05 0.32

8. Peer Learning -0.12 -0.11 0.02 0.02 -0.17 0.02

9. Help Seeking -0.15 0.03 0.02 -0.03 -0.14 0.08

Success Comparison in Math

Example: Math Differences Failure/Withdrawal Passing Grade Difference

Motivational Constructs

Intrinsic Goal Orientation 58.3% 66.8% 8.5%

Extrinsic Goal Orientation 79.6% 84.0% 4.5%

Task Value 48.1% 59.7% 11.6%

Control Beliefs 63.4% 77.9% 14.5%

Self-Efficacy 50.0% 73.8% 23.7%

Test Anxiety 67.6% 55.6% -11.9%

Learning Strategies Constructs

Rehearsal 65.1% 67.1% 2.0%

Elaboration 65.9% 68.3% 2.4%

Organization 67.6% 70.1% 2.5%

Critical Thinking 50.0% 48.9% -1.1%

Meta-Cognitive Self-Regulation 61.5% 64.8% 3.3%

Time/Study Management 67.1% 73.4% 6.3%

Effort Regulation 70.4% 81.2% 10.7%

Peer Learning 36.0% 37.3% 1.4%

Help Seeking 53.7% 56.1% 2.5%

Multivariate Regression I

Multivariate Regression Analyses (Total Sample)

Predicting Course Performance

F-Value Pr > F R-Square

Fit of Model 42.1 <.0001 0.24

Parameter Estimates t-Value Pr > |t| St-B

Intrinsic Goal Orientation -1.97 0.05 -0.06

Extrinsic Goal Orientation -2.52 0.01 -0.06

Task Value 0.51 0.61 0.01

Control of Learning Beliefs -1.84 0.07 -0.06

Self-Efficacy for Learning 14.84 <.0001 0.52

Test Anxiety -1.46 0.14 -0.03

Rehearsal 1.27 0.20 0.03

Elaboration -0.47 0.64 -0.01

Organization -1.09 0.27 -0.03

Critical Thinking -0.75 0.45 -0.02

Metacognitive Self-Regulation -1.84 0.07 -0.07

Time/Study Management 2.26 0.02 0.07

Effort Regulation 2.95 0.00 0.09

Peer Learning -1.07 0.29 -0.03

Help Seeking -0.42 0.67 -0.01

Multivariate Regression II

Multivariate Regression Analyses w/ Performance

Online Courses Face-to-Face Classes

F-Value Pr > F R-Square F-Value Pr > F R-Square

Fit of Model 7.1 <.0001 0.22 33.2 <.0001 0.24

Parameter Estimates t-Value Pr > |t| St-B t-Value Pr > |t| St-B

Intrinsic Goal Orientation -0.35 0.73 -0.03 -1.47 0.14 -0.05

Extrinsic Goal Orientation -0.54 0.59 -0.03 -1.66 0.10 -0.04

Task Value -0.25 0.80 -0.02 0.27 0.78 0.01

Control of Learning Beliefs 0.57 0.57 0.04 -2.30 0.02 -0.08

Self-Efficacy for Learning 5.36 <.0001 0.44 13.60 <.0001 0.54

Test Anxiety -0.40 0.69 -0.02 -1.83 0.07 -0.05

Rehearsal -0.77 0.44 -0.05 2.14 0.03 0.07

Elaboration -0.26 0.80 -0.02 -0.76 0.45 -0.02

Organization -0.18 0.86 -0.01 -1.17 0.24 -0.04

Critical Thinking 1.54 0.13 0.11 -1.88 0.06 -0.06

Metacognitive Self-Regulation -1.67 0.10 -0.15 -1.12 0.26 -0.05

Time/Study Management 2.37 0.02 0.17 0.85 0.39 0.03Effort Regulation -0.05 0.96 0.00 2.92 0.00 0.10Peer Learning -0.99 0.33 -0.07 -0.48 0.63 -0.01

Help Seeking -0.04 0.97 0.00 -0.14 0.89 0.00

Multivariate Regression III

Multivariate Regression Analyses w/ Performance

Developmental College Level

F-Value Pr > F R-Square F-Value Pr > F R-Square

Fit of Model 14.0 <.0001 0.25 30.0 <.0001 0.24

Parameter Estimates t-Value Pr > |t| St-B t-Value Pr > |t| St-B

Intrinsic Goal Orientation -0.51 0.61 -0.03 -2.07 0.04 -0.07

Extrinsic Goal Orientation -2.20 0.03 -0.09 -1.21 0.23 -0.03

Task Value 0.34 0.73 0.02 -0.27 0.78 -0.01

Control of Learning Beliefs -0.42 0.67 -0.02 -2.08 0.04 -0.08

Self-Efficacy for Learning 7.31 <.0001 0.48 13.29 <.0001 0.55

Test Anxiety -1.90 0.06 -0.08 -0.72 0.47 -0.02

Rehearsal 1.64 0.10 0.08 0.74 0.46 0.02

Elaboration -1.97 0.05 -0.11 -0.30 0.76 -0.01

Organization -0.36 0.72 -0.02 -0.39 0.70 -0.01

Critical Thinking -2.03 0.04 -0.11 0.14 0.89 0.00

Metacognitive Self-Regulation 1.08 0.28 0.07 -2.55 0.01 -0.11

Time/Study Management 2.33 0.02 0.13 1.44 0.15 0.05

Effort Regulation 0.69 0.49 0.04 3.07 0.00 0.11

Peer Learning -0.33 0.75 -0.02 -1.04 0.30 -0.03

Help Seeking -0.77 0.44 -0.04 0.47 0.64 0.02

• MSLQ scales: excellent reliability & context sensitivity• Motivational factors clearly driving student success• Learning Strategies (cognitive skills) show multi-faceted instructional improvement (learning) opportunities 

• Peer Learning/Help Seeking: Underrepresented within cohorts, not an integrated/facilitated college activity

• Many underprepared students succeed largely via intrinsic motivation and confidence/self-efficacy, combined with hard work and study management

• The lack of exposure to meta-cognitive abilities and critical thinking skills nurtures course environments in support of compliant learners instead of self-directed learners!

Conclusions

Study Impact / Next Steps

• Online: Assess Quality Matters impact across delivery formats and feed information back into process

• 1st Year/EWS: Compare demographic differences associated with the various success patterns

• SLS: Evaluate impact with SLS courses when applying the MSLQ as formative assessment tool early in Student Success or Developmental Ed classes

• Overall: Conduct additional focus groups at college sites; seek expansion of study to inform interventions

• FCS: Seek study replication across service areas

Thank You!

pusinger@polk.edu