THE STUDENT EXPERIENCE OF STEM VS NON-STEM

DEGREE PROGRAMMES: A COMPARATIVE STUDY

Chris Pawson

STEM in the UK

• Significant shortfall in STEM skilled applicants in the U.K. (CBI, 2011)

• Attempts to increase uptake of STEM subjects:- National Curriculum changes in 2008- Science So What? So Everything (DIUS, 2009)

• School children’s reasons for not studying STEM:- No clear application to employment¹- Transmissive and unappealing pedagogy²

¹Archer et al. (2010) ² Lyons (2006); Osborne (2007)

STEM Promotion: The Role of HE?

• Reasons for leaving HE STEM study- Perception of better teaching in non-STEM- Loss of interest in science

(Olds & Miller, 2004; Seymour and Hewitt, 1997)

• Increasing interest in science and access to STEM careers in both schools and Universities

• Improving teaching and perceptions of teaching in STEM subjects

Student Satisfaction

• Stable differences between subjects (HEFCE, 2010)

• Historical / Philosophical studies most satisfied (HEFCE,2010)• Computer & Mathematical Science < Law (Surridge, 2008)

• Subject area exerts larger effects on satisfaction than characteristics of students or institution

• Variance in overall satisfaction: - 2.5% accounted for by differences between institution- 3.5 – 7% accounted for by subject area

(Surridge, 2009)

Analysis of HEFCE 2011 Data

• Satisfaction rates from all UK Universities organised by broad subject groups

• STEM: Engineering & Tech; Computer Sciences; Maths Sciences

• Non-STEM: Creative Arts; Geography; Education; History & Philosophy; Languages

• Analysis of satisfaction with teaching by subject area compared with global satisfaction

Staff Have Made the Subject Interesting

STEM

NON-STEM

Staff Are Enthusiastic About What They are Teaching

STEM

NON-STEM

Analysis of NSS 2011 Data• Comparison of 18 UK Universities that all had the

same broad provision • STEM: Psychology; Elec Eng; Chemistry; Maths/Stats• Non-STEM: English; History; Sociology; Law

• Compared NSS teaching ratings: i) Staff are good at ability to explain

STEM = 88% < Non-STEM = 93%ii) Staff make the subject interesting

STEM = 78% < Non-STEM = 87%

Staff Are Good At Explaining Things

Staff Have Made the Subject Interesting

Participants

STEM (n = 583)

- Health and Bioscience (e.g. Biomedical Science)

- Computing and Engineering (e.g. Civil Engineering)

- Psychology (e.g. Forensic Psychology)

• M= 46% & F = 54%• < 20 yrs = 23%

21-25 yrs = 35% > 25 yrs = 42%

Non-STEM (n = 597)

- Law (e.g. Criminology; LLB)

- Humanities (e.g. Anthropology)

- Arts (e.g. Fine Art; Design)

• M= 38% & F = 62%• < 20 yrs = 27%

21-25 yrs = 33% > 25 yrs = 40%

Satisfaction Survey

• NSS questions administered via online questionnaire at end of first year of study

• Five sub-dimensions:- Teaching (α = .90)- Assessment (α = .82) - Academic Support (α = .85)- Organisation (α = .87)- Learning Resources (α = .84)

Results

NSS Sub-dimension STEM Non-STEM

Teaching 3.67 (0.89) 3.80 (0.93) Assessment & Feedback 3.49 (0.93) 3.48 (0.98)

Support & Guidance 3.54 (0.98) 3.48 (1.02)

Organisation 3.62 (0.95) 3.58 (1.00)

Resources 3.76 (0.99) 3.60 (1.04)

• No significant Class Size or Age differences• Significant UCAS entry points differences: STEM

Results

• MANCOVA revealed significant difference between NSS scores of STEM vs Non-STEM (Wilks Λ = .97, F(6, 1169) = 5.75, p = .000)

• Resources: STEM > Non-STEM (F(1, 1174) = 5.86, p = .016)

• Teaching: STEM < Non-STEM (F(1, 1174) = 10.23, p = .001)

STEM Status x Sex Interaction(Wilks Λ = .99, F(6, 1169) = 2.75, p = .012)

STEM males < non-STEM males & STEM females with

Teaching

STEM males and non-STEM females < non-STEM males with

Assessment and Feedback

Satisfaction with Teaching(F(4, 1167) = 2.37, p = .05)

NSS ITEMS Non-STEM

STEMANCOVA (1, 1170)

Teaching staff are good at explaining things

3.75(0.99)

3.61(1.01)

F = 10.36**

Teaching staff have made the subject interesting

3.71(1.03)

3.53 (1.02)

F = 15.39***

Teaching staff are enthusiastic about what they are teaching

3.88 (1.08)

3.73(1.01)

F = 11.52**

My Programme is Intellectually Stimulating

3.92(1.10)

3.81 (1.04)

F = 8.06*

* = p<.01 ** = p<.005 *** = p<.001

Good at Explaining Things Made the Subject Interesting

Enthusiastic about Content

Programme Intellectually Stimulating

Satisfaction withAssessment & Feedback

NSS ITEMS Non-STEM

STEMMANCOV

A (1, 1170)

The Assessment Criteria have been clear and given in advance

3.57(1.15)

3.76(1.09)

F = 6.84**

Assessment arrangements and marking have been fair

3.53(1.14)

3.56 (1.10)

F = .003 NS

Feedback on assignments has been within 4 weeks

3.33 (1.28)

3.19(1.30)

F = 7.24**

Feedback on assignments has been useful

3.54(1.20)

3.47(1.23)

F = 4.32*

* = p<.05 ** = p<.01

Feedback Within 4 Weeks

Feedback has been Useful

• STEM females and Non-STEM females do not differ in satisfaction with feedback

• STEM Males are less satisfied than non-STEM Males with timing and utility of feedback

Conclusions

• STEM students are generally less satisfied with the quality of teaching, enthusiasm of their teachers and intellectual stimulation of their degree

• There is also evidence that STEM students are less satisfied with particular assessment practices relating to feedback

• This may be due to sex differences but needs further exploration (e.g. perceptions of teaching practices, student expectations, parental/school expectations)

Conclusions

• Relative STEM dissatisfaction may be due to STEM vs non-STEM teaching differences; student differences or an issue of expectations and branding

• HEFCE (2011) and QAA (2012) analyses of student experience both consider subject differences but do not attend sufficiently to patterns in the data

• Ignoring differences for short-term gains may have significant long-term implications (e.g. recruitment & retention)