Bubble in The System:

Presenters: Simon Winberg & Robyn Verinder

Department of Electrical EngineeringUniversity of Cape Town

November 2009

Accommodating twice the students… and more than usual who aren’t meeting expectations.

Bubble in The System:

A look at risk planning, what went wrong, how things progressed regardless, and what the future might hold…

UCT Department of Electrical Engineering

Outline The Context Problems and what was done about

them– How they happened– How we planned for these potential risks– What was done in practice

Will these problems go away? Tracking the bubble Reflections Conclusions2


The Context 1st year Computing for Electrical

Engineers course– For students who did not do Computer

Studies (CS) for matric OR– For students who did CS but achieved

a lowCS mark (below a C).Students (w/o CS) but did wellfor math and science have thenoption to do 1st year computercourses in the CS dept.

3


Focus of this study

4

EE1st yearIntake

EEE1003W

CS100X

Computing forElectrical Engineers

Computer Science 1st year


Class composition (2008)

87 registered (4 repeating) 84 wrote final exam (3 dropped

out or transferred)

5

Computing for Electrical Engineers


Class composition (2009)

167 registered (3 repeating) 152 wrote final exam (15 dropped

out or transferred – 9% dropped vs. 3% in ‘08)

6

85 more students at start of 200968 more throughout 2009 than 2008 81% more students in 2009

Total 1st year intake for Electrical Engineeringwas 49% higher this year than average of 2006-8

Computing for Electrical Engineers


Problem 1: Large intake Larger than anticipated

intake for EE 1st year intake up

by 49% from 2008 Much of this 49% additional students

seems to have gone into the Computing for Electrical Engineers course

7


Enrolment per programme

8

2006-8 the split between programmes remained fairly consistent;

2009 more mechatronics

162 184

175 260


Enrolment figures:2006 - 2009

Department 2006 2007 2008 2009Electrical and computer engineering (ECE)

34 49 37 51

Electrical Engineering (general programme)

83 79 87 108

Mechatronics (ME) 45 56 51 101

Total 162 184 175 260

9


Total EE first year intake by nationality

10

162184 175

262


Problem 1 consequences Finding a big enough

lecture venue; Lab space limitations &

availability; Difficulty finding

enough good tutors. Problems down the

line…11


Problems downthe line…

Lab expansions– Size of 2nd & 3rd year laboratories increased– Acquisition of more equipment

Staffing– More contract staff

(TAs, tutors, etc.) – More duplicated lectures,– More double-period lectures

Administrative headaches– Finding & focusing funding towards

these ‘immediate’ needs12

LECTURE 1 LECTURE 1I’m Bob I’m John


Problem 2:Fundamental Problem

“A big bubble in the system”– Significantly weaker students– Possibly a once-off event

A larger portion of students found the course work more difficult than previously… although the level of the coursework has not changed much

13


Is it a once-off bubble? But is this a bubble,

i.e. a once-off event,or is it the new norm?

Will the proportion ofweak students returnto what it was before?

Perhaps… will be discussed later…

14


2010 Intake estimates:‘admission probable’

Department 2010 admission probable

Expected

Chemical Engineering 150 95Civil Engineering 117 95Electrical Engineering 170 180Mechanical Engineering

152 140

15

*Figures provided by UCT EBE Faculty (20 Nov 2009)

• The latest estimated intake figures above have been calculated by the EBE faculty based on provisional results.

• The actual figures can only be confirmed once this year’s matric marks have been obtained and processed.

• It might just be a once-off bubble


How did it happen? Change in matric subjects

and examination Removal of higher grade option Change to 2 papers for math

exam– Removed geometry paper

Matric results appear to be skewed– No longer a good predictor for

performance in engineering, which higher grade math used to be.

16

Oops!


NationalBenchmark Tests (NBTs)

National benchmark tests confirm these experience in electrical engineering:– Only 7% proficient in math– 25% proficient quantitative literacy– 47% proficient academic literacy in

English

17

Statistics from HEQC (2009). MacGregor, K. ‘South Africa: Shocking results from university tests’, World University News – Africa Edition. Issue 35, August (2009).


NBTs Should these tests still be

used just to benchmarkmatric results, and notinfluence admission?

OR could these be used todecide university admission?

18


2008 Risk planning related to Problem 1 (intake) Faculty provided 2009 estimation:

– Plan for 20% increase: i.e., 100 students (but not an 80% increase!)

– Based on total EE intake estimations for past 5 years (giving average of 10% increase per year over 5 years) *

Risk planning for Problem 1:– Ensuring sufficient budget for two

additional tutors.– More multiple choice / other techniques

for faster marking19

* Based on email correspondence November 2008


2008 Risk planning related to Problem 2 (weaker students)

Mostly based on 2008 reflections…

Initial diagnostic assessment (IDA)– Held in 1st week to identify

students needing extra support Bi-weekly extra tuition sessions

– TA to assist struggling students Hot seat during lab times

– Chief tutor / tutors’ tutor help students with more tricky theory-related questions or assist other tutors.

20


What was done in practice…

Our risk planning was beneficial, but:

The problem wasoutside theenvisagedboundaries

21

www.drollthings.com/?p=2384

http://www.drollthings.com/?p=2384


Initial diagnostic assessment (IDA) Results Students achieved basic requirements

of:– Computer literacy requirements

(use of MS Word, Google searches, etc.).– Basic computer skills (using MS Excel, etc.) for

basic engineering type problems. But later showed difficulties in:

– Mathematics: difficulty in more complexproblem-solving tasks;

– Academic English proficiency: understandingproblem descriptions; articulating solutions.22


What was done More tutors Lab assignments made

smaller, given more time– 2008: 6 pracs– 2009: 8 pracs

Project separated into two– 2008: 1 project– 2009: 2 projects

Additional tests, to help improve the pass rate for June and Nov exams23


What was done

24

… essentially a lot of WORK - chopping and changing aspects of the course …


Throughput Despite challenges, the

pass rate was not too farbelow previous years…

Pass rate 2009: 82% (18% failed)– 141 wrote exam (11 did not get a DP)– 125 of 152 students passed. 8 borderline.

Pass rate 2008: 86% (14% failed)– 82 wrote final exam (2 did not get a DP)– 72 of 84 students passed. 2 borderline.

25

UCT Department of Electrical Engineering26

But…Will the problems go away?


± 1,500,000 learners

started school in 1997± 500,000 learners (30%) wrote matric in 2008

<100,000 (20%) achieved a university endorsement

Will the problems go away?

… So, the problem is likely to remain for a while longer Approximate HEQC figures (2009)

UniversityIntake

The bubble

=100,000 learners

27

UCT Department of Electrical EngineeringM

athe

mat

ics, P

robl

em-s

olvi

ng,

hig

h le

vel a

cade

mic

liter

acy

Limited depth of knowledge Students have basic

computer ‘skills’ But lacking

foundation knowledge (maths, advanced academic literacy) needed for programming-based solving of engineering problems

Computer literacy, basic applications, basic academic

English

Adapted from: European Science Foundation (2002)28


Tracking the bubble...


Tracking the bubble: how did the students progress?

Which of the weak become strong? Methodology

– Utilizing an adapted for of skills matrix presented by Nicholls (1995)

– Matric results used to determine initial placement of averaged math & science mark (left = higher mark)

– Initial diagnostic assessment used to determine initial placement of computer skill

30


Knowledge and skills matrix

31

Broad engineering knowledge / logical

High computer programming skills

Narrow engineering knowledge / poor logic

Programming-based engineering problem solving

Can develop programs; but only solve simple problems

Understand the theory; but unable to apply this knowledge in computer programs

Basic level; low potential

Low computer programming skillsRepresentation schema adapted from Nicholls, J. (1995).

"The MCC decision matrix", Journal of Management Decision: 33(6): 4-5.


Overview of results

32



33








Low computer programming skills

Results based on cohort of 2009 students that exhibited good math & science matric grades, but demonstrated only mediocre computer proficiency in computer test 1.

Full steamahead!



34









Results based on cohort of 2009 students that exhibited good math & science matric grades, but demonstrated only mediocre computer proficiency in computer test 1.

Guideme!



35









Results based on cohort of 2009 students that exhibited comparatively lower math & science matric grades, but demonstrated good computer proficiency in computer test 1.

Potentialtraps

Full steamahead!



36









Results based on cohort of 2009 students that exhibited comparatively lower math & science matric grades, but demonstrated good computer proficiency in computer test 1.

SOS!


Tracking the bubble:How did the weaker students’ positions within the skills matrix change?


Knowledge and Skills matrix – starting point

38





=10x students

55

Failed IDA: 4

60

33


Knowledge and Skills matrix – progression

39





=10x students

54

Failed IDA: 5

60

33

1



40





=10x students

54

Failed IDA: 5

60

33

6 really good



41





=10x students

54

Failed IDA: 5

60

33



42





=10x students

54

Failed IDA: 5

9

84

51 passed



43





=10x students

54

Failed IDA: 5

84

(2)

7



44




Low computer programming skills=10x students

13

Failed IDA: 5

127

(5)

7

41


Knowledge and Skills matrix – final point

45




Low computer programming skills=10x students

13

Failed IDA: 5

127

(5)

7


Reflections Students with low math/science and

low computer skills generally achieved notably less progression than their class mates

Many reasons for this– Lack of motivation, lack of confidence

(e.g., not daring to ask for help)– Insufficient support structures…

see:

46

Crooks, T. (1988). "The impact of classroom evaluation practices on students." Review of educational research 58(4): 438.Zimmerman & Bandura, et al. (1992). "Self-motivation for academic attainment: The role of self-efficacy beliefs and personal goal setting." American Educational Research Journal 29(3): 663.


Reflections Students showing good

math/science and lower computer skills generally performed successfully– Confirmed by many studies e.g. Baron

& Norman (1992)* This cohort had the necessary

foundational knowledge to succeed. BUT: why were 7 of the 60 left

behind?47

* Baron, J. and M. F. Norman (1992). "SATs Achievement Tests, and High-School Class Rank as Predictors of College Performance." Educational and Psychological Measurement 52: 1047-1047.


Reflections Students showing low math/science but

good computer skills generally performed successfully… but some didn’t.

This cohort had to build the necessary foundational knowledge, which more advanced knowledge depended on.– Starting with good computer skills, in the

information age, is a likely facilitating factor to learning*

13 of the 55 students failed

48

* Bergin, S. and R. Reilly (2005). "Programming: factors that influence success." ACM SIGCSE Bulletin 37(1): 411-415.


Reflections Students showing high

math/science and good computer skills generally performed excellently.

This cohort had the advantage of the needed foundational knowledge, in addition to good computer skills to help them learn new material.– These students went from good to

great!– These are what we all hope for.49


Reflections What could have caused the 7 of

the 60 students that did reasonably for math & science to fail?– I would have expected good math and

science marks to demonstrate that the students would quickly learn computer skills

– Perhaps 7/60 (12%) isn’t a major concern

– Something may be wrong with my approach

– Could it be an effect of last year’s matric results?

50

Conclusion The UCT EE department intake

may be lower next year – but we still have at least one ‘bubble’ to deal with…

These may be more bubbles, perhaps of different types, on the way in…

–What will the difficult math paper cause?

–Further skewing of matric results?

Time will tell…

51

52

Questions??

Presenters: Simon Winberg & Robyn VerinderDepartment of Electrical Engineering

University of Cape TownNovember 2009

Accommodating twice the students and morethan usual who aren’t meeting expectations.

Bubble in The System:

Documents

Bubble in The System: