Teaching @IITB – Some Data

1

Teaching @IITB – Some Data

Institute Faculty MeetingIndian Institute of Technology Bombay,

MumbaiFebruary 10, 2010

2

Teaching @IITB

Teaching

• Important activity

• Less ‘discussed/tracked’ compared to research

• Data, data every where!

• Data? Number game here too?

• Looking at data can help

• If we start looking at the data, what we collect and the process of collecting it will improve.

3

Teaching @Aero –Survey by Students

4

Teaching – Measures?

• Quality. How well? Student feedback?

– Influenced by liberal grading

– Senior students evaluate stringently

– Teach less, teach well

– Etc.

5

Teaching @Aero

• 5 Years, 10 Semesters, 22 faculty

• Number of courses – Total offered = 293– Not evaluated = 77 (26%)

• Each course has– Credits 4, 6 or 8– Taught by 1, 2 or 3 faculty– Average grade awarded 0 to 10– Average student evaluation 0 to 100

6

Teaching @Aero – Evaluation Vs Grading?

Grading Vs Evaluation

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8 9 10

Grade (0 to 10)

Eva

luat

ion

Correlation Coefficient = 0.28Average grade = 6.77Average evaluation = 75.9

7

20

40

60

80

100

1 2 3 4

Year

Eva

luat

ion

Teaching @Aero – Sr Students Evaluate Stringently!

No of Courses No of Deliveries Average Std Dev

1st Year 1 3 63.14 11.9

2nd Year 5 18 59.62 10.9

3rd Year 10 21 60.08 10.6

4th Year 7 9 59.01 5.6

8

Teaching @Aero – Load

Each ‘’ represents one faculty

0

10

20

30

40

50

0 1 2 3

No of (6 Credit) Courses/semester

No

of S

tud

en

ts

9

Teaching @Aero– Quantity Vs QualityData for 2002 to 2006

40

50

60

70

80

90

100

10 20 30 40 50

Students taught per semester

Eva

luat

ion


10

Teaching @AeroEvaluation Vs Class Strength!

y = -0.4323x + 84.5170

20

40

60

80

100

0 10 20 30 40 50

Class strength

Eva

luat

ion

Larger the class, tougher to get good evaluation

11

Teaching Data of IITB 1999-2007

12

UG – Grades & Evaluations

Department AE EE ME CH MT CE

Grade

Average 6.42 6.95 6.88 6.74 6.95 7.11

Std dev 1.42 1.42 1.31 1.25 1.61 1.03

Course Evaluation

Average 68.8566.9

765.15 66.64 66.60 66.34

Std dev 13.3013.2

414.50 13.70 14.14 13.27

No of courses not evaluated

47 46 41 48 47 44

Avg ratio no of evaluations to

no of registrations0.69 0.61 0.52 0.60 0.63 0.61

13

PG – Grades & Evaluations


Grade

Average 7.28 7.96 7.74 7.90 8.02 8.23

Std dev 1.66 1.06 1.15 1.11 1.42 0.90

Course Evaluation

Average 80.72 80.7277.0

580.75

79.06

81.04

Std dev 9.25 9.5411.0

09.17

12.52

11.28

No courses not evaluated

48 42 46 45 36 53

Avg ratio no of evaluations

to no of registrations

0.70 0.67 0.67 0.72 0.71 0.79

14

UG+PG Teaching Load

AE EE ME CH MT CE

Total no of courses 502 690 705 510 500 624

Average per sem 27.88 38.3639.1

928.31 27.77 34.67

Faculty teaching load

Total Faculty# man-sems

319 581 614 480 452 267

Avg Faculty per sem 17.72 32.2834.1

126.67 25.11 14.83

No course/faculty/sem 1.57 1.19 1.15 1.06 1.11 1.25

Avg class strength 18.35 50.1840.5

343.53 33.77 30.63

15

Trends in Grading

16

I used to think!

“Academic standards have fallen. Students are not as good as they used to be, etc” Underlying assumption students are the problem

See the data

17

Over the years! Dept A : Course-1

0

1

2

3

4

5

6

7

8

9

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

4*F1, 4*F2, F3

Avera

ge g

rad

e &

Std

Dev

Average Grade Point Grade Standard deviation

Linear (Average Grade Point) Linear (Grade Standard deviation)

CS-152

18

Over the years! Dept A : Course-2

0

1

2

3

4

5

6

7

8

9

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

3*F1, F2, F1, 4*F3

Ave

rag

e G

rad

e &

Std

Dev



CS-207

19

Over the years! Dept B : Course-1

0

1

2

3

4

5

6

7

8

9

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

F1, F2, F3, 2*F2, F4, F5, 2*F6

Avera

ge G

rad

e &

Std

Dev



CS-212 (EE)

This course presents a trend of reducing grades. But if this one data point is discarded then we have a flat variation

20

Over the years! Dept B : Course-2

0

1

2

3

4

5

6

7

8

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

2*F1, F2, 3*F3, 3*F4

Avera

ge G

rad

e &

Std

Dev



EE-002

21

Over the years!Dept C : Course-1

0

1

2

3

4

5

6

7

8

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

F1, 5*F2, F1, 2*F3

Ave

rag

e G

rad

e &

Std

Dev



AE-152

22

Over the years! Dept C : Course-2

0

1

2

3

4

5

6

7

8

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

6*F1, 2*(F1+ F2), F1+F3

AV

erag

e g

rad

e &

Std

Dev



AE-330

23

• Several comments come to mind

– Quality of data

– What additional data must be collected

– What can be done with the data

• But, a more thorough study required

• We must pay more attention to these things

24

Publications in the area of Education1999-2009

25


In Journals Journals + Conf

Stanford 306 365

26



Stanford 306 365

MIT 91 117

27



Stanford 306 365

MIT 91 117

IITB 4 12

28



Stanford 306 365

MIT 91 117

IITB 4 12

IITK 10 11

• > 160 Journals covering education• 35 Journals covering engineering education

29

Several Initiatives Worldwide

• National Academy of Engineering, USA is concerned about “Educating the Engineer of 2020”

• Interventions recommended & tried out

– FYEP (First Year Engineering Projects)

– Purdue EPICS Project in experiential learning

– Etc.

• CDIO - Educational framework for producing the next generation of engineers.

(Conceiving, Designing, Implementing, Operating real-world systems and products).

30

We need to take teaching lot more seriously

Thank You

31

Extra slides

32

Some Suggestions

• Validate student evaluations with registration details before accepting

• Capture data on faculty status ‘Lien’, ‘Sabbatical’, ‘Not teaching this sem’, etc. Above data on ‘Faculty man semesters’ cannot account for those who are in the department but do not teach.

• Enable logging of unequal sharing of courses by faculty (ie. If 2 faculty are sharing a course presently they get credit of 0.5 each)

33

UG Teaching Load


Total no of courses* 224 255 317 280 271 267

Average courses per sem

12.44 14.15 17.62 15.56 15.05 14.81



218 267 355 308 295 299

Faculty available per sem

12.11 14.83 19.72 17.11 16.39 16.61


Average class strength 29 78 63 62 50 54

* Courses are normalized to 6 credit courses # Only a sub-set of the dept faculty may be involved in UG Teaching. This is the average over the faculty who are involved in UG teaching

34

PG Teaching Load


Total no of courses* 278 436 388 230 229 358

Average per semester 15.44 24.21 21.57 12.75 12.72 19.87



240 410 395 247 232 314

Avg Faculty per sem 13.33 22.78 21.94 13.72 12.89 17.44


Avg class strength 9.57 33.72 21.77 20.70 14.80 13.52

* Courses are normalized to 6 credit courses # Only a sub-set of the dept faculty may be involved in PG Teaching. This is the average over the faculty who are involved in PG teaching

35

Teaching Data 1999-2007

This summary based on teaching data for 9 years is presented with following

comments

• Study is more to see what data can ‘tell’• Since the data may not have been

captured with a view to use it thus, we may have to tighten the processes to correctly capture the data

• Some observations & suggestions have also been made

36

Some Suggestions/Recommendations

• Max & Average feedback for a course to be intimated to faculty designated for a course

• Course evaluation to be done for all courses.

• Capture un-equal sharing of course load• Need to log summer courses• 21 Qs in Course Evaluations?

37

Grades, Evaluations : Some Observations

• In all departments many courses have gone without getting evaluated.

• Most departments have shown more than one course that has got evaluated by more students than are registered for it. To be looked into!

• Civil has highest average grade for both UG & PG with least standard deviation. Metallurgy comes next.

Summary of data

38

Teaching – Histograms

Distribution of Class Size

0

20

40

60

80

100

0 5 10 15 20 25 30 35 40 45 50

Class size

No

of C

ou

rse

s

No of courses that had class strength between 0 to 5

39


Distribution of Grading

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8 9 9.5 10

Grade (0 to 10)

No

of C

ou

rse

s

Average grade = 6.77How does this compare across departments?Aero students find our grading very stringent.

40


Distribution of Evaluation

0

20

40

60

80

0 10 20 30 40 50 60 70 80 90 95 100

Evaluation

No

of C

ou

rse

s

Average evaluation = 75.92

41

Teaching – Course / Student Load


0

10

20

30

40

50

0 1 2 3

No of (6 Credit) Courses/Sem/Faculty

No

of

Stu

de

nts

Average Courses/faculty/sem = 1.25 Average students/course = 30(normalized to 6 credit course)

42

Teaching Quality - Comparison

020406080

100

2002 2003 2004 2005 2006 2006

Year

Feed

back

020406080

100

2002 2003 2004 2004 2005 2006

Year

Feed

back

Same course– Wide variation across faculty– Less variation for same faculty

43

Some Suggestions/Recommendations

• Max & Average feedback for a course to be intimated to faculty designated for a course

• Course evaluation to be done for all courses.

• Targeted evaluation > 70

• Capture un-equal sharing of course load• Need to log summer courses• 21 Qs in Course Evaluations too much?

44

Detailed study of teaching related data planned

Data available 1999 onwardsStudent evaluations 2002 onwards

45

Some Data!

• Numbers, numbers Quality?– Numbers alone not sufficient . . . – Numbers may be necessary indicator . . .

• Assorted data collected over 2-3 years• Aim

– Not to judge anyone /anything– Not to make a point– Data can help if captured thoughtfully and

processed with care– We must also start talking about teaching! It is

important.

46

Teaching – Across Faculty

Each faculty

– Nsem = no of semesters taught ≤ 10

– Nc = total courses taught

Ci = credits,

Fi = 1.0 not shared,

= 0.5 shared with another

Ni = no of students registered

Ei = Evaluation by students

i = 1, Nc

47

Teaching – Some Indices

Teaching load related

• Equi 6 Cr courses, Nc6 = (Nc Fi Ci )/ 6

• Avg courses/sem, Nc-sem = Nc6 / Nsem

• Avg students/course, Ns-c = (Nc Fi Ci Ni )/ (Nc Fi Ci )

= (Nc Fi Ci Ni )/ (6 Nc6)

• Avg students/sem, Ns-sem = Ns-c * Nc-sem

48


Evaluation related

Avg evaluation, E = (Nc Fi Ci Ni Ei)/(Nc Fi Ci Ni )

= (Nc Fi Ci Ni Ei)/(6 Nc6 Ns-c)

AE-123 AE-321

Ci 6 6

Fi 1 1

Ni 20 10

Ei 70 80

E = (70*20+80*10)/2/15 = 73.3

49

Teaching – Across Faculty

Each faculty

– Nsem = no of semesters taught ≤ 10

– Nc = total courses taught

Ci = credits,

Fi = 1.0 not shared,

= 0.5 shared with another

Ni = no of students registered

Ei = Evaluation by students

i = 1, Nc

50


Teaching load related

• Equi 6 Cr courses, Nc6 = (Nc Fi Ci )/ 6

• Avg courses/sem, Nc-sem = Nc6 / Nsem

• Avg students/course, Ns-c = (Nc Fi Ci Ni )/ (Nc Fi Ci )

= (Nc Fi Ci Ni )/ (6 Nc6)

• Avg students/sem, Ns-sem = Ns-c * Nc-sem

51


Evaluation related

Avg evaluation, E = (Nc Fi Ci Ni Ei)/(Nc Fi Ci Ni )

= (Nc Fi Ci Ni Ei)/(6 Nc6 Ns-c)

AE-123 AE-321

Ci 6 6

Fi 1 1

Ni 20 10

Ei 70 80

E = (70*20+80*10)/2/15 = 73.3

Documents

Teaching @IITB – Some Data