CBL as seen through Students’ Eyes

CBL as seen through Students’ Eyes

How students’ feedback can be used to evaluate CBL-measures

Clemens Reindl, 10.9.2019

W. Edwards Deming 1900 – 1993

Deming Cycle / PDCA Cycle

Designing the CBL-Policy/ -Strategy/ -Initiative/ -Project

Implementing the CBL-Policy/ -Strategy/ -Initiative/ -Project

Evaluating the CBL-Policy/ -Strategy/ -Initiative/ -Project

Deriving measuresfrom the evaluation results

Students’ feedback

Impact analyses: The result-chain

Inputs: Things we need (e.g. Lecturerswith didactic skills)

Activities: What we do (e.g. use of activating teaching methods)

Outputs: the immediate effects (e.g. student skills)

Outcomes: medium term effects (e.g. graduates use skills in jobs)

Impact: long term effects (e.g. lowunemployment due to well skilledworkforce)

2%

7%

4%

5%

8%

12%

15%

23%

18%

28%

40%

41%

46%

59%

66%

15%

14%

19%

27%

29%

29%

35%

34%

40%

42%

34%

36%

35%

32%

26%

29%

32%

33%

42%

38%

37%

39%

31%

37%

26%

21%

20%

16%

8%

7%

53%

48%

43%

26%

25%

22%

11%

12%

5%

4%

6%

3%

3%

1%

0%

0% 20% 40% 60% 80% 100%

Flipped (inverted) Classroom

Other: (e.g: flashlight, world-cafe, group puzzle etc.)

Learning by Teaching (Peer-Teaching)

Research-based learning

Problem-based Learning (PBL)

Role Play

Brainstorming / mindmap creation

Impulse presentation

Practical scientific methods

Project-oriented work

Student presentation with handout

Case studies

Discussion

Guided self-study

Frontal teaching

frequently sometimes rarely never

How often were you confronted with the following methods in the courses?

End-of-Study Survey

Recommendations

Check the activitiesAre CBL-measures implemented as planned?

4,7

4,8

4,9

4,8

5,1

5,3

1 2 3 4 5 6

I can develop and apply learning strategies

I can put my acquired knowledge intopractice well

I acquire the ability to apply theories as wellas methods and concepts

I acquire key competences such as the abilityto work in a team, the ability to solve

problems and the ability to organise myself.

I develop competences to act independentlyand on my own responsibility

With this study I can extend my specializedknowledge

Scale from 1= "disagree" to 6 = "totally agree"

Please state in how far you agree with the following statements regarding knowledge acquisition.

Student Survey

4,1

4,4

4,5

4,4

4,6

5,1

4,2

4,5

4,6

4,4

4,6

5,2

4,7

4,8

4,9

4,8

5,1

5,4

4,6

4,7

4,8

4,7

5,0

5,3

4,7

4,8

4,9

4,8

5,1

5,3

1 2 3 4 5 6

I can develop and apply learning strategies

I can put my acquired knowledge intopractice well

I acquire the ability to apply theories as wellas methods and concepts

I acquire key competences such as the abilityto work in a team, the ability to solve

problems and the ability to organise myself.

I develop competences to act independentlyand on my own responsibility

With this study I can extend my specializedknowledge

Scale from 1= "disagree" to 6 = "totally agree"

2018

2017

2016

2015

2014

Please state in how far you agree with the following statements regarding knowledge acquisition.

Student Survey

Recommendations


Check the outputDo the CBL-measures have an effect on the students?

Check again and againWhat trends are the CBL-measures leading to?

How important are the following aspects of the profession to you personally? Scale from 1 = "Very

important" to 5 = "Not important at all"

Graduate Survey

99%

98%

94%

92%

92%

91%

89%

84%

80%

75%

74%

73%

71%

67%

66%

62%

55%

37%

1%

2%

5%

6%

7%

7%

9%

13%

15%

21%

19%

17%

21%

23%

27%

23%

31%

29%

0%

0%

1%

2%

1%

2%

2%

3%

4%

4%

8%

10%

8%

10%

7%

15%

14%

35%

0% 20% 40% 60% 80% 100%

Good working atmosphere

Interesting work content

Possibility to use acquired competences

Largely independent work planning

Possibility of further professional qualification

job security

To have a job that challenges me

Possibility to realize your own ideas

Enough time for leisure activities

High income

Possibility to do useful things for the general public

Good opportunity to combine family responsibilities with a career

Clear and regulated work tasks

Good career opportunities

Social respect and recognition

Takeover of coordination and management tasks

Possibility to exert influence on society

Opportunity for scientific work

1 + 2 3 4 + 5

To what extent do the following aspects apply to your current professional situation? Scale from 1 = “To

a very high degree" to 5 = "Not at all"

Graduate Survey

83%

82%

80%

79%

78%

76%

65%

63%

63%

61%

61%

59%

56%

46%

39%

35%

32%

26%

11%

11%

13%

14%

17%

14%

21%

22%

26%

21%

24%

27%

31%

22%

38%

38%

31%

21%

6%

7%

7%

7%

5%

10%

13%

14%

11%

18%

15%

14%

14%

32%

23%

27%

37%

54%

0% 20% 40% 60% 80% 100%

Largely independent work planning

Good working atmosphere

job security

Possibility to use acquired competences

Interesting work content

To have a job that challenges me

Enough time for leisure activities

Good opportunity to combine family responsibilities with a career

Clear and regulated work tasks

Possibility of further professional qualification

Possibility to do useful things for the general public

Possibility to realize your own ideas

Social respect and recognition

Takeover of coordination and management tasks

High income

Possibility to exert influence on society

Good career opportunities

Opportunity for scientific work

1 + 2 3 4 + 5

Recommendations




Check the outcomeDo the CBL-measures help the students in their working life?

94%

92%

89%

79%

85%

88%

83%

85%

54%

86%

81%

85%

92%

80%

87%

75%

92%

85%

4%

7%

10%

18%

13%

10%

14%

12%

30%

12%

16%

13%

6%

14%

11%

20%

6%

11%

1%

1%

1%

4%

2%

2%

3%

3%

16%

2%

3%

2%

1%

6%

2%

5%

2%

5%

0% 20% 40% 60% 80% 100%

Ability to obtain information from different sources

Ability to recognize interconnections

Ability to assess the relevance of information

Ability to make decisions

Ability to develop problem-solving strategies

Ability to plan and organize

Ability to express oneself adequately orally

Ability to express yourself adequately in writing

Ability to communicate in English

Ability to reflect on oneself

Ability to deal with criticism received

Ability to constructively criticize

Ability to work as part of a team

Ability to apply theoretical knowledge in practice

Ability to adapt flexibly to new situations

Ability to develop new ideas

Ability to work independently

Mastery of one's own subject, one's own discipline

1 + 2 3 4 + 5

To what extent did you have the following skills/competences at the time of graduation? Scale from 1= "To a very high degree" to 5 = "Not at all"

Graduate Survey

76%

93%

92%

88%

88%

87%

89%

81%

40%

67%

75%

74%

89%

85%

94%

70%

95%

91%

15%

5%

5%

9%

9%

11%

8%

14%

20%

20%

19%

17%

7%

10%

4%

21%

3%

5%

9%

2%

3%

3%

3%

2%

3%

6%

40%

13%

6%

9%

4%

6%

2%

9%

2%

4%

0% 20% 40% 60% 80% 100%



















1 + 2 3 4 + 5

To what extent are the following skills/ competences required in your current job? Scale from 1= "To a very high degree" to 5 = "Not at all"

Graduate Survey


















Mastery of one's own subject, one's own discipline -0,4

-0,3

0,0

-0,1

0,1

0,0

-0,3

0,5

0,4

0,0

-0,1

-0,2

0,1

-0,1

-0,3

-0,2

difference

0,3

-0,1

Required versus acquired competencesGraduate Survey

Recommendations





Check on a detailed levelFor which competences are the CBL-measures sufficient and for which not?



















-0,8

0,6

-0,4

0,4

-0,6

0,9

-0,6

0,7

-0,2

-0,6

1,0

-0,3

0,5

0,5

Study Programme D

difference

0,5

-0,1

0,3

0,0

Ability to obtain information from different sourcesAbility to recognize interconnectionsAbility to assess the relevance of informationAbility to make decisionsAbility to develop problem-solving strategiesAbility to plan and organizeAbility to express oneself adequately orallyAbility to express yourself adequately in writingAbility to communicate in EnglishAbility to reflect on oneselfAbility to deal with criticism receivedAbility to constructively criticizeAbility to work as part of a teamAbility to apply theoretical knowledge in practiceAbility to adapt flexibly to new situationsAbility to develop new ideasAbility to work independentlyMastery of one's own subject, one's own discipline

-0,6

0,6

-0,2

0,9

0,5

0,5

-0,5

-1,0

-0,5

0,8

1,0

0,3

0,7

0,4

-1,0

-0,5

-0,7

0,4

Study Programme C

difference

Ability to obtain information from different sourcesAbility to recognize interconnectionsAbility to assess the relevance of informationAbility to make decisionsAbility to develop problem-solving strategiesAbility to plan and organizeAbility to express oneself adequately orallyAbility to express yourself adequately in writingAbility to communicate in EnglishAbility to reflect on oneselfAbility to deal with criticism receivedAbility to constructively criticizeAbility to work as part of a teamAbility to apply theoretical knowledge in practiceAbility to adapt flexibly to new situationsAbility to develop new ideasAbility to work independentlyMastery of one's own subject, one's own discipline

-0,9

-0,4

-0,6

-0,7

-0,9

0,8

0,6

-0,3

0,5

0,6

0,5

0,2

0,7

0,4

0,9

0,0

Study Programme B

difference

1,2

0,0



















-0,1

0,7

-0,6

0,7

0,2

-0,4

0,8

0,4

-0,9

-0,9

-0,1

0,4

0,6

0,7

Study Programme Adifference

-0,9

1,0

0,6

-0,1

Recommendations






Segment the resultsWhere in my organization are the CBL-measures sufficient and where not?

90%

82%

80%

84%

73%

77%

0% 20% 40% 60% 80% 100%

High level of instrumentalcompetence

High interpersonal skills

High systemic competencies

Competencies at the end of Study(Scores 1 + 2 on a five-step response scale from 1 = 'very high' to 5 =

'not at all')

FH Campus Wien

other HEIs

Recommendations







Benchmark yourself with othersHow successful are we with our CBL-measures in comparison to other HEIs?

Recommendations







Benchmark yourself with othersHow successful are we with our CBL-measures in comparison to other HEIs?

Group Work

Consider your own institution (university, ministry, agency, etc.). Whichmeasures/initiatives/projects could be carried out there with regard toCBL (or are already being carried out)? Try to find at least one example. Present it to the group (volunteers first). Find together in the group ideas and answers to the following

questions:

• How could one check how successful these measures are (the Check

in the PDCA-Cycle)?

• What resources and know-how would be available for this evaluation?

• What additional resources and know-how would be needed?

• Which problems could arise? How could they be solved?

Try to find and discuss several case studies from differen group members.

Documents

CBL as seen through Students’ Eyes