Sample Evaluation Report

For

FDP Design Activity on Flipped Classroom(FPC 001)

ANALYSIS OF ENGAGEMENT3

A. What was the gross engagement in the course? [Macro level]        3

B. What was the engagement pattern of individual LeDs, LbDs, LxIs, and LxTs? [Micro Level]        4

C. What is the transition pattern of engagement of learners across LeD, LbD, LxI, and LxT?        6

ANALYSIS OF LEARNING8

A. What is the gross learning in the course?[Macro]        8

B. What is the learning pattern across LbDs and Quizzes? [Micro]        9

C. What is the transition pattern of learning across LbDs and Quizzes? [Meso Level]        10

ANALYSIS OF PERCEPTION        10

What was the perception of learners about the course? [Macro]        10

REFLECTIONS ON ANALYSIS15

A. What do gross data about learning, perception and engagement tell about the course?        15

B. What does the analysis tell about the course experience?        15

C. What needs improvement in the current course offering?        15

WORKLOAD DISTRIBUTION       16

1. ANALYSIS OF ENGAGEMENT

A) What was the gross engagement in the course? [Macro level]

The MOODLE course FPC 001 on Flipped Classroom was designed as part of a blended course which was to be taken by participants of STTP (Short Term Training Program). There were around 9 - 10 participants who took the course as part of the STTP. Around 57 participants have also registered for the course which makes the total number of participants around 67. The course involved the following activities that were to be completed by participants during the duration of the course:

· LeDs on PI- 4 Nos

· LbDs on PI- 3 Nos

· Discussion on Flipped Classroom

· Reflection Quiz (RQ)

· Advancing Your Learning (Lxt) - 5 Videos and Resources

· Assimilation Quiz (AQ)

· Knowledge Quiz (KQ)

· Activity- Designing your own Flipped classroom

· Peer review of Flipped classroom

· Evaluation and Feedback

The activity completion report generated from the course Reports section shows the engagement of participants in the activities as shown in Table 1.[1] 

Table 1: Activity Completion Summary

Activity

Completion Status

LeDs

1

42

2

42

3

42

LbDs

1

67

2

50

3

67

Discussion on Flipped Classroom

42

RQ

42

LxTs

42

AQ

42

KQ

42

Design your own Flipped Classroom

17

Peer Review Flipped Classroom

17

B) What was the engagement pattern of individual LeDs, LbDs, LxIs, and LxTs? [Micro Level]

Looking into the performance and engagement of individuals in the course activities, following is the engagement pattern of selected participants in the course. Out of 12 assigned participants, 12 didn’t register for the course. One person didn’t do any activity. Here, three participants were selected for the analysis so that we have sample groups showing high engagement, medium engagement and low engagement pattern. Table 2 shows the level of engagement of the three sampled participants in the course.

Table 2: Participation of individual learners[2] in the course

Participant 1

Participant 4

Participant 9

LeDs

1

NO

Yes

Yes

2

NO

Yes

Yes

3

NO

Yes

Yes

LbDs

1

NO

Yes

Yes

2

NO

Yes

Yes

3

NO

Yes

Yes

Discussion on Flipped Classroom

NO

Yes

Yes

RQ

NO

Yes

Yes

LxTs

NO

Yes

Yes

AQ

NO

Yes

Yes

KQ

NO

Yes

Yes

Design your own PI

NO

Yes

NO

Peer Review PI

NO

Yes

NO

C) What is the transition pattern of engagement of learners across LeD, LbD, LxI, and LxT?

2. ANALYSIS OF LEARNING

A) What is the gross learning in the course?[Macro]

Analysis of the learning of the students is done by analysing the marks obtained by the participants in attempting various quizzes and activities. There were five types of quizzes and activities for the participants. The weighted marks for each of them is given below.

1. Learning by Doing (LbD) Activities- 3 Nos [20 marks]

2. Reflection Quiz                               [10 marks]

3. Assimilation Quiz                                 [20 marks]

4. Knowledge Quiz                                 [25 marks]

5. Peer Instruction Design Activity                [25 marks]

                        Total                                        100 marks

Table 3: Number of participants who attempted each activity and the average, marks and percentage they scored.

LbDs

RQ

AQ

KQ

Activity

Any

All

No of

persons attempted

8

7

9

9

2

9

2

Average Marks

13.96

7.85

14.74

20.28

22.25

13.22

12.33

Maximum Marks

20

10

20

25

25

100

100

Average Percentage

13.22

12.33

B) What is the learning pattern across LbDs and Quizzes? [Micro]

Table 4: Number of participants who have attempted each LbDs. The total is calculated for 20 as it was the weightage given to the LbDs in the whole course scoring.

Lbd No

LbD 1

LbD 2

LbD 3

Any of the LbDs

All of the  LbDs

No of persons attempted

8

6

8

8

6

Average Marks

4.68

8.16

3.375

16.215

11.35

Maximum Marks

5

10

5

20 (Weighted)

20 (Weighted)

Average Percentage

93

81

67

81

56

Table 6: Number of participants who submitted the flipped classroom design activity and the marks they have been given. The total is calculated for 25 as it was the weightage given to the Design activity in the whole course scoring.

Flipped Classroom Design Activity Submission

Flipped Classroom Design Activity peer review

Total Marks

No of persons attempted

2

25

Average Marks

78

22.25

Maximum Marks

80

25

Average Percentage

97

90

C) What is the transition pattern of learning across LbDs and Quizzes? [Meso[4]]

3. ANALYSIS OF PERCEPTION

1. What was the perception of learners about the course? [Macro]

We can analyse the overall perception of learners through their feedback in the three surveys[5]:

· Perception of Learning

· Perception of Engagement

· Perception of Usability

Perception of Learning

For each of the parameters to analyse perception of learning, we can see that the participants seem to have a good perception about the learning offered by the course.

Talking about the learning avenue offered by additional resources/reference materials, some of the learners are neutral about their learning potential. This suggests the need to have a fresh look at the materials and check if they really offer in-depth knowledge of the topic.

Fig 1la. Perception of Learning

Fig 1lb. Perception of Learning

Perception of Engagement

It is clearly evident that the perception of engagement has been mildly low. Observing each criteria, we can find that the participants are not easily able to get actively involved each time they access the course and don’t find the additional resources/reference materials engaging enough.

Fig 1ea. Perception of Engagement

Fig 1eb. Perception of Engagement

However one positive aspect are the quizzes/activities. From Fig 1eb it can be observed that although the opinion about engagement is diverse, a good number of learners find the quizzes/activities mildly engaging.

Perception of Usability

Here’s a visual summary of the survey responses of participants.

Fig 1ub. Perception of Usability

Fig 1uc. Perception of Usability

From the graph, we can analyse each of the 9 categories used to evaluate the usability and further drill down into the response of each participant. Going by the bar graphs, we can see that most of the participants are Mildly or Strongly agree about the usability of the course.

REFLECTIONS ON ANALYSIS

1. What do gross data about learning, perception and engagement tell about the course?

The gross data about engagement shows that the number of participants completing the activities/quizzes have decreased as the course progressed. Only 20% of the participants completed RCA. This indicates that the perceived learning value of the course needs improvement. From the participants' perception of the course, it can be observed that the pace of the course and consistency of course material design needs improvement.

2. What does the analysis tell about the course experience?

The analysis shows that the course was engaging enough and the levels of usability can be improved, although the levels of learning were not that bad.

 

3. What needs improvement in the current course offering?

The following sections need considerable improvement:

Engagement:•        Engagement in Discussion forums•        Activities and Quizzes – The questions in quizzes level can be reduced to reach no of participants.Usability•        Consistency in course material design•        Duration of the session by reducing textual additional resourcesLearning

Additional resources should be videos instead of textual resources.•        Reference materials/additional resources

4. WORKLOAD DISTRIBUTION

The team consisted of 4 participants. The workload is distributed among them as follows:[6]

1. Moturi Sreelatha: (Team leader)

1. Preparing the workload distribution

2. LeD 1,3;

3. LbD 1,3;

4. LXI ;

5. LxT 1,2,3,4;

6. Uploaded Led 1,2,3 and LbD 1,2,3 to Gnomio.

7. Survey report on Perception of Engagement, Learning and Usability

8. Entire FDP design report

9. Gnomio administration

10. Feedback form

11. Grade Book Setup

12. Progress bar setup

13. Reflection Quiz

14. Assimilation Quiz

15. Knowledge Quiz

2. Reshma Shinde

1. LeD 2.1,2.2;

2. LbD 2;

3. LXT 5

3. Hemakshi Chaudhari

No participation at all

4. Saranya A

No participation at all