Learning outcomes at IJSO-2019 - GP Globalize

Page | 1

LEARNING OUTCOMES AT IJSO-2019

GPG Journal of Science Education Volume 1 Issue 1 July-September 2020

Learning outcomes at IJSO-2019

E-ISSN No. 2582-7073

Hala Sultan Saif Al-EasaChemistry and Earth Sciences Department,

Qatar University, P. O. Box 2713, Doha Qatar

AbstractThe 16th International Junior Science Olympiad was held at Doha from 2-12 December 2019. Theanalyses of the test results indicates a strong correlation between the results of MCQ, theory andpractical exam and also a positive correlation in the performance of the students in all the threebranches of sciences (Biology, Chemistry and Physics). The study also discusses the difficultylevel of the test and the quality of the test items based on Discrimination Index calculation.

Keywords: Correlations, Difficulty Index, Discrimination Index, Jr. Science Olympiad

Hala Sultan Saif Al-EasaGPG Journal of Science Education

Vol 1 Issue 1 2020 : pp 1 - 13

1. INTRODUCTION

International Junior Science Olympiad (IJSO) is a sciencecompetition for students under the age of 16, held everyyear in the month of December. The 16th IJSO was heldat Doha, Qatar, by the Ministry of Education and HigherEducation.

Competition was held at Qatar National ConventionCenter from 2-12 December 2019. It consisted of 3separate tests as per the pattern of IJSO: the multiple-choice test (MCQ), Theory part (or the long questions)and the practical part. Practical part is a group eventwhere 3 students from each country form a group toconduct the practical test and each student gets the samemarks for the group event, to be added to their individualMCQ and Theory marks. Each country gets to send 2teams of 3 students each. The host country can sendadditional teams as “guest teams” and hence additionalgroups are allowed.

2. EVENT OF IJSO

The academic event constituted three tests. Multiplechoice test, worth 30 marks was held on the 4th ofDecember 2019, theory questions (long questions), worth30 marks, was held on the 6th of December 2019 and thepractical tests worth 40 marks was held on the 8th ofDecember, 2019. The tasks with their solutions can beseen in reference [1].

The tasks were developed by a team of experts from 18various institutions. The committees consisted of 55number of faculty who were responsible for generatingthe questions. The questions were generated in 23 numberof meetings where each question was discussed at lengthfor the correctness, accuracies, clarity in understandingand deciding if appropriate for the age group underconsideration. All questions covered as many topics ofIJSO syllabus as possible. Table 1 shows the MCQ andtheoretical questions number in relation to the IJSOsyllabus. The practical part, general science skills part

(Received 19 June 2020; Accepted 19 July 2020; Published 20 December 2020)

email: [email protected]

Page | 2

GPG JOURNAL OF SCIENCE EDUCATIONHala Sultan Saif Al-Easa


of the syllabus in addition the following criteria were taken into consideration:1. Apply safety regulations2. Acquire basic techniques for measuring3. Observation using all senses4. Identify and use basic laboratory equipment5. Collect data6. Do correct calculations and drawings

Table 1. MCQ and Theoretical Questions in relation to the IJSO Syllabus

Syllabus MCQ Questions Theoretical Questions 1. Particles, waves and matter 4, 9, 12, 13, 23 1, 2, 3, 5 2. Energy 1, 27 2, 3, 4 3. Interactions 2, 14, 15, 16, 17, 19, 20, 22, 24, 25,

26, 28, 30 1, 2, 4, 5

4. Structure, properties and functions 18, 28, 2, 3, 4, 5 5. Systems 3, 5, 8, 10, 11, 21, 29 1, 3, 4, 5 6. Development and Evolution 6, 7 1, 4

Table 2. Distribution of MCQ over the three subjects

Table 3. Distribution of the theoretical questions over the three subjects

Subject Question Number Biology 3, 6, 7, 8, 10, 11, 21, 27, 28, 29 Chemistry 4, 5, 12, 14, 15, 16, 17, 18, 20, 22 Physics 1, 2, 9, 13, 19, 23, 24, 25, 26, 30

Subject Question Number Biology 1 part 3, 2 part 3, 3 part 3, 4 part 3, 5 part 2 Chemistry 1 part 1, 2 part 1, 3 part 1, 4 part 2, 5 part 1 Physics 1 part 2, 2 part 2, 3 part 2, 4 part 1, 5 part 3

The theme of the Olympiad was related to Qatar, but thequestions did not require detailed local knowledge. Whichmeant that the level of questions had to be sufficient toallow differentiation between students in different tests.The main intention was to test not only the students’knowledge, but also their understanding of scientificproblems and their ability to solve them. In addition, theexperimental skills were to be tested in the practicaltests.

The theory questions were tested on students ofbackground in science and with high grade point average(GPA) and practicals were tried out by members of the

Scientific committee for their accuracy of proceduresetc. It is important to highlight that the three scientificdisciplines were integrated into each question, which isclear in the open-ended questions.

In the 16th IJSO, 68 countries participated with team ofstudents (6 each) out of which 13 were guest teams. Thelist of the countries is presented in Table 1. The guestteams are indicated by an *, where these teams were notregular participants but invited for the first time by thehost country.

Page | 3



Figure 1. Correlations between physics, chemistry, and biology MCQs

3. DATA AND ANALYSIS

A total of 409 students participated in the event (including2 absent) and as per the statues of IJSO [1], 32 studentsfrom 12 countries received gold medals, 64 studentsreceived silver medals and 97 students received bronzemedals. The highest scored student in all guest countrieswere awarded a medal as a sign of appreciation. The

medal distribution of IJSO-2019 is listed in Table 1.

3.1. Multiple choice questionsWhen marks of MCQ were correlated between the threebranches, (Biology, Chemistry and Physics), thecorrelations can be seen in Figure 1, with r value forphysics-chemistry being 0.77, for physics-biology being0.57 and for chemistry-biology being 0.63

Difficulty index for all the 38 questions (30 MCQs, 5 Long questions and 3 practical questions) are plotted inFigure 2 where index of 1.0 is a very easy question answered by every student and 0.0 indicating that no studentanswered it correctly. The difficulty index was calculated using equation (1) which was modified for MCQ withnegative markings as shown in equation (2) where it has a factor of 0.25 x N for a set up when only one optionis correct out of 4 options, and wrong answer is given a negative credit of -0.25. [3]

(r = 0.769)

10

8

6

4

2

0

-2

-4

MC

-Che

mis

try

-4 -2 0 2 4 6 8 10MC-Physics

(r = 0.569)

10

8

6

4

2

0

-2

-4

MC

-Bio

log

y

-4 -2 0 2 4 6 8 10MC-Physics

(r = 0.629)

10

8

6

4

2

0

-2

-4

MC

-Bio

log

y

-4 -2 0 2 4 6 8 10MC-Chemistry

j

j

Total of marks received by all studentsDiff-I = (1)

N × M

Total of marks received by all students+ 0.25 ×NDiff-I = (2)

N× M × 1.25

Page | 4



Figure 2. Difficulty Index for all 38 questions

Figure 3. Fraction of correct answer for each question

Figure 3 indicated that more students were able to answer most of the biology questions as compared to other twodisciplines, nevertheless that the difference was relatively small. The distribution of the MCQ marks as shown inthe below histogram (Figure 4) had a wide distribution all over the range of marks thus indicating that the MCQswere neither too easy nor too difficult. Some students received total less than zero in MCQ, because they belongedto some of the guest teams which were participating for the first time and were not familiar with the process ofselection and training students.

1.1

1.0

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0 10 20 30 40 50 60 70

Diff

-I

Q. No.

Fra

ctio

n C

orre

ct M

C t

est

0.5

0.4

0.3

0.2

0.1

0.0

SubjectC4

Bio Chem Physsubject

BioChem

Phys

Page | 5



Figure 4. Histogram of the total score in MCQ

Num

ber

of s

tude

nts

MCQ Marks

50

40

30

20

10

00 5 10 15 20 25 30

3.2. Theory questionsThe 5 theory questions were integrated in nature and each one of them had a physics, a chemistry, and a biologycomponent. Marks for physics from each question were tabled separately and the same was done for chemistry andbiology. A correlation between physics, chemistry and biology marks can be seen in Figure 5.

Phy

sics

Chemistry

10

8

6

4

2

0

0 2 4 6 8 10

Page | 6



Figure 5. Correlation between biology, chemistry and physics marks in long questions

Figure 5 shows the correlation between biology-chemistry (r = 0.51), biology-physics (r = 0.54) and chemistry-physics (r = 0.85); where r is Pearson’s coefficients [2]. It also indicated that many students who scored reasonablywell in biology performed quite badly in physics. This is the reason IJSO has been striving hard to integrate thesciences to remove anti-correlation between the three branches of sciences.

Bio

logy

Chemistry

10

8

6

4

2

0

0 2 4 6 8 10

Bio

logy

Physics

10

8

6

4

2

0

0 2 4 6 8 10

Page | 7



Figure 6. Average marks for each of biology, chemistry and physics for each of the 5 long questions

Figure 6 depicts the average marks received for physics, chemistry, and biology component of each for the longquestion. It is interesting to note that in 4 out of 5 questions, students have scored higher in biology sections andlowest in physics. The situation is however, reversed in question number 5. The detailed analysis of the possiblereasons for such performance is under study and scope of further analysis.

The distribution of the marks received in the long questions is plotted in the below histogram (Figure 7). Thedistribution is skewed towards the lower marks indicating a more challenging set of questions as compared toMultiple choice questions in Figure 4.

Figure 7. Histogram of marks received in the long questions

Num

ber

of s

tude

nts

Theory marks

80

70

60

50

40

30

20

10

00 5 10 15 20 25 30

Q1-Bio

Q1-Phys

Q1-Chem

Q1-Total

Q2-Phys

Q2-Chem

Q2-Bio

Q2-Total

Q3-Phys

Q3-Chem

Q3-Bio

Q3-Total

Q4-Phys

Q4-Chem

Q4-Bio

Q4-Total

Q5-Phys

Q5-Chem

Q5-Bio

Q5-Total

70

60

50

40

30

20

Ave

rage

Sco

re P

erde

nt

Page | 8



Figure 8. Correlation between physics, chemistry and biology components of experimental questions

3.3. Practical TestsThe practical Test had 3 questions totaling 40 marks. Each of the 3 tasks had integrated questions and thecorrelation between the physics, chemistry and biology can be seen in Figure 8.

(r = 0.494)

14

12

10

8

6

4

2

0

Pra

ctic

al B

iolo

gy

0 2 4 6 8 10 12 14 16

Practical Chemistry

(r = 0.648)

Pra

ctic

al P

hysi

cs

Practical Chemistry

(r = 0.548)

Pra

ctic

al B

iolo

gy

Practical Physics

14

12

10

8

6

4

2

00 2 4 6 8 10 12 14 16

14

12

10

8

6

4

2

00 2 4 6 8 10 12 14 16

Page | 9



The correlation between biology-chemistry (r = 0.49), and biology-physics (r = 0.55) is relatively poor comparedto correlation between chemistry-physics (r = 0.65). The data in Figure 8 represents teams not individual studentsand the work was left for the team to arrange work between them which is not reflected here. The overalldistribution of marks of the practical tests can be seen in Figure 9.

The way experimental questions are designed in IJSO, recording observations in experiments carries certainweightage. Since students are always enthusiastic about carrying out experiments (experience of hands-on activity),it is generally observed that this distribution is skewed towards higher marks and hence the peak of the distributionis not mid-way but more towards 33 marks (out of maximum of 40). Bearing in mind that in such competitionnot only knowledge is measured but also skills and precision.

3.4. Correlation between the three tests formatsCorrelation between different branches of sciences gives an insight on the aspect of interaction. The interactionbetween MCQ, long question and practical can be seen in Figure 10. This correlation is carried out on data whereall the teams attending for the first time are removed, which varies the correlation slightly.

Figure 10. Correlation between MCQ- Long questions and MCQ- practical’s

Num

ber

of s

tude

nts

80

60

40

20

0

Figure 9. Distribution of experimental marks

Practical marks

0 5 10 15 20 25 30 35 40

30

25

20

15

10

5

0

-5 0 5 10 15 20 25 30 35

MCQ

The

ory

30

25

20

15

10

5

0

-5 0 5 10 15 20 25 30 35

MCQ

Pra

ct

Page | 10



Figure 11. Correlation between theory questions and practical

The correlation between MCQ and long questions was extremely high with Pearson’s coefficient value of 0.86 forthe correlation between MCQ and theory and the r value was 0.70 for the correlation between MCQ and practical.From Figure 10, it can also be concluded that students scored little higher in MCQ as compared to theory test.Figure 11 shows the correlation between theory and practical where the Pearson’s coefficient was found to be 0.68.

3.5. Discrimination IndexThe Difficulty index and Discrimination Index were calculated for all the 38 questions. These include 38 MCQ,5 theory and 3 Practical tests. The Difficulty index for each of the question was plotted in Figure 2.

The Discrimination Index (DI) for each of the question is plotted in Figure 11.

Figure 12. Discrimination Index values for all 38 questions

40

35

30

25

20

15

10

5

0

Pra

ct

0 5 10 15 20 25 30

Theory

DI

Q No

0 10 20 30 40 50 60 70

1.0

0.8

0.6

0.4

0.2

0.0

-0.2

......Min DI Di......Max DI

Page | 11



As per established literature, DI of value > 0.40 is an excellent question,0.30 <DI < 0.39 is a good question,0.20< DI < 0.29 is an average/acceptable question whereas DI < 0.19 is a poor question and should be investigatedfor the sources of low value of DI [2].

The DI values calculated for the present data are given in Figure 11. The DI values were calculated using themethod described in reference [3]. The dotted lines indicate the max and minimum value of DI that is possible foreach of the questions, whereas the average value is indicated with a tolerance level for each question. Themaximum and minimum values of DI result from a subgroup of 38-63 questions.

Figure 13. Summary of correlations between biology, chemistry and biology, in all three forms of test

The DI values are calculated using modified scheme as defined in [3], which handles MCQ with their negativemarking and the long questions scores scaled down to 1 mark. Hence the criteria for deciding the quality ofquestions, based on DI values stated in reference [2], is retained for this work.

4. SUMMARY

From the data in previous sections the overall correlation between physics- chemistry and biology sections of eachof the questions can be summarized in Figure 13. In this event in 2019, the student group presented highercorrelation between chemistry and physics in all the three different forms of competition. However, interestingly,from Figure 14, students scored higher in Biology.

Page | 12



Figure 14. Average grades in all the three different formats of competition

The correlation between the three different formats ofcompetition is particularly good. Students who performedwell in the IJSO 2019 performed equally well in allaspects of questions, thus indicating their preparednessin handling the integrated questions, which is one of themain aims of IJSO.

Out of 38 questions, 30 questions come under “excellent”category, based on the Discrimination Index. FromFigure 2, it can be concluded that this group of studentscould not perform well in 7 out of these 38 questions.To understand and investigate these 7 or 8 questions(with some possible overlap) is the further scope of studyand is under progress.

Similarly, from Figure 2, around this group of studentsperformed very well in about 11 questions. It remains tobe investigated, if these questions were outright simpleor there were other reasons for the group to attemptsolutions to these questions with relative ease.

Figure 8 reflects not only knowledge of individual subjectbut the performance and acquisition of laboratory skills

for the team since not taking into account whichindividual student did the work in a specific subject.

5. ACKNOWLEDGEMENT

Special thanks to Dr. Temadher Khalifa S. Al-Maadeedthe Head of Mathematics, Statistics and PhysicsDepartment, College of Arts and Sciences for directingthe Statistical Consulting UNIT to do the statisticalanalyses and Prof. Ayman Bakleezi for running theanalyses. and. I am also grateful to Dr. P. K. Joshi thepresident of IJSO for his continuous support and guidanceduring the preparation of this paper especially in theparts for correlation between the three test formats andthe Discrimination Index.

I would also like to thank the following institutions forproviding the experts who helped in preparing the testsfor this competition: Ministry of Education and higherEducation, Al Resalah Secondary School, Um HakeemSecondary School, Noor Al Khaleej International Scholl,Al Bayan Secondary School, Qatar Science andTechnology School, Omar Bin Al Khattab Secondary

Page | 13



School, Mohammed bin Abdulwahab School, Jassim bin Hamad Secondary School, Qatar Secondary School, AliBin Jassim Secondary School, Mohammed Al Mana Secondary School, Omar bin Abdul Aziz Secondary School,Al Shaymaa Secondary School, Doha Secondary School, Qatar University, Weil Cornell University (Qatar) andTexas A& M Qatar University.

6. REFERENCES

1. https://www.ijsoweb.org/downloads

2. Ebel R.L., Frisbie, D.A. Essentials of EducationalMeasurement. 5th edition, Englewood Cliffs, NewJersey: Prentice-Hall Inc; 1991, p232.

3. Joshi P. K., Jain Y., Khunyakari R.K. and Basu S Anovel alternative to analyzing multiple choicequestions via discrimination index (2019).: https://arxiv.org/abs/1906.07941

Documents

Learning outcomes at IJSO-2019 - GP Globalize