Report about the formative and the summative evaluation … · Report about the formative and the summative evaluation of the ... summative evaluation of the project . ... We used

The project mascil has received funding from the European Union

Seventh Framework Programme (FP7/2013-2017) under grant agreement n° 320693.

Report about the formative and the summative evaluation of the project

2

The project mascil has received funding from the European Union Seventh Framework Programme (FP7/2013-2017) under grant agreement n° 320693.

Project Information Project no. 320693 Project acronym: mascil Start date of project: 01/01/2013 Duration: 48 months Project title:

mathematics and science for life

Dissemination level Thematic Priority: Science in Society Funding scheme: FP7

Information about the deliverable Deliverable N° 10.3 Due date of deliverable: Month 48 Actual submission date: 22/12/2016 Deliverable title:

Report about the formative and the summative evaluation of the project

Contact Information Coordinator: University of Education Freiburg, Prof. Dr. Katja Maaß Lead partner for this deliverable: Dr. Katrin Engeln and Elena Schäfer Website: www.mascil-project.eu

The mascil consortium grants the permission to distribute the licensed materials under the Creative Commons Attribution-Noncommercial-Share Alike license as described at https://creativecommons.org/licenses/by-nc-sa/4.0/

http://www.mascil-project.eu/

https://creativecommons.org/licenses/by-nc-sa/4.0/

3


Table of Contents

Executive Summary .................................................................................................. 4

Main Report .......................................................................................................... 6 1.

1.1 Inquiry-based learning and the world of work................................................... 6

1.2 Research question ........................................................................................... 8

1.4 Multiple case study ........................................................................................ 10

1.4.1 Teachers’ understanding of IBL and WoW ................................................. 10

1.4.2 Teaching: Implementation in daily teaching practice .................................. 15

1.4.3 CPD experience ......................................................................................... 22

1.5 Pre-post study ................................................................................................ 25

1.5.1 Sample ....................................................................................................... 26

1.5.2 Inquiry based learning and the world of work context ................................. 29

1.5.3 Teaching practice ....................................................................................... 37

1.5.4 Pre-post comparison .................................................................................. 39

Conclusions and Recommendations .................................................................. 49 2.

References ......................................................................................................... 52 3.

Appendix ............................................................................................................. 54 4.

4.1. Appendix 1: Item scale documentation ....................................................... 54

4


Executive Summary Background The European project mascil aims to promote a widespread use of inquiry-based learning (IBL) and the world of work (WoW) context in mathematics and science education through teacher professionalization. Within the project partners from thirteen different European countries are working together. Several activities are run in all participating countries to reach the aim of the project. Among these activities are continuous professional development (CPD) courses using a spiral model. Teachers work with colleagues in professional reflection followed by classroom experimentation. This document reports on the qualitative evaluation (task 2, work package 10) and the quantitative pre-post-study (task3, work package 10) of those national CPD courses. Aims and purpose Within the work of the project, work package 10 (WP 10) aims to evaluate the impact of the mascil CPD courses. We want to find out what the impact of our overall PD concept is and what the reasons are. In particular we want to find out whether attitude, perceived hindrance and the implementation of IBL and the WoW context have changed after taking part in a mascil CPD course. Methodology and research design We combined mixed methods of inquiry to evaluate the impact the mascil CPD courses. We used a quantitative pre-post study and a qualitative multiple case study. Therefore we are able to check insight from one form of inquiry against insight from an alternative form. The pre-post-questionnaires study gives insights on a large scale and is complemented by the multiple case study giving deeper insight. For the purpose of the studies the pre- and post-questionnaires, interview guides and observation have been commonly used be all partners of the mascil consortium. Summary and recommendations The studies show an overall positive attitude towards IBL and the WoW context in all countries of the mascil consortium. Nevertheless, neither IBL nor the WoW context is frequently used in daily teaching practice. Science teachers report to use more IBL and more WoW context than mathematics teachers. Both studies show that policy (curriculum) including assessment, time issues and missing materials are seen as a hindrance to implement IBL and the WoW context. Classroom management is not seen as hindrance. Data being collected for the multiple case study and for the pre-post study after the CPD intervention show that teachers very much value the CPD experience. Teachers experienced facets of IBL during the CPD themselves. We are able to confirm that

5


through the professional development courses the mascil project promotes a European-wide implementation of IBL and the WoW context. Our analysis of the data of the pre-post survey showed that teaching practices changed significantly. This is approved by the multiple case study. After the CPD intervention teachers use IBL and the WoW context significantly more frequent. Teaching practice is more relevant, student-oriented, hands-on and investigative and more WoW oriented. Teachers have even a more positive attitude towards IBL and the WoW context. Policy including assessment is still seen a hindrance but not as strong as before the CPD intervention. There is an urgent need for coherence in policy rhetoric, curriculum, assessment practice and teacher education. IBL needs to be anchored visible in the curriculum and also in school books and further teaching materials. Assessment practice needs to be aligned to the curriculum. Furthermore, there is a need to value the WoW context. Policy needs to value the connections between schooling and world of work. Teachers need to be familiar with the WoW context After the CPD intervention teachers do not see missing material any longer as a hindrance to implement IBL and the WoW context. The project mascil is very successful in providing such material. The mascil material has proved to be very useful for teaching in the different European countries. CPD is strongly cultural dependent. In many countries CPD is not compulsory and CPD is not accredited. Therefore, teachers do not feel motivated to participate because of their daily work load. Especially, teachers are not used to longer lasting CPD courses. Mostly, teachers who already know about IBL and the WoW context attend the courses. Having accredited courses makes it easier to attract teachers and to implement longer lasting courses. Therefore, CPD practice in general has to be rethought. CPD should be a national policy priority to make sure that all teachers are well prepared to teach for the 21st century. We need to implement longer lasting CPD courses attracting all teachers, requiring significant and sustained investment in continuous professional development. These CPD course should support the exchange of teachers to establish learning communities.

6


Main Report 1.

The aim of the project mascil is to promote a widespread use of inquiry-based learning (IBL) in mathematics and science education. Additionally, mascil intends to establish a connection between mathematics and science education and the world of work (WoW). The aim of WP10 is to conduct and to coordinate an evaluation within the project. A status-quo survey on the beliefs and experiences of teachers throughout Europe regarding the implementation of inquiry-based learning, especially with a focus on rich vocational contexts has been conducted and discussed with the consortium (D10.2). In the following report we summarize the quantitative case studies which have been conducted in all participating countries following a common framework. We look for similarities and for aspects which can be generalized regarding the different CPD courses within the different national setting of the participating countries. Furthermore, we present the result of the quantitative pre-post study. The aim of both studies is to evaluate the implementation of mascil and also to identify supports and hindrances for implementing IBL. In part 2 (Conclusions and Recommendations) the results of both studies, the qualitative case study and the quantitative pre-post study, are brought together.

1.1 Inquiry-based learning and the world of work

Engaging students in inquiry-based learning (IBL) is nowadays regarded as a means of improving education especially science and mathematics education on a global level (Abd-el-Khalick et al., 2004). In the European context, reports by expert groups have expressed consensus that the uptake of science and technology related studies and professions by young people is not sufficient to keep up the pace of innovation and to react adequately to the economical, ecological and social challenges of a rapidly changing world. They assert the necessity of a renewed pedagogy in school that transforms the traditional (mainly deductive) teaching styles towards more appealing and cognitively activating forms of learning. Inquiry based education is identified as the method of choice to increase students’ interest and achievement in science (Dorier & Garcia, 2013; Gago, 2004; Hazelkorn et al., 2015; Kikis-Papadakis, 2013; Osborne & Dillon, 2008). As a consequence, a variety of new educational programs and projects to improve the quality of science and mathematics education were created on a regional, national and international level. Many of those focus on fostering inquiry based approaches in mathematics and science education. Even though inquiry-based learning (IBL) is widely accepted as a means of improving education there is still no common understanding of IBL. Artigue and Blomhoej (2013) analyse the concept of inquiry-based mathematics education (IBME) in relation to well-established theoretical frameworks in mathematics education and conclude “.:.there is

7


no doubt that IBME is likely to take a diversity of forms, according to the institutional conditions and constraints where it develops (p.809)”. Based on the work done in the project PRIMAS (PRIMAS, 2010) (funded by the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 244380) the following definition is used within the project Mascil:

“Inquiry-based learning aims to develop the inquiring minds and attitudes that are required to cope with an uncertain future. Fundamentally, IBL is based on students adopting an active, questioning approach. Students inquire and pose questions, explore and evaluate, and the problems they address are relevant to them. Learning is driven by open questions and multiple solution strategies. Teachers are proactive, supporting struggling students and extending those that are succeeding through the use of carefully chosen strategic questions. They value students’ contributions, including their mistakes, and scaffold learning using students’ reasoning and experience. In the classroom there is a shared sense of purpose and ownership.”

These aspects are summarized in the mascil framework. We identified four core elements of IBL: student orientation, relevance, hands-on activities and investigations. In addition this framework also includes the world of work context. Resources for teaching and learning can refer to the world of work by using a specific workplace context or by giving students a task or role that reflects a workplace practice. The level of including the world of work can vary from an activity in a workplace to solving a textbook problem in school. Research findings show that students experience and understand the functionality, purpose and utility of disciplinary knowledge in the workplace (Ainley, Pratt, & Hansen, 2006; Dierdorp, Bakker, Eijkelhof, & Van Maanen, 2011). For this to happen however, it is important that tasks are carefully designed and fit the goals of the curriculum. In the context of work the use of science and mathematics emerges from the activities and tasks of the workplace (Hoyles, Noss, Kent, & Bakker, 2010). The use of rich vocational contexts asks a lot from teachers. They have to master contextual knowledge and skills as well as connecting content-context knowledge and skills. We do not want to suggest that every lesson should be cast in a vocational context, but the starting point for mascil is that these contexts are an important ingredient of good education.

8


Figure 1: mascil framework

1.2 Research question

The aim of the multiple case study and the pre-post study is to gain insight into effects of the implementation of the mascil CDP courses within the countries of the mascil consortium. Therefore we aim to look at the implementation of inquiry based learning (IBL) and the “world of work” (WoW) context in the thirteen countries involved in the project. We want to answer from the perspective of the teachers (pre-post-study and case studies) and the multipliers (only case studies) in the different European countries of the consortium following research questions:

• What are the personal backgrounds and the expectations of the participating teachers in different European countries?

• What is the status of IBL? What is the effect of the country /cultural background and of the subject on the status of IBL?

• What is the status of WoW? What is the effect of the country /cultural background and of the subject on the status of WoW?

• How is the political context for implementing IBL perceived by the teachers in the different countries?

9


• What factors are seen as a hindrance for the implementation of IBL?

• What factors are seen as a hindrance for the implementation of WoW?

• How did the mascil model work in the different European countries? Which aspects proved to be successful and why?

• To what extend do teachers install learning communities?

• How does the participating teacher evolve during the course of the PD (in relation to his beliefs about teaching, his intentions, his teaching)?

1.3 Instrument and Method

We combined mixed methods of inquiry to evaluate the impact the mascil CPD courses. We used a quantitative pre-post study and a qualitative multiple case study. Therefore we are able to check insight from one form of inquiry against insight from an alternative form (Raudenbush, 2005). For the purpose of the pre-post study, a teacher questionnaire has been designed. We adapted the questionnaire of the baseline study (Deliverable 10.1 and Deliverable 10.2).Constructing the questionnaires we tried to keep the questionnaire as short as possible for not deterring the teachers to take part in the survey and at the same time long enough to get rich data. On the basis of the PISA study (OECD 2009) four-point Likert-type items were used whenever suitable. These items do not allow teachers to opt for a neutral response. Based on the PISA study the used categories of the four-point scales reflect frequencies or agreement (never or hardly ever, in some lessons, in most lessons, in almost all lessons and strongly disagree, disagree, agree, strongly agree).The pre-questionnaire asks teacher for personal data, information about beliefs and practice in relation to implementing IBL and the WoW context, current teaching practice and their experiences with CPD. All but the last are also part of the post questionnaire. In addition, the post questionnaire contains questions about the use of mascil material and about the mascil CPD the teacher attended. An overview about characteristics of the items used in the teacher questionnaire is given in 4.1 Appendix 1: Item scale documentation. The pre-post study is conducted to gain insight into the effects of the mascil intervention. Teachers being engaged in mascil CPD courses in the different national contexts of the mascil consortium took part in the survey. They filled in the mascil pre-questionnaires at the beginning of the interventions and the post-questionnaire after the mascil intervention. Data collection for the case studies involves interviews with teachers and multipliers, PD observations and classroom observations, short teacher questionnaires as well as portfolios of evidence from the teachers and the multipliers. By means of these methods of data gathering, amongst others, we aim to receive feedback relating to the professional development courses, the mascil professional development toolkit as well

10


as the material collection. To be able to perform a multiple-case study interview guides and observation grids have been commonly used be all partners of the mascil consortium. The overarching concept of the case studies constitutes a common framework. But due to the different national settings all countries had the opportunity to adapt the general evaluation framework to their specific culture-related needs. A specific CPD course has been defined as a case of our multiple case studies. Within each case we focus on three participating teachers and the multiplier. This will give us an inside on the specific CPD course and on how teachers evolve in this CPD course. Members of the mascil consortium in all participating countries conducted a case study of national CPD course of their choice. A national report following provided guidelines was written. Within this report we consider these multiple cases (Stake, 2013). The multiple-case study interview guides, observation grids and the questionnaires have been translated by members of the mascil consortium. Austria and Germany and also Greece and Cyprus used the same versions.

1.4 Multiple case study

To provide insight into teachers’ evolvement during their participation in the mascil CPD courses we analysed each case study according to different dimensions. These included: teachers beliefs on IBL and the WoW context, teachers´ own teaching, the role and experiences of the multipliers and the impact of the PD courses. The results are based on case studies conducted during mascil PD courses in thirteen different European countries. Altogether, thirteen multipliers and 37 math and science teachers contributed their experiences and engagement to the mascil case studies. In the following, we present all results as an overview. It also important to mention that the following quotes are chosen exemplarily to illustrate, give example and verify the evaluation results.

1.4.1 Teachers’ understanding of IBL and WoW

Most of the teachers have worked with some kinds of IBL tasks before the CPD courses. Only for a few teachers IBL has been a completely new approach. Nevertheless, all teachers had a certain understanding of IBL. The main emphasis was placed differently:

“Inquiring, that you give them a problem and they have to think about it and explain it later on.” (Teacher from Germany) “Exploring the number of roots of a quadratic equation in relation to the sign of the discriminant, for instance, is considered as an inquiry-based task at the school level. This is not an open problem for me because you know from the beginning the steps you need to take. Open problems are the ones in which you need to explore from the

11


beginning which might be the potential directions and the cases for the possible solutions. This process involves experimentation and mistakes.” (Multiplier from Greece) “Students discussed with other students, answered open questions. They did experiment work in the groups. Students checked hypothesis held by the teacher. Students could use internet and each other.“ (Teacher from Lithuania) “The IBL principle lies in learner´s independent work, in discoveries, trials, considering collected data and results. Learner´s activity is the core, as well as observation, deducing statements and verifying them.” (Teacher from Czech Republic)

After participating in the mascil PD most teachers mentioned additional features of IBL according to the mascil framework they have been working on. A main outcome of the post interviews is that most of the teachers believe the IBL-concept to be very useful in assisting their students with needed competencies. The participating teachers developed an understanding that working IBL-oriented not only helps their students to acquire competencies in mathematics and science but also to acquire various other skills, e.g. formulating statements, interpreting solutions and results as well as dealing with mistakes.

“And I notice if you teach this way, the students are more competent in reading and to express themselves on their own, to interpret solutions and so on. Or to find mistakes because there are so many tasks, where some things are wrong and they have to find mistakes. And justify why it doesn’t work like that. And all those theories maths needs are permanently discussed this way.” (Teacher from Germany)

Furthermore, when working according to the IBL concepts students have to deal with different opinions. Understanding the perceptions of the other students, to vouch for their own opinion and to tolerate other opinions and approaches is sometimes very instructive for students’ further development. These are competencies apparently that support students´ development in all areas of life.

“[It is] to teach students how to look to the world with their own eyes.” (Teacher from Romania)

A large number of teachers are convinced that giving students the freedom of finding their own way to solve a problem will encourage them to discover the world in their own way and raise their self-confidence. This is apparently even more important considering low achievers in class.

“The approach of inquiry based learning is more interesting for students. Usually the question is: for what do we need this? Or the task is posed in a way that the signification is clear, in any case that’s the way tasks should be. And this way, that there is no false or true, or that the task is pointed this way, where the result is true or false, but there are different approaches possible and you can talk about those. This way, those with little self-confidence in maths feel animated to contribute. They gain security,

12


they find their place or they find their way to cope with tasks. They can feel competent even if they don’t find the right answer in the end but they notice what they can do. And notice too, that it is something they can do. Especially weak students are often able to establish a relationship with the meaning, the context. Because they have a very high competence to cope with their everyday life. And that can be very valuable for those tasks.” (Teacher from Germany)

Most of the teachers participating in the mascil CPD phrased such meta-outcomes of IBL. These outcomes go beyond the learning of specific subject contents and aim on sustainably established competencies students benefit in their future life. Therefore, students get a certain understanding of the reason why they work on a specific task in a lesson: to learn about the key element of a subject on the one hand and to develop and improve different methods and ways of learning (alone or together) which the can use also in other areas on the other hand. This in turn leads to a high motivation to work with IBL tasks also on the side of the students.

“When one can give reason for why we should learn this (…) so in my experience the outer motivation to learn increase. Because they understand that, hey, this I have to master in ten years so that I can manage my life, or I can get this or that job. (…) When one gets some outer motivation to learn something, then perhaps one will learn. When one learns, so often things become more fun, then it goes over to become inner motivation also.” (Teacher from Norway)

The connection of ´learning for school´ (subject contents and related needed competencies) and ‘learning for life´ (acquire competencies that are needed beyond classroom contexts) is for most teachers deeply anchored in their pedagogies and they value teaching in an inquiry-based way as it supports this basic connection.

“In my opinion, a good lesson is if learners are engaged, independently search, make inquiries, set hypotheses and verify them. I like learners working together and co-operate, as well as creative lessons – where learners think up and create something unique […]. Because in a group they learn to be tolerant and responsible.” (Teacher from Austria)

Through IBL students become involved in authentic practices and their learning results in flexible and adaptive competencies. They acquire these competencies and methods of problem solving in school to tackle the problems and tasks they need to solve in these school contexts, but they benefit from the acquired competencies also in out of school contexts as well as in their professional future. The vast majority of the teachers that took part in mascil CPD support that view, even stronger after participating in the CPD courses.

13


Teachers have an understanding of IBL before participating in the mascil PD. After their participation teachers adapted their understanding to the mascil framework. Teachers believe IBL to be important for developing students´ competencies for future life. Most teachers as well as multipliers have not made any experiences concerning the WoW in their life. In general, they went from school to university and after graduation back to school as a teacher.

“Working with WoW is completely different because usually you are not a specialist in the field, so you do not have the same safety in handling specific notions, motivations.” (Teacher Romania)

The lack of experiences is a hindering factor of implementing WoW tasks in class. Teachers have problems to translate something into practice to which they have no extensive and comprehensive background knowledge or experiences. They are uncertain what tasks with connections to the WoW should/must include to be WoW-authentic. (This is also a reason why they often prefer real life contexts more than WoW contexts.) With respect to the implementation of WoW some teachers recognize that they might be an impediment themselves.

“I think if you are more preoccupied with [scientific or mathematical] concepts, understandings and applications of concepts, then you don’t focus so much on WoW or such. I myself am like that […] it is possible that I am at a too low level […]. Well, I know that this is a good excuse.” (Teacher Norway)

As in this quote teachers often justify not connecting to the WoW by a lack of a clear understanding of what it means to connect to WoW and how to do it. Not being directly involved in the WoW makes it challenging for many teachers to develop appropriate tasks. For this reason, the vast majority teachers really appreciated taking part in the mascil PD course to get ideas and examples on how to clearly make connections to the WoW within class. Before participating in a mascil CPD the WoW context is a mainly unknown terrain for most teachers (and multipliers) One reason for this might be that they have never worked in the WoW outside school. Teachers, and to some extent multipliers too, have difficulties with grasping the expression ´world of work´. The attempts to define the term WoW within the sample of the case studies are very multi-faced and show different backgrounds and assumptions. To give an example of the range (interviews before mascil PD):

14


“WoW is the word of professions, occupations and positions (and also money), where the man reaches self-fulfilment, i.e. he brings sense to everything what was given to him as inherited, through educations and upbringing (as well as trying to make himself financially independent).” (Teacher from Czech Republic) “WoW is to find variety of methods how to involve students to make lessons more interesting. The things what you learn at school are used in variety of profession and students need to learn that.” (Teacher from Lithuania) “It is difficult to define the WoW, because it is actually wide, diverse and complex. Thinking from the perspective of education it could be interpreted as those practices that take place in professional contexts the most authentic as possible, and tackling with problems the most similar as possible to those that people working on these places have to solve.” (Teacher from Spain)

These quotes show very diverse views on an understanding of WoW. The first teacher connects the world of work highly to positions and financial well-being but also understands the WoW as a meaningful part of everyone´s life. The second teacher considers WoW to be a method which targets to learner´s preparation for future profession including how to solve possible problems, how to work efficiently, how to build his own career. WoW motivates students and also equips them with competencies which are needed for their future lives. The definition of the third teacher is more directed to the definition that is the basis for mascil and the mascil PD. From a pedagogical perspective, this teacher beliefs that the Wow could be interpreted as a collection of practices that are most similar and authentic to those happening in workplaces, tackling with problems most similar to the ones that people working on this contexts have to face. Nevertheless, also for the third teacher it is difficult to define WoW, because it is something diverse and complex. This belief often hinders a real implementation in classroom. Comparison of the pre- and post-interviews make it evident that the understanding of the WoW concept changed during the mascil PD. After successfully participating in the mascil PD teachers have a more clear understanding of the WoW and also are more aware of the necessity and benefits of including WoW in class. Before the Mascil PD quotes like the following were often made:

“Not all topics are equally easy to be associated to WoW. I try, but I find some topics are difficult to connect to WoW. […]Then the WoW-implementation was more like a ´happening´, rather than something that was done regularly.” (Teacher from Norway)

After the participation in the mascil PD statements like the following are predominat: “Every lesson has to be oriented to WoW. The teacher has to relate topics to professions and enterprises to show how to apply knowledge in real life. It can help students to choose professions in the future.” (Teacher from Lithuania)

15


The mascil CPD courses supported many teachers in understanding the WoW concept and its relevance in class. Many teachers have difficulties in understanding the “world of work” concept. The participation in mascil CPD apparently could resolve this to a certain degree. Most teachers see the relevance of WoW contexts in lessons.

1.4.2 Teaching: Implementation in daily teaching practice

In the following emphasis is put on the supports and impediments teachers experience when implementing IBL and WoW in their daily teaching practice. Implementation of WoW WoW context is understood as connecting contexts of maths and science to real life contextsl. Teachers do not differ between real life contexts and WoW contexts. Also, the lesson observations show that teacher often thought they connect their lessons to the WoW, but it were actually real life contexts they referred to.

“I understand that tasks are posed in a way they exist in the professional world, people having this profession have to solve those tasks. But of course, this is the ideal conception. And you can’t always do it like this in practice. That means, usually you have to simplify it a bit or so. Maybe you don’t have the models or the problem is too complex or something like it.” (Teacher from Germany)

The concept of WoW in relation to school can here be seen as implementing realistic tasks from professionals within class. This means that the students deal with tasks that are taken from the WoW. Teachers often try to dissolve the ambiguity of WoW versus real life contexts by directly visiting workplaces.

“I relate my subject with WoW. We are going to factories of glass, manure, and to waterworks. The aim of these visits is to introduce various jobs, professions and to show where chemistry is used in real life.” (Teacher from Lithuania)

In doing so, they do not only support students´ understanding of WoW but often also their own understanding, as generally teachers do not have workplace experiences (1.4.1). Often, teachers´ attitude towards WoW contexts in the classroom are more critical as they have high demands on authenticity of the problem situation and the context. Another way to integrate the WoW in classroom in an appropriate way is inviting experts to a lessons and work together with students on realistic tasks. Many teachers tell about

16


their positive experiences with WoW-experts in classroom and this way having very authentic and IBL-oriented lessons. Many teachers have difficulties to distinguish WoW contexts from real life contexts. Through mascil CPD teachers were motivated to visiting workplaces or invite experts. This supported the distinction between WoW contexts and real life contexts. Putting an emphasis on application-oriented tasks can be very meaningful as knowledge is directly applicable. However, the analysis of the case studies also showed: many teachers belief that for older students it is more meaningful to make connections to the WoW than for younger students.

I think it is exciting if one connects it [a task] to a job or where one will need it. There is plenty of math, for example, in an accounting job or something related to computers. I don’t think young students see it because they are not “in that world”, but I think later, when they get older they will find their way, and will see what will be needed to get there. […] The challenge is the age. […]They don’t see themselves to be there [in the working place] yet. They are so young. (Teacher from Norway)

One reason for this assumption is the belief that older students are more likely to have an idea of their possible future profession and see themselves in work life. Another teacher phrased the same problem as following:

“I think that the vocational reference is mostly interesting for older students, who always have questions in their minds: What will happen later? What will I do in future? What will I need later on from the stuff that I’m doing in school? I think, that’s really interesting class 7 or 8 onwards. You can do it beforehand, but there it’s really relevant. And then it’s certainly motivating for students, if they notice what we’re doing at the moment can be used later on. And this and that I need to be able to do to solve professional tasks later on. Yes, that should be very motivating.” (Multiplier from Germany)

Also pointing out the age problem, this teacher also indicates that making connections to the world of work within class is very motivating and meaningful for students – especially for older ones, e.g. the twelve to sixteen year old students. Apparently, a link between students´ age, the personal relevance of a task and their motivation is made by teachers.

If a mathematical problem appears, it can happen that I ask „Do you have an idea, where this can be needed?“ And the younger ones have different notions. And the older ones that had some internships already, they always go in the more vocational direction. But the younger ones, they use connections to their surroundings. It’s not necessarily vocational but domestic, with measuring jugs. You weigh something at home; you measure something. Or with scales, you weigh food. Or the mother weighs if

17


she bakes a cake. Such connections come up. Yes and sometimes, they have ideas because their granddad did some handicrafts. (Teacher from Germany)

Like this teachers, many teachers all over the mascil sample believe it is more sensible for younger students to make connections to real life contexts than to parts of the WoW. The way they see it, it is easier for younger students because they can establish a relationship more easily to real life contexts than to the WoW.

I connected the horticulture context to the life world of the students of taking care of a vegetable garden rather than to a workplace context, because I had the feeling this connected better. Not many of them will become a horticultural scientist. (Teacher Netherlands) I understand that connecting the lesson to the real world is very beneficial for students. It is also helpful for the teacher to make concepts more explicit to students. I am not so sure about the added value of connecting the lesson to the professional work. Such problems are too complex and, at least for the elementary school, it is more possible to confuse students rather than help them. (Teacher from Cyprus)

In accordance to this quote, teacher often phrased the concern that the connection to the WoW might be too difficult or even confusing for younger students as, apparently, they lack the reference to their own world. Teaches are afraid that there is no gain for student´s learning when doing this kind of tasks in class, even worse, they fear that younger students completely lose the connection and motivation to the subject due to tasks that are too far away from their daily life and experiences. For this reason, they prefer connections to real life contexts as WoW in particular for younger students. Teachers prefer real life contexts more than the WoW contexts, especially when teaching young students. Many mathematics teachers and some science teachers believe that it is easier to make connections to the WoW in sciences than in mathematics. Teachers believe that mathematics always follows clear rules, processes and steps. Science in comparison implies per se methods of experimentation. Therefore, they believe that science offers more possibilities for (in particular IBL-oriented tasks) with a connection to the WoW than maths.

“Maths emphasizes a thorough practicing of calculating processes and standard algorithms of task solving. On the contrary, in physics experiments can be made and principles discovered (actually rediscovered) with simple aids.” (Teacher from Czech Republic)

This belief is partly also associated with the assumption that students learned early on which methods and processes they have to use in particular in mathematics – as a

18


specific characteristic of mathematics - and therefore it is problematic to change these habits especially in mathematics.

“[Students] already learned in small steps what it means to set their own theory, and this is in mathematics more difficult [than in chemistry] since in mathematics the are socialised in first getting a given theory, then a prime example […] and then many analogously solvable problems.” (Teacher from Austria)

This also shows a very rigid understanding of mathematics. In this way, some teachers describe mathematics ´only´ as an auxiliary discipline for science to describe and postulate the results of experiments. Also:

“The connection [to the WoW] was much better for science teachers where the problem was not just the translation to mathematics. The problems were much more directly linked to workplace and the respective professionals.” (Multiplier from Greece)

Here, the level of applicability becomes evident. As on the one hand mathematics is still often understood as an uncreative discipline in which experimentations are not possible teachers on the other hand see connections to the WoW more directly linked in science as the predominantly disciplines of experimentation. A multiplier from Greece gave the following example:

“The teachers in my group planned to develop an activity for the construction of a bridge. The question for the students was to find the exact point in which the bridge should be build. A landscape engineer who was invited in school informed us that in reality the professionals decide according to a multiplicity of factors related to the cost of the construction. If we were to take these factors into account the problem would have become too complicated for school students. Thus, it was not possible to achieve a real connection to the WoW at the level of school mathematics. It seemed that reality ‘flattened’ the problem.” (Multiplier from Greece)

The demands on a connection to reality (which is needed to include the WoW) are often seen as too complicated for a real application in school with regard to mathematics. Science teachers instead phrased often that they are more confident that their subjects can create this applicability to the WoW. They belief science to provide richer contexts and opportunities for making connections between classroom teaching and WoW:

“Science provides you with explanations of various phenomena in the world, which is important for numerous professions.” (Teacher from Czech Republic) “The situation was much better with physics. The problems were related to workplace and professionals in a much more direct way. The link to the workplace existed before and after modelling the problem.” (Multiplier from Greece)

The implementation of WoW contexts in science seems easier and more natural or directly linked for many teachers.

19


“If we lead learners to the cognition there were no cars, planes (and engines driving them), computers, TV, smartphones, they can easily understand how and why physics is important for various professions. I believe that connecting the instruction to WoW is a strong motivation which cannot be omitted.” (Teacher from Czech Republic)

Teachers think that connections to the WoW in sciences lessons are easier than in mathematics lessons. Implementation of WoW and IBL Teachers see missing WoW oriented tasks in school books as a hindrance to implement IBL.

“In mathematics I always think if it’s technical professions the normal lessons we do fits of course. Like I said, the best example is always the toolmaker, they do exactly those angles and trigonometry tasks we do, exactly the same they do in the vocational school again. […] Well I, we often do small roll plays, for example where they decant small liquids and measure and that results in a curve and such things. But I have no clue if they really need that as a nurse or if it is artificial from me. That is missing from the textbook.” (Teacher from Romania)

Apparently, teachers not only have the need for tasks relating to WoW in school books but also for some comprehensive background information about their authenticity, as teachers are not expert in these professions. Teachers see it difficult to obtain and to develop adequate WoW materials. Teachers would welcome the opportunity to affiliate WoW-related tasks to school books, e.g. listing certain competences which are needed for a certain profession and illustrating these requirements by listing a practically oriented task.

“It’s missing in the textbooks. Well, the mathematics books don’t always have a part with tasks if you want to become a nurse. There are tasks if you want to become an office clerk. It would probably interest the students to have such parts. This and that you need in the vocational school for office clerks, you need that for automotive mechatronics technician. But that’s not in the textbooks. Probably because it would be too much for a textbook. […] If you would start to bring tasks connected to professions, yes, but it would be nice of course. And so we have the problem that every teacher does a bit by themselves with more or less success.” (Teacher from Germany)

Mascil PD is seen as a supporting factor in overcoming the lack of comprehensive aspects regarding the WoW (as well as of IBL) in school books. Having a look on IBL oriented task, similarities regarding the use in school books could also be identified. The vast majority of teachers explained that IBL aspects are not used comprehensively in their school books. Teacher frequently indicated the need for explaining IBL aspects as well as IBL-tasks in school book as a basis for their lessons.

20


“There are not many of this type of tasks in the textbooks, at least not for grade 1-4. I am teaching mostly grade 1-4, not much there (…) for instance statistics. We recommend that we take the students out and count cars; or count people – how many walk, how many run, how many bike. […] In the textbooks so you have one page with a picture and you have to do statistics. […] I think (we should) put the book aside and think outside the book.” (Teacher from Norway) “Sometimes I have no materials for inquiry working. […] But if I decide to implement IBL I found solutions of these problems.“ (Teacher from Lithuania)

In both exemplary quotes the need for appropriate IBL materials becomes evident. Furthermore, another aspect is obvious. Teachers are willing to find solutions for the missing support of the textbooks. Regarding this, we could identify a change of attitudes between pre and post interviews that also is obvious in the two quotes. While in the pre interviews one the one hand the lack of background information of IBL aspects and on the other hand the lack of appropriate IBL tasks in school books was often given as a reason for difficulties in implementing IBL tasks in lessons, the point of view changed after the participation in the mascil PD. In the post interviews it became more evident that the teachers now had gained background knowledge and experiences in dealing with IBL in class so that they (still or even more) criticised the lack of comprehensive IBL aspects in school books, but now felt more secure in also finding alternatives to school books or even (re-) designing tasks in an IBL-oriented way. After the mascil PD many teachers valued the IBL concept and could refrain from the tasks used in school books.

“I think it [IBL] is useful and to work with the inquiry tasks is more valuable for the students than when they are doing exercises from their textbook.” (Teacher from Norway)

WoW as well as IBL aspects are not used comprehensively in school books. Teachers have a need for material. After the mascil CPD they need for material is not that strong anymore. Analysing the case studies, three different impediments were identified that hinder the implementation of IBL in classroom to the full extend intended by the mascil PD: time issues, the curricula, (final) exams/assessment, and school books. A large number of teachers complain that IBL tasks in classroom demand a lot of time which they do not have. This refers to the preparation as well as to the implementation in lessons.

"The biggest problem is the time.” (Teacher from the Czech Republic)

21


The time issue was criticised by at least one teacher in every case study (means in every participating country). Frequently, the teachers would like to invest time to include IBL more often in class, because for them the positive effects of IBL task are predominant.

“They [students] see something different compared to teachers, many times also to me. They perceive it in a different way. But it is demanding so that they discovered and correctly draw the inductive lines. Whereas 5 minutes was enough for the first approach, 30 minutes is needed for IBL. But it is worth doing.” (Multiplier from the Czech Republic)

The curriculum itself demands a lot of time and IBL as something ´additional´ would take time teachers need to teach the curriculum contents.

“The flip side of that [of the positives sides of IBL] is I can see the time pressures of having to teach x, y and z in so many lessons, which will then be tested on and we try and put in additional things and sometimes we have to take something out and you choose to take out what you can do to try and achieve that need.” (Teacher from the UK) “The main problem is that we do not have time enough to teach the curriculum […] the problem, as I mentioned before, is time.” (Teacher from Spain) “Curriculum time is precious, we have a lot to get through.” (Teacher from the Netherlands)

Also, in many countries IBL is not included in the curriculum at all or not to the extent the teachers wish for. In many countries a striking impediment to implement the WoW context is also the curriculum. In general, the curriculum is reflected in school books and assessments are oriented to the curriculum. Often, teachers see the structure of the curriculum as implemented in the textbook as a hindering factor to implement WoW tasks.

“It is not a must [to connect to WoW] in the curriculum.” (Teacher from Norway) “Especially in Holland, they follow the book.” (Teacher from the Netherlands) “World of work is largely something that isn’t ingrained in the curriculum. It is alluded to [when] students need to understand an application or a particular economic, environmental or social consequence but I don’t think it used as a vehicle to establish understanding….there are possibilities to do more…I know from personal experience that there have been times when I have used that as a tool for students to engage with a particular topic but up until this point we have only done that on an ad hoc teacher by teacher basis, it has not been part of a wider teaching picture, it has just been used as and when teachers feel it appropriate.” (Teacher from UK)

22


“Whatever I do in the lesson, I first of all think about what they have to know for the final exams.” (Teacher Netherlands) “With grade 9, 10 like I said, because we’re in this restriction with the exams, I have to push it through. To be honest I don’t want to experiment. I know that my existing system kind of works. Quite well. I incorporate those things. But I have to say that this is a quite good class. If they would be a weak one, which we have here too, then it would be even more difficult. Because then the students would come and say “Yes, we didn’t really understand the stuff for the exams. Why do we do something else now which we don’t really need for the exams?” And for that reason, it’s a fine line. That’s why I would say grad 9, 10 the students and of course the parents want that we prepare for the exams. And then inquiring is, because it’s not asked in exams, is second- or third rate, of course. That’s why I would say in the higher grades my teaching stayed the same but that I tried to incorporate different small projects. And in the lower grades I try to do more open, communicative tasks.” (Teacher from Germany) “I imagine it’s such a hard, difficult thing to formulate, isn’t it. You have to really train pupils. You know, develop that ability to think with an inquiry mindset, which takes time and effort and when you’re, sort of, stripped for time anyway with a busy school day, when you’ve got a curriculum to get through before the exams, it’s difficult to separate those lessons specifically to develop that inquiry way of thinking and it can just become a time pressure thing as you’ve got a lot to get through.” (Multiplier from the UK)

Summarising, it can be said that even if the vast amount of teachers advocate to include IBL tasks as well as WoW contexts in class, they see themselves apparently restricted by the curricula and in further consequence by the time, the support through materials in school books and the exams. Therefore the mascil concept often could not be implemented to the full extent even if the PD courses were considered as supportive in implementing IBL and WoW in class. Curriculum, assessment, time issues and missing materials are seen as a hindrance to implement IBL and the WoW context. Even after the CPD curriculum and assessment are still seen as a hindrance.

1.4.3 CPD experience

Teachers and multipliers valued the exchange with other teachers and the multiplier. The exchange was very supportive for implementing IBL and WoW contexts in class. Teachers got insights into the teaching of other teachers and were able to talk about challenges. Many teachers are in favour of an intensification of this exchange, for example through work shadowing, peer-support in school or a lasting learning community.

23


“[Discussing and collaborating with other colleagues] was one of the most important parts of the course. Besides, because other teachers from my school also attended the course, the debate and exchange of opinions has been transferred from the course to our school, and made it extensive to the rest of teachers within it, which are also adopting this way of teaching.” (Teacher from Lithuania) “Arne and Bjarne [the two multipliers] often said that you had to do this in your class (…) for instance connecting to the world of work or have some practical (…) they could have had brainstorming with me or given me suggestions. Even better if they could have come into my classroom and followed up, so in a way set a much higher expectation [on me].” (Teacher Norway) "I like to collaborate […] listening to other ideas is something very important for me […] my collaboration with [name of Teacher B] was something new for me. We developed together the task step by step, we tried to find the best way to introduce it to our students, we think together what type of manipulatives could be better to use [they used backgammon checkers as trees] during the implementation my colleague supported me in my teaching […] it was a nice experience.” (Teacher from Greece) “Yes, I would love to be part of the professional learning community. I am looking forward to the next mascil course.” (Teacher from Turkey)

The exchange between teacher and the multiplier during the PD courses was seen as supportive by the participants. The overall evaluation of the mascil PD courses was to a great extent very positive. One important topic was the satisfaction of most participants regarding the content.

“Making links with the WoW is also considered by her important in school arguing that "I am fascinated by the laws and the achievement of science. However, students quite often question the usefulness of what they learn. We have to answer them and the applications in the WoW are a good answer. […] I think that the aims of mascil could help in this direction.” (Teacher from Greece)

After the end of the mascil PD courses the vast majority of the multipliers and teachers indicated that they were very satisfied with the course concept itself as well as the effects of their participation.

“The PD courses have been extremely useful to me. There were things I felt I had to do but now they were in a more systematized form. When implementing them in a class setting I was passing the same (exploratory spirit) to them. These courses helped me make a connection with other like-minded teachers and I know that this is the way to work – as a part of a big family – colleagues with whom I work, we communicate, share what we have done, and who make you feel ‘great and significant’.” (Teacher from Bulgaria)

24


“Every time I get new ideas or refresh my knowledge and skills when participate in such PD course.“ (Teacher from Lituhania) “I feel know more excited and open to other teaching and learning processes.” (Teacher from Spain) “So, when mascil came on, I thought, here is precisely what I have been looking for, to get to share experiences, to get the engagement in the sciences seminars. Because there has been no structure before.” (Teacher from Norway)

Not only working together in a peer group of teachers was valued as described above, also the mascil PD course was helpful for teachers to better interpret in-class activities and processes. Furthermore teachers updated their knowledge of new trends and methods in the teaching of mathematics and science. Apparently the course also has often a positive impact on the self-perception as a teacher and the disposition towards different teaching approaches. Often teachers commented that the PD course also was good to provide on the one hand a theoretical basis of IBL and on the other gave them space to explore themselves how to meaningful and effective implement IBL in day to day teaching.

“Something that I was trying to do all these years in an unorganized and fragmental way turned into something structured and complete.” (Teacher from Greece)

Also all multipliers consider the PD course to be highly positive for the participating teachers.

“I think the participants were very satisfied, most of them considered the course useful, enriching their teaching practice, some of them became motivated to teaching again.” (Multiplier from the Czech Republic) “I have the impression that the teachers got something out of it. That, yes, all who took part in the process benefited. “ (Multiplier from Germany)

Having a look on the multipliers themselves it could be identified that they valued their work as a multiplier. They also benefit from their role as a multiplier to improve their own teaching as a teacher:

“This project is the one I have had the greatest benefit from since I started as a teacher. It is at least what changed my teaching most, it is. Or, developed could be more right to say.” (Multiplier from Norway) “I am extremely pleased with my role in mascil. […] I am very grateful for this opportunity.” (Multiplier from Bulagaria)

25


“I was attracted by the direct connection of IBL with WoW, it seemed to me new and very contributively. The work of mascil multiplier fulfilled my expectations in all aspects. […] This experience helps to improve my skills.“ (Multiplier from Lithuania)

To summarise, the mascil CPD course all over the participating countries had positive effects on the teachers’ teaching abilities according to the teachers own statements. Most often they felt supported by the course structure and materials as well as by the support of the other teachers and the multiplier. The same effects can be seen from the side of the multipliers who in turn were satisfied with their work as a mascil multiplier. Both, teacher and multipliers pointed out that they benefit from the course and the new experiences made regarding their own motivation, improvement of their skills and new impulses in teaching. Thereby, the participation in the course was positive not only regarding their day to day teaching practice but also on a personal level (e.g. motivation, self-confidence). Teachers as well as multipliers appreciated the mascil PD course (for example regarding contents, structures, achieved skills, support by others) and benefit from the experiences in daily teaching.

1.5 Pre-post study

All partners implemented CPD courses within their national context. The aim of all courses is to promote a widespread use of IBL and in particular to connect IBL with the WoW context. All CPD courses were delivered following the mascil framework. Nevertheless, due to different national contexts as reported in the deliverables of WP2 national adoptions were necessary. These adaptations did not concern main principles but rather the organisational framework of the professional development courses as reported in the deliverables of WP8. Following this short introduction, we will report the results from the collected teacher data. To begin with, we give an overview of the sample. Following this, we will analyse the pre teacher data, especially with respect to subject, age group and cultural differences. Finally, we will inform the reader about matching the pre- and the post- data and the results of paired-samples t-tests. The paired t-test is commonly used to compare a sample group’s scores before and after an intervention. In this case we test the effect of the professional development courses on beliefs of the teachers and their teaching.

26


1.5.1 Sample

In all thirteen participating CPD courses were selected to be part of the mascil pre-post study. The participating teachers in each country are not randomized; therefore, the sample is not representative for the whole country. We assume that the sample is representative for the teachers who are involved in the teacher professionalization offered by the project. Altogether the pre-questionnaire was filled in by 1420 teachers. The number of teachers varies between 49 (Netherlands) and 243 (Spain) within the countries. In all, 33.9% of the sample is male. The proportion of male teachers varies significantly between the countries. In Lithuania and Bulgaria, for example, only 11.3% and 10% of the sample are male while in Spain nearly half of the teachers is male. Significantly different gender distribution of teachers within different countries has also been detected by the OECD Talis 2013 study (OECD, 2014). This study focuses on lower secondary teachers of all subjects while our study included mathematics and science subject teachers of from primary to vocational level (Table 1). The average age of the sample is 41 years. Again, significant differences exist between the European countries. The average age of the Lithuanian sample is 49 while the average age of the Romanian sample is 34. The OECD Talis 2103 study identifies different age distribution between the participating countries as well. Comparing the mascil data with the Talis data it becomes evident that in most cases the average age is likely to be representative for the country in question. For example, Bulgarian teachers are among the oldest while in the United Kingdom are rather young. The Romanian teachers of the mascil sample are conspicuously young compared to the other countries of the mascil sample as well as to the TALIS 2013 Romanian data (Table 2). Teachers also have varying teaching experience in the different countries. This is expected as a result of the present age distribution (Figure 2). It becomes visible that teachers started their profession at different ages. The samples of the countries not only differ in the cultural background (country), gender and age distribution but also in the subject and in the age group taught most frequently. These are the learning groups the teachers bear on. Due to the aim of the mascil project only teachers referring to mathematics or a science related subject are in our sample. The science related subjects are Physics, Biology, Chemistry, Engineering, Natural Science, Geography, Technology are Information Technology. Teachers referring to one of the named subject are called science teachers for the purpose of our study. In the sample 55.2% of the teachers relate to mathematics while 37.6% relate to science (Percentage of mathematics and science teachers does not add to 100% because not all teachers opt for a specific subject). This proportion is country-specific. Within the samples of the different countries the proportion of mathematics teachers varies between 18.6% (Turkey) and 87.8% (Netherlands) (Table 3).

27


Furthermore, the teachers in the sample refer to different age groups when answering the questions on IBL, WoW and professional development of our questionnaire. 27% of the teachers of the sample refer to primary level, while 33.9% chose lower secondary level and 27.4% upper secondary level. Only 9.3% relate to vocational level. Again, conspicuous differences between the countries exist. For example, in Norway and Spain mainly teachers from primary and lower secondary level participated in our survey (more than 90%). In contrast, in the United Kingdom and in the Netherlands only about one third of the teacher refer to primary or lower secondary level. Gives an overview of the grades teachers refer to in the different countries There are differences between the teacher pre-samples of the countries. Especially, teachers have a different subject background and teach different age groups. There are also significant crucial differences regarding gender and age (teaching experience. These differences are partly cultural related but some of them are due to our specific mascil sample. Furthermore, the sample varies in the subject and the age group the teachers refer to. These differences are due to the country-specific mascil CPD and cannot be ascribed to the specific cultural background.

Table 1

Gender of teachers

country N male1 female female2 TALIS 2013

Austria 50 38.8% 61.2% Bulgaria 50 10.0% 90.0% 81.2% Cyprus 67 41.8% 58.2% 70.1% Czech Republic 61 39.3% 60.7% 76.5 Germany 64 17.7% 82.3% Greece 81 39.3% 60.5% Lithuania 161 11.3% 88.7% Netherlands 49 40.8% 59.2% 54.6% Norway 94 43.6% 56.4% 61.0% Romania 99 26.3% 73.7% 69.2% Spain 243 48.8% 51.2% 58.9 Turkey 228 35.6% 64.4% UK 173 34.9% 65.1% all 1420 33.9% 65.4% 1percentage of male and female teachers does not add to 100% because not all teachers filled in their gender. 2data is taken form OECD Talis 2013 Study(OECD, 2014, p. 258). This study focuses on lower secondary education teachers from all subjects.

28


Table 2

Age of teachers

country N1 age mean Standard deviation age mean TALIS 20132

Austria 43 42 11 43 Bulgaria 45 47 8 47 Cyprus 65 39 8 43 Czech Republic 59 41 11 44 Germany 62 39 13 old Greece 79 48 6 Lithuania 149 49 8 old Netherlands 48 44 11 43 Norway 93 41 9 44 Romania 98 34 7 42 Spain 230 43 9 46 Turkey 221 38 8 UK 168 35 12 39 all 1360 41 11 1Number of teachers who filled in the information about their age 2Data is taken form OECD Talis 2013 Study(OECD, 2014, p. 258). This study focuses on lower secondary education teachers from all subjects

Figure 2

Scatterplot of teaching experience versus age in the different countries (y=x-25 and y=x-30 are included to indicate different ages of beginning

29


Table 3

Gender of teachers

country Science Mathematics1

Austria 40.0% 44.0% Bulgaria 26.0% 74.0% Cyprus 40.3% 59.7% Czech Republic 70.5% 29.5% Germany 12.5% 82.8% Greece 37.0% 60.5% Lithuania 57.1% 28.6% Netherlands 12.2% 87.8% Norway 25.5% 70.2% Romania 32.3% 67.7% Spain 18.1% 73.7% Turkey 79.6% 18.6% UK 8.1% 69.9% All 37.6% 55.2% 1percentage of male and female teachers does not add to 100% because not all teachers filled in their gender. 2data is taken form OECD Talis 2013 Study(OECD, 2014, p. 258). This study focuses on lower secondary education teachers from all subjects.

1.5.2 Inquiry based learning and the world of work context

Implementation and attitude The questionnaire measured the implementation of IBL and the WoW in daily teaching practice (rou) from the perspective of the teacher. Furthermore, the teachers’ attitude towards IBL and the WoW of context (ori) are assessed. Each of the four mentioned scales is constructed with three items (5.1). Generally, neither IBL nor the WoW context is part of the daily teaching practice of the teachers. A one-way ANOVA was used to test for differences among the thirteen participating countries. Implementation of IBL and the WoW context differs significantly across the thirteen countries, F(12,1393)=15.23, p<0.01, F(12,1379)=15.52, p<0.01. Tamhane post-hoc comparisons of the twelve countries confirm that significant differences between the countries exist. For example, Czech teachers report significantly less implementation of IBL than teachers in most of the other countries. On the other hand, the Turkish teachers report significantly more implementation of IBL than teachers in most of the other countries. Romanian teachers implement the WoW context significantly less than teachers in all other countries. Additionally, Norwegian and Greek teachers also implement the WoW context less frequent than teachers in most other countries (Table 4, Figure 4).

30


Science teachers use IBL and the WoW context significantly more frequently than Mathematics teachers (Table 5). Moreover, vocational teachers implement significantly more IBL and WoW context than primary and lower and upper secondary teachers. Interestingly, upper secondary teachers implement significantly less IBL than primary teachers and significantly less WoW than lower secondary teachers (Figure 5). Furthermore, results indicate a significant higher implementation of IBL (M=2.26, SD=.64) than WoW (M=2.15, SD=.61), t(1271) = 6.13, p < .001. All teachers participating in mascil are positively oriented towards IBL and WoW. A one-way ANOVA was used to test for differences among the thirteen participating countries. Implementation of IBL and the WoW context differs significantly across the thirteen countries, F(12,1394)=13.21, p<0.01, F(12,1383)=7.79, p<0.01. Tamhane post-hoc comparisons of the twelve countries confirm that significant differences between the countries exist. For example, Czech, Lithuanian, Norwegian, and British teachers have are significantly less positive attitude towards IBL and the WoW context than Austrian, and Romanian teacher. Science and Mathematics teacher have the same positive attitude towards IBL and the WoW context (Table 5). Neither IBL nor the WoW context is frequently used in daily teaching practice. The implementation of IBL and the WoW context significantly depends on the cultural background (country), the subject and the age group. Teachers have a positive attitude about IBL and the WoW context. The attitude significantly depends on the cultural background (country) and the age group, but not on the subject. In all, IBL is implemented more than WoW context. Teachers also have a more positive attitude towards IBL than towards the WoW context.

31


Table 4

Implementation attitude towards inquiry-based-learning (IBL) and the world of work context (WOW)

country implentation attitude

IBL WOW IBL WOW Austria 2.55 2.41 3.45 3.44 Bulgaria 2.32 2.37 3.34 3.28 Cyprus 2.35 2.17 3.25 3.19 Czech Republic 1.94 2.12 3.12 3.08 Germany 2.43 2.03 3.52 3.23 Greece 2.42 1.99 3.23 3.12 Lithuania 2.13 2.33 3.03 3.08 Netherlands 2.02 2.06 3.19 3.10 Norway 2.26 2.01 3.17 3.14 Romania 2.10 1.61 3.42 3.35 Spain 2.01 2.34 3.23 3.27 Turkey 2.60 2.29 3.20 3.22 UK 2.28 2.27 3.01 2.99 all 2.26 2.19 3.21 3.18 Note: 1: strongly disagree, 2: disagree, 3: agree, 4: strongly agree

Figure 3

Scatterplot of implementation of IBL versus implementation of WoW context in the different countries (1: strongly disagree, 4: strongly agree).

32


Figure 4

Scatterplot of attitude towards IBL versus attitude towards WoW context in the different countries (1: strongly disagree, 4: strongly agree).

Table 5

Implementation of IBLand WoW Context, Means for Science and Mathematics teachers

Subject

Science Maths t df IBL 2.40 2.17 5.86*** 1216 (.64) (.64) WOW 2.23 2.07 4.67*** 1211 (.61) (.60) Note: *p<.05, ***p<.001.Standard Deviations appear in parentheses below means,Sample is weighted by country

33


Figure 5

Implementation of IBL and WoW for primary level, lower and upper secondary level and vocational level (data weighted by country).

Table 6

Attitude towards IBLand WoW Context, Means for Science and Mathematics teachers

Subject

Science Maths t df IBL 3.24 3.27 -.98 1216 (.42) (.42) WOW 3.21 3.19 .67 1209 (.42) (.47) Note:Standard Deviations appear in parentheses below means, Sample is weighted by country

Figure 6

Implementation of IBL and WoW for primary level, lower and upper secondary level and vocational level (data weighted by country).

34


Based on the literature (e.g. Colburn, 2006; Walker, 2007) and on our findings of the baseline study (Deliverable 10.2) the questionnaire included items regarding the situation to implement IBL and the WoW context. The data gives information about hindrances and support teachers face when implementing IBL. Three different factors are asked for

• Teachers view on the national system including the national curriculum with respect to the implementation IBL and the WoW context

• Teachers’ perception of classroom management issues regarding IBL and the WoW context

• Teachers’ need for supportive teaching material to implement IBL and the WoW context

The items are given in 4.1 Appendix 1: Item scale documentation Despite national political documents on a macro level teachers on the micro level do not feel supported by the political system to implement IBL and the WoW context. (Deliverable 2.1: National working papers on analysis of policy context). Significant differences exist between the countries. E.g. teachers taking part in mascil CPD courses in Norway and Lithuania feel more supported by the system than teachers in about half of the other countries. On the other hand, the Romanian teachers feel less supported than all the others (Figure 7). Classroom management issues are seen neither as a hindrance to IBL nor to implement the WoW contexts. Teachers participating in our CPD courses do not worry about classroom management. Again, significant differences between the countries exist (Figure 8). E.g. Austrian teachers taking part in the mascil CPD worry less about classroom management than teachers in the Czech Republic, Lithuania, Spain, Romania and the United Kingdom. Romanian teachers worry significantly more than teachers from all other participating countries. Moreover, teachers express a need for teaching material to implement IBL and the WoW context. Again, we find significant differences between the countries. Especially, the Romania teachers state a need for more teaching materials (Figure 9). Comparing IBL and the WoW teachers experience more policy support to implement IBL than to implement WoW. No significant differences exist regarding classroom management issues. The need for WoW context teaching materials is stronger than for IBL teaching materials (Table 7). The described support and hindrance factors are perceived nearly the same by Science and Mathematics teachers. Significant differences only exist for the need for IBL teaching materials. Mathematics teachers have a stronger need for IBL teaching materials than Science teachers (Table 8). Furthermore, the support and hindrance factors are influence by the grade the teacher teaches. Vocational and primary teachers perceive more political support than lower and upper secondary teachers to implement IBL and also to implement the WoW context. Depending on the grade taught no

35


differences exits regarding the ability of classroom management of IBL and the WoW context. Teachers of all age group report a similar need for teaching material except that vocational school teachers have more adequate teaching materials addressing the WoW context.

According to the perception of the teachers there is only limited policy support to implement IBL and the WoW context in their countries. Vocational and primary teachers report more support than lower and upper secondary teachers to implement IBL as well as the WoW context. All teachers feel able to handle classroom management when implementing IBL and the WoW context. All teachers do not feel well equipped with teaching materials on IBL and the WoW context. Figure 7

Scatterplot of policy support for IBL versus policy support for WoW context in the different countries (1: strongly disagree, 4: strongly agree).

36


Figure 8

Scatterplot of IBL classroom management skill versus WoW classroom management skill in the different countries (1: strongly disagree, 4: strongly agree).

Figure 9

Scatterplot of IBL teaching materials availability versus WoW teaching materials availability in the different countries (1: strongly disagree, 4: strongly agree).

37


Table 7

Support of policy system, classroom management competency, material Means for IBL and WoW context (depent-samples t test)

IBL WoW t df

SYS 2.23 2.10 10.51*** 1271 (.50) (.51) CLAS 2.70 2.70 .18 1246 (.54) (.55) MAT 2.00 1.96 2.46* 1251 (.54) (.54) Note: *p<.05, ***p<.001.Standard Deviations appear in parentheses below means, Sample is weighted by country

Table 8

Support of policy system, classroom management competency, material Means for Science and Mathematics teacher

Subject Science Maths t df

SYS_IBL 2.27 2.22 1.832 1223 (.51) (.50) SYS_WOW 2.11 2.08 1.093 1201 (.53) (.50) CLAS_IBL 2.71 2.70 .166 1218 (.55) (.53) CLAS_WOW 2.69 2.71 -.449 1174 (.55) (.53) MAT_IBL 2.05 1.96 2.778*** 1220 (.53) (.54) MAT_WOW 1.97 1.94 1.15 1179 (.53) (.53) Note: *p<.05, ***p<.001.Standard Deviations appear in parentheses below means, Sample is weighted by country

1.5.3 Teaching practice

From our perspective IBL is multi-faceted as summarized in the mascil framework (1.1). Inquiry-based learning is relevant, student-oriented, hands-on and investigative. To get insight into classroom practice teachers are asked about the frequency of these specific teaching practices. In addition, they are also asked about the frequency of making connection to the world of work context. The items are given in 4.1 Appendix 1: Item scale documentation. In general, teachers describe their teaching to be relevant to daily

38


live. Student-orientation is more visible than hands-on activities and the world of work context. In contrast, investigation is not implemented in daily teaching practice. As expected, all aspects of classroom practice vary depending on the country (Table 9). Furthermore, the frequency of all five facets of teaching practice significantly depends on the subject. Science teachers report more IBL-oriented teaching practice and also a greater implementation of the WOW-context (Table 10, see also 1.5.2). Also, the frequency of the five facets significantly depends on the age group. Secondary teachers report the least implementation of all facets of IBL and also of the WoW context. In contrast, vocational teachers report a significantly higher frequency of implementation of the IBL facets and especially, the WoW context (Figure 10, see also 1.5.2).

The different facets of IBL and the WoW context are implemented differently in daily teaching practice. While teaching is already relevant and student-oriented, hands-on activities, the WoW context and investigations are used infrequently. The implementation of the IBL facets and the WoW context significantly depends on the cultural background (country), the subject and the age group. Table 9

Teching practice: Mean1 of Relevance, Student-Orientation, Hands-On, Investigation and WoW depending on the country

country Relevance Student-Orientation

Hands-on Investigation WoW

Austria 2.86 2.39 2.23 1.83 2.40 Bulgaria 2.97 2.17 2.20 1.88 2.53 Cyprus 2.41 2.30 1.91 1.58 1.98 Czech Republic 3.00 2.46 2.31 1.97 2.53 Germany 2.65 2.45 2.31 1.83 1.84 Greece 2.43 2.56 2.01 1.58 1.88 Lithunania 2.63 2.46 2.19 1.93 2.36 Netherlands 2.19 1.98 1.81 1.39 1.86 Norway 2.58 2.45 2.20 1.69 2.01 Romania 2.93 2.71 2.42 2.03 1.80 Spain 2.79 2.30 2.13 1.60 2.19 Turkey 3.00 2.81 2.44 2.25 2.22 UK 2.64 2.64 2.30 1.75 2.43 ALL 2.74 2.50 2.23 1.83 2.18 11:never or hardly ever, 4: almost every lesson

39


Table 10

Teaching practice: Relevance, Student-Orientation, Hands-On, Investigation and WoW context, Means for Science and Mathematics teacher

Subject Science Maths t df

Relevance 2.90 2.57 8.676*** 1206 (.65) (.66) Student Orientation 2.50 2.38 3.202*** 1208 (.58) (.62) Hands-On 2.37 2.07 8.832*** 1209 (.55) (.57) Investigation 1.93 1.69 6.607*** 1207 (.63) (.59) WoW context 2.28 2.05 5.841*** 1205 (.66) (.68) Note: *p<.05, ***p<.001.Standard Deviations appear in parentheses below means, Sample is weighted by country Figure 10

Teaching practice: Relevance, Student-Orientation, Hands-On, Investigation and WoW context, Means for Primary Level, Lower Secondary Level, Upper Secondary Level, Vocational Level

1.5.4 Pre-post comparison

Sample The post questionnaire was filled in by 9911teachers. The return rate is therefore 70%. This rate is still acceptable, but the return rate varied significantly between the countries (Table 11). There are several reasons for the different return rates: Some teachers quitted the CPD course ore just missed the last meeting. Questionnaires which were 1 We also got 51 post questionnaires from Cyprus. These questionnaires were returned too late to be considered in this report. We will inform the Cyprus mascil team afterwards.

40


given to be taken home never returned. In some cases the teachers also had to fill in a national questionnaire and were not willing to fill in another questionnaire. Certificated CPD courses had a higher return rate. Probably, the whole course was taken more seriously. The teachers had different numbers of CPD meeting and also different hours of CPD meeting (Potari, 2016). Within the post questionnaire teachers were asked to what extent they experienced IBL with the CPD (5.1 Appendix 1: Item scale documentation). In all countries teachers experienced facets of IBL in the mascil CPD courses themselves. Country-specific differences exist (Figure 11). Especially, the Romanian teachers experienced IBL during the CPD themselves. Science teachers experience more IBL than Mathematics teachers during the CPD. Furthermore, teachers are also asked about their interest. These items include enjoyment, but also impact and the request to learn more (5.1 Appendix 1: Item scale documentation). In all countries teachers report the CPD courses to be of significance for them.

Then post questionnaire was filled in by 991 teachers. The return rate varies between the countries for different reasons. On the whole, the return rate is acceptable. All teachers experienced facets of IBL themselves during the CPD and valued the CPD experience.

Table 11

Number of matched pre and post questionnaires depending on the country

Country N pre N post Return rate Austria 50 17 34% Bulgaria 50 45 90% Cyprus 67 0 0% Czech Republic 61 57 93%

Germany 64 47 73% Greece 81 79 98% Lithunania 161 155 96% Netherlands 49 26 53% Norway 94 61 65% Romania 99 94 95% Spain 243 125 51% Turkey 228 193 85% UK 173 92 53% All 1420 991 70%

41


Figure 11

Boxplot2 of IBL experience during the mascil CPD depending on the country (1: strongly disagree, 4: strongly agree)

Figure 12

Boxplot of IBL experience during the mascil CPD depending on the country (1: strongly disagree, 4: strongly agree)

2 A boxplot shows the median and the interquartile range (IQR). The bottom and top of the box are the 25th and 75th percentile. The so-called whiskers show the data still within 1.5 IQR of the lower quartile, and the highest data still within 1.5 IQR of the upper quartile.

42


Material E-learning Only 29% of the sample used the e-learning platform. The number might even be lower. Looking at the implementation report we surmise that teachers counted using the mascil resources on the web-side as e-learning. On the other hand there also were e-learning mascil course in which the pre-post study was not conducted. Due to development and translation of the e-learning platform e-learning could only take place towards the end of the project when the pre-post survey was already done. Nevertheless, in some e-learning is not seen to be functional. Teachers prefer to have face-to-face meetings including IBL group work and are not used to e-learning. Teaching Material Most of the teachers got teaching material during the CPD course. The teaching material has been useful for the teachers. Over 90% of the teachers who got teaching material during the CPD used this material in class (Table 13, Table 16). Material collection At least 60% of the teachers visited a mascil material collection on the internet. The use of a mascil material collection varies between the countries. Again, the material is useful to the teachers who visited the mascil material collection. At least 75% of the teachers visiting the material collection used the material in class (Table 14, Table 16). Problem of the month Less than half of the teachers report to have discussed the Problem of the Month3. Again the material is useful. At least 70% of the teachers who discussed the Problem of the Month are able to use it in class.

Due to varied reasons e-learning is limited within the mascil project. The mascil project provides teaching material which is useful for the teachers.

3 The number might be higher. Discussion with the consortium showed that in some cases the Problem of the Month was used but not explicitly called “Problem of the Month”.

43


Table 12

Use of e-learning depending on the country

Use of e-learning Country Yes NO missing Austria 5.9% 94.1% Bulgaria 68.9% 8.9% 22.2% Czech Republic 33.3% 61.4% 5.3% Germany 8.5% 89.4% 2.1% Greece 10.1% 77.2% 12.7% Lithuania 30.3% 52.3% 17.4% Netherlands 0% 92.3% 7.7% Norway 14.8% 60.7% 24.6% Romania 4.3% 94.7% 1.1% Spain 52.0% 45.6% 2.4% Turkey 33.7% 51.8% 14.5% UK 4.3% 89.1% 6.5% all 25.9% 63.4% 10.7%

Table 13

Use of classroom material during CPD depending on the country

Classroom material during CPD

Country Yes NO missing Austria 100.0% 0% 0% Bulgaria 82.2% 0% 17.8% Czech Republic 96.5% 1.8% 1.8% Germany 89.4% 2.1% 8.5% Greece 43.0% 48.1% 8.9% Lithuania 82.6% 9.0% 8.4% Netherlands 92.3% 3.8% 3.8% Norway 55.7% 11.5% 32.8% Romania 100.0% 0% 0% Spain 76.8% 20.8% 2.4% Turkey 48.7% 34.2% 17.1% UK 44.6% 46.7% 8.7% all 70.2% 19.9% 9.9%

44


Table 14

Knowledge about mascil material collection at the internet

mascil material collection Country Yes NO missing Austria 41.2% 52.9% 5.9% Bulgaria 68.9% 11.1% 20.0% Czech Republic 57.9% 40.4% 1.8% Germany 53.2% 42.6% 4.3% Greece 69.6% 20.3% 10.1% Lithuania 80.6% 9.0% 10.3% Netherlands 11.5% 80.8% 7.7% Norway 47.5% 26.2% 26.2% Romania 100.0% 0% 0% Spain 44.8% 52.8% 2.4% Turkey 62.7% 26.4% 10.9% UK 10.9% 79.3% 9.8% all 59.4% 31.7% 8.9%

Table 15

Problem of the month

Problem of the month Country Yes NO missing Austria 35.3% 58.8% 5.9% Bulgaria 73.3% 6.7% 20.0% Czech Republic 61.4% 29.8% 8.8% Germany 2.1% 87.2% 10.6% Greece 17.7% 70.9% 11.4% Lithuania 78.1% 11.6% 10.3% Netherlands 3.8% 92.3% 3.8% Norway 39.3% 32.8% 27.9% Romania 100.0% 0% 0% Spain 10.4% 80.8% 8.8% Turkey 30.6% 58.0% 11.4% UK 12.0% 76.1% 12.0% all 41.6% 47.6% 10.8%

45


Table 16

Use of CPD material, material from the internet collection and of the PoM in class

Use in class

CPD material Mascil material collection POM N N N use without changes 156 22.4% 149 25.3% 104 25.2% use with changes 150 21.6% 145 24.6% 92 22.3% use with and without changes 269 38.6% 144 24.4% 99 24.0% no use 50 7.2% 86 14.6% 76 18.4% all 625 89.8% 524 89.0% 371 90.0% missing 71 10.2% 65 11.0% 41 10.0%

Pre-Post Comparison The pre- and the post-questionnaire include a code to match the pre and the post questionnaires. Due to this code we are able to match 763 pre-post questionnaires. Table 17 gives an overview of the matched sample4. The pre and the post questionnaire included identical items to look for changes between the answers before and after the CPD courses. Both questionnaires include the items on implementation of IBL and WoW context, attitude towards IBL and the WoW context, policy support to implement IBL and WoW context, classroom management issues regarding IBL and the WoW context and the need for teaching material on IBL and the WoW context (1.5.2). Furthermore, both questionnaires included the items on classroom practice regarding relevance, student-orientation, hands-on investigation, investigation and the World of Work context (1.5.3). The teachers are assessed at beginning and the end of the CPD course to find differences regarding these dimensions between the two occasions. A dependent t-test is used to test for significant differences between the two measuring points. The results indicate that teacher use more IBL and more WoW contexts after the CPD. Science teachers still use IBL and the WoW context more frequently than Mathematics teachers. Furthermore, the attitude towards IBL and the WoW context is significantly more positive after the CPD courses. This increase exists even though the attitude has been already positive at the beginning of the CPD courses (Table 19). Likewise, the results indicate that teachers view policy more supportive after the CPD. Also, the feel have more competent regarding classroom management issues. After the CPD teachers report a significantly higher satisfaction with the availability of classroom material. The effect size for the change of the perception of policy support and the ability of classroom management is small. However, the effect for the availability of

4 In some cases teachers refused to enter he necessary information: first two letters of your mother’s first name. In other cases this information is obviously wrong: Teachers enter the actual date instead of the date of their birthday.

46


classroom material is large. Before the CPD courses teachers expressed a need for materials to implement IBL and the WoW context. After the CPD intervention teachers do not see a need for material anymore (Table 19). Furthermore, teachers describe their own teaching practice to be more IBL-oriented after the CPD intervention. The average use of the identified facets of IBL is significant higher after the CPD. The average implementation of relevant contexts, student-orientation, hands-on and investigation is significantly more frequently after the CPD courses. In addition, the WoW context is also used more frequently (Table 20).

After the CPD intervention teachers use IBL and the WoW context significantly more frequent. Teaching practice is more relevant, student-oriented, hands-on and investigative and more WoW oriented. Teachers have even a more positive attitude towards IBL and the WoW context.

Table 17

Number of matched pre and post questionnaires depending on the country

Country N pre N post matches Austria 50 17 14 Bulgaria 50 45 33 Cyprus 67 0 0 Czech Republic 61 57 55 Germany 64 47 38 Greece 81 79 50 Lithuania 161 155 127 Netherlands 49 26 23 Norway 94 61 27 Romania 99 94 93 Spain 243 125 68 Turkey 228 193 153 UK 173 92 82 All 1420 991 763

47


Table 18

Implementation of IBL and the WoW context and attitude towards IBL and WoW context. Means for PRE and POST (dependent-samples t test)

PRE POST t df d

ROU_IBL 2.28 2.46 -9.04*** 746 .33 (.64) (.57) ROU_WOW 2.17 2.33 -6.98*** 738 .26 (.61) (.66) ORI__IBL 3.21 3.29 -5.57*** 747 .20 (.43) (.42) ORI_WOW 3.16 3.25 -5.12*** 739 .19 (.45) (.47) Note: *p<.05, ***p<.001.Standard Deviations appear in parentheses below means, Sample is weighted by country

Table 19

Support of policy system, classroom management competency, material, Means for IBL and WoW context (dependent-samples t test), Means for PRE and POST (dependent-samples t test)

PRE POST t df d

SYS_IBL 2.23 2.31 -5.112*** 752 .19 (.53) (.55) SYS_WOW 2.08 2.17 -6.142*** 737 .23 (.54) (.56) CLAS_IBL 2.62 2.75 -6.923*** 752 .25 (.55) (.51) CLAS_WOW 2.60 2.74 -7.876*** 717 .29 (.55) (.52) MAT_IBL 2.01 2.82 -24.254*** 753 .88 (.54) (.54) MAT_WOW 1.94 2.83 -27.444*** 722 1.02 (.53) (.50) Note: *p<.05, ***p<.001.Standard Deviations appear in parentheses below means, Sample is weighted by country

48


Table 20

Teaching practice: Relevance, Student-Orientation, Hands-On, Investigation and WoW context, (dependent-samples t test), Means for PRE and POST (dependent-samples t test)

PRE POST t df d

Relevance 2.78 2.88 -4.684*** 740 .17 .65564 .67103 Student Orientation 2.57 2.70 -6.952*** 742 .26 .60300 .57232 Hands-On 2.29 2.41 -7.082*** 743 .26 .60246 .59639 Investigation 1.93 2.09 -8.008*** 741 .29 .66 .65 WoW context 2.20 2.41 -9.798*** 737 .36 .67434 .65499 Note: *p<.05, ***p<.001.Standard Deviations appear in parentheses below means, Sample is weighted by country

49


Conclusions and Recommendations 2.

The European project mascil started with the objective to promote a widespread use of inquiry-based learning (IBL) and the world of work (WoW) context in mathematics and science through teacher professionalization. Within the project partners from thirteen different European countries worked together being on par with each other. CPD concepts and teaching material were developed and discussed. In all countries these courses were successfully offered for teachers among other resources and support measures. The multiple case study and the pre-post study showed that teacher appreciated the CPD courses and that the CPD courses had a positive impact on implementing IBL and the WoW context. The project mascil met its target to promote widespread use of IBL and the WoW context. The project mascil gave valuable insight into the teaching situation with respect to the implementation of IBL and the WoW context in the different countries of the consortium. The multiple case study and the pre-post study show, that neither IBL nor the WoW context is frequently used in daily teaching practice. The implementation of IBL and the WoW context significantly depends on the cultural background (country), the subject and the age group. Science teachers incorporate IBL and the WoW context more frequently than Math teachers. The connection between IBL, the WoW context and science seems to be more subject-inherent. In all, IBL is implemented more than WoW context. Both studies show, that teachers have a positive attitude about IBL and the WoW context. Teachers are convinced that IBL has the potential to overcome learning problems, to motivate students and to contribute to so-called 21st century skills like critical thinking, communication and collaboration. The attitude significantly depends on the cultural background (country) and the age group, but not on the subject. Teachers also have a more positive attitude towards IBL than towards the WoW context. In our multiple case study it becomes evident that teachers have limited experience with the WoW context. Both studies show that policy (curriculum) including assessment, time issues and missing materials are seen as a hindrance to implement IBL and the WoW context. Classroom management is not seen as hindrance. Data being collected for the multiple case study and for the pre-post study after the CPD intervention shows that teachers very much value the CPD experience. Teachers experienced facets of IBL during the CPD themselves. Within the multiple case study it is evident that teachers appreciate discussion and exchange with colleges. Teachers also valued the teaching material. Finally, the evaluation studies show that the professional development courses within mascil are held successfully. After the CPD intervention teachers use IBL and the WoW context significantly more frequent.

50


Teaching practice is more relevant, student-oriented, hands-on and investigative and more WoW oriented. Teachers have even a more positive attitude towards IBL and the WoW context. Policy including assessment is still seen a hindrance but not as strong as before the CPD intervention. After the CPD intervention teachers do not see missing material any longer as a hindrance to implement IBL and the WoW context. They feel competent to asses other teaching materials than the textbook and to redesign tasks. This affirms that the project mascil met its goal to closely link practice and theory. The model for professional development based on phases of analysis, implementation and reflection has proven to be successful. Policy including the curriculum and assessment practice is seen as a hindrance to implement IBL and the WoW context. Even though IBL is generally prioritized in mathematics and sciences policy agenda in the vast majority of the countries teachers experience policy as a hindrance to implement IBL. There is an urgent need for coherence in policy rhetoric, curriculum, assessment practice and teacher education. IBL needs to be anchored visible in the curriculum and also in school books and further teaching materials. Assessment practice needs to be aligned to the curriculum. Furthermore, there is a need to value the WoW context. Policy needs to value the connections between schooling and world of work. Teachers need to be familiar with the WoW context (see also WP2 Kikis-Papadakis, 2014). Company visits and contact with staff are helpful. Teachers have a need to for educational materials to help them in implementing IBL the WoW. The project mascil is very successful in providing such material. The mascil material has proved to be useful for teaching in the different European countries. The material collection has proven to be useful in different cultural contexts, for different age groups and for different subjects. We would recommend on the one hand further dissemination of the mascil material. On the other hand we also recommend further research on designing high quality teaching material for IBL and WoW context. Also research on the interplay between teaching materials, CPD experience and teacher learning is necessary. Furthermore, it is very important that teachers learn to redesign appropriate tasks to their needs. For further projects it is necessary to rethink e-learning and to plan it carefully. E-learning not only has to handle technical issues but also the attitude of the teachers. Teachers value the exchange with other teachers. Therefore we recommend carefully designed blended learning experiences if e-learning seems to be appropriate. CPD is strongly cultural dependent. In many countries CPD is not compulsory and CPD is not accredited. Therefore, teachers do not feel motivated to participate because of their daily work load. Especially, teachers are not used to longer lasting CPD courses.. Mostly, teachers who already know about IBL and the WoW context attend the courses. Having accredited courses makes it easier to attract teachers and to implement longer lasting courses. Therefore, CPD practice in general has to be rethought. CPD should be

51


a national policy priority to make sure that all teachers are well prepared to teach for the 21st century. We need to implement longer lasting CPD courses attracting all teachers, requiring significant and sustained investment in continuous professional development. These CPD course should support the exchange of teachers to establish learning communities.

52


References 3.

Abd-el-Khalick, F., Boujaoude, S., Duschl, R., Ledermann, N. G., Mamlok-Naaman, R., Hofstein, A., Tuan, H.-L. (2004). Inquiry in Science Education: International Perspectives. Science Education, 88(3), 398-419. Ainley, J., Pratt, D., & Hansen, A. (2006). Connecting engagement and focus in pedagogic task design. British Educational Research Journal, 32(1), 23-38. doi: 10.1080/01411920500401971. Artigue, M., & Blomhoej, M. (2013). Conceptualising inquiry based education in mathematics. ZDM, 45(6). Colburn, A. (2006). What teacher educators need to know about inquiry-based instruction. Paper presented at the Annual meeting of the Association for the Education of Teachers in Science Akron, OH. Dierdorp, A., Bakker, A., Eijkelhof, H. M. C., & Van Maanen, J. A. (2011). Authentic practices as contexts for learning to draw inferences beyond correlated data Mathematical Thinking and Learning, 13(1&2), 132-151., 13(1&2). Dorier, J., & Garcia, F. J. (2013). Challenges and opportunities for the implementation of inquiry-based learning in day-to-day teaching. ZDM, 45(6), 1-13. Gago, J. M. (2004). Europe needs More Scientists: Report by the High Level Group on Increasing Human Resources for Science and Technology. Brussels: European Commission. Hazelkorn, E., Ryan, C., Beernaert, Y., Constantinou, C. P., Deca, L., Grangeat, M., . . . Welzel-Breuer, M. (2015). Science for responsible Citizenship. EUR 26893 EN, European Commission. Hoyles, C., Noss, R., Kent, P., & Bakker, A. (2010). Improving mathematics at work: The need for techno-mathematical literacies. London: Routledge. Kikis-Papadakis, K. (2013). Deliverable N° 2.1: National working papers on analysis of policy context. mascil: mathematics and science for life. OECD. (2014). TALIS 2013 Results: An International Perspective on Teaching and Learning: OECD Publishing. Osborne, J., & Dillon, J. (2008). Science Education in Europe: Critical Reflections. A Report to the Nuffield Foundation. London: Nuffield Foundation. Potari, D. (2016). Deliverable 8.2: International report on implementation strategies and their effectiveness mascil project.

53


Raudenbush, S. W. (2005). Learning from attempts to improve schooling: The contribution of methodological diversity. Educational Researcher, 34(5), 25-31. Stake, R. E. (2013). Multiple case study analysis: Guilford Press. Walker, M. D. (2007). Teaching inquiry-based science - A guide for middle and high school teachers. LaVergne, TN: Lightning Source.

54


Appendix 4.

4.1. Appendix 1: Item scale documentation The following value will be reported for each scale used in the analysis.

𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a Cronbachs α N Mean of item Standard

derivation Correlation of item with scale

Cronbachs α, if item is deleted

Size of sample

When teaching this subject to this class, how often do the following activities occur in your lessons? rou_ibl: Implementation of IBL Item 𝒙𝒙� S 𝒓𝒓𝒊𝒊𝒊𝒊 a 13a I regularly do projects using IBL with my students. 2.30 .713 .675 .807 13e IBL is part of my daily teaching. 2.24 .733 .728 .757 13l I already use IBL a great deal. 2.21 .691 .714 .771 Cronbachs α=.841

N=1374

rou_wow: Implementation of WoW Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a 13b I regularly do projects using WoW with my students. 2.17 .664 .643 .765 13f Makin connection to the WoW is part of my daily teaching. 2.23 .715 .701 .704 13g I already use WoW a great deal. 2.18 .705 .650 .758 Cronbachs α=.813

N=1361

ori_ibl: Attitude towards IBL

Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a

13b I would like to implement more IBL practices in my lessons. 3,24 0,591 0,483 .568

13d Students benefit from IBL 3,10 0,630 0,442 .579

13f Implementing IBL makes my lessons more interesting. 3,19 0,618 0,581 .483

Cronbachs α=0,642 N=11372

55


ori_wow: Attitude towards WoW Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a 13d I would like to make more connections to the world of work.. 3.13 .571 .492 .850

13j Connecting to the world of work makes lessons more interesting. 3.24 .585 .714 .614

13k Students will benefit from making connections to the world of work. 3.27 .568 .691 .643

Cronbachs α=.787 N=1364

pol_ibl: Policy Support of IBL

Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a 11a Policy making prioritises IBL approaches in my country. 2.23 .683 .543 .703 11b There are curriculum support materials that support IBL

approaches. 2.40 .715 .505 .717

11e The curriculum encourages IBL. 2.28 .680 .573 .693 11h Assessment of students draws on and is based on notions of

IBL (i.e. IBL skills/ competences are assessed). 2.14 .751 .417 .751

11j National curricula prioritize IBL approaches in science and mathematics subjects in my country. 2.19 .678 .579 .690


pol_WoW: Policy Support of WoW

Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a 12a Policy making prioritises WoW approaches in my country. 2.10 .683 .622 .762 12b There are curriculum support materials that support WoW

approaches. 2.14 .667 .598 .769

12e The curriculum encourages WoW. 2.19 .647 .606 .767 12h Assessment of students draws on and is based on notions of

WoW (i.e. IBL skills/ competences are assessed). 1.98 .677 .525 .792

12j National curricula prioritize WoW approaches in science and mathematics subjects in my country. 2.08 .666 .618 .763


cla_ibl: Classroom Management of IBL Lessons

Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a 11a IBL is too difficult for many students. 2.37 .707 .357 .501 11h I think that IBL lessons are difficult to manage. 2.43 .736 .403 .432 11j I worry about student’ s discipline being more disruptive in

IBL lessons. 2.24 .773 .378 .470

All items have been inverted to calculate the value of cla_ibl Cronbachs α=.569 N=1369

56


cla_WoW: Classroom Management of WoW Lessons Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a

12k WoW is too difficult for many students. 2.42 .684 .359 .595 12m I think that WoW lessons are difficult to manage. 2.40 .715 .455 .463 12o I worry about student’ s discipline being more disruptive in

WoW lessons. 2.22 .761 .452 .466

All items have been inverted to calculate the value of cla_WoW. Cronbachs α=.612 N=1315

mat_ibl: Material for IBL Lessons

Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a 11i To implement IBL I would need more adequate materials.1 1.8830 .66735 .293 11p I have enough/adequate teaching materials to implement IBL

approaches. 2.1163 .66477 .293

1Item has been inverted. Cronbachs α=.463 N=1393

mat_WoW: Material for WoW Lessons

Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a 12i To implement IBL I would need more adequate materials.1 1.92 .619 .316 12p I have enough/adequate teaching materials to implement IBL

approaches. 2.00 .667 .316 1Item has been inverted. Cronbachs α=.480

N=1340

rel: Relevance Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a v14b I use this subject to help students understand the world

outside school. 2.64 .788 .295 .781

v14f I show the students how this subject is relevant to society. 2.73 .812 .471 .631 v14p I explain the relevance of this subject to our daily lives. 2.85 .817 .434 .680 Cronbachs α=.779

N=1359

57


sto: Student- orientation Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a v14d My students are given opportunities to explain their ideas. 3.00 .830 .536 .748 v14h My students have discussions about the topic. 2.62 .862 .603 .725 v14l My students have the possibility to try out their own ideas. 2.18 .817 .534 .749 v14m My students are involved in class debate or discussion, 2.72 .846 .651 .709 v14n My students choose which questions to do or which ideas to

discuss. 1.97 .812 .462 .771


hon: Hands-On Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a v14a My students do practical activities. 2.38 .755 .391 .664

v14e My students do experiments/simulations/modelling by following my instructions.

2.18 .782 .508 .511

V14j My students draw conclusions from experiments/simulations/ modelling they have conducted.

2.12 .776 .516 .500


inv: Investigation Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a v14c My students design their own experiments/investigation. 1.74 .735 .675 .716 v14g My students to experiments/simulations/modelling to test out

their own ideas. 1.95 .762 .642 .747

v14k My students have the chance to choose their own experiments/investigations.

1.78 .808 .649 .743


wow: World of Work Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a v14i I make connections to the world of work. 2.21 .810 .613 .712 v14o I use tasks that resemble the tasks of real workers. 1.95 .797 .621 .704 v14q I make connections to the possible future work of the

students. 2.39 .873 .627 .698


58


cpd_ibl: Method of CPD Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a v18ap We exchanged teaching material with colleagues. 3.23 .658 .620 .916 v18bp We discussed and decided with colleges on the selection of

instructional media. 3.22 .690 .711 .910

v18cp We were given opportunity to explain our ideas. 3.44 .568 .742 .908 v18dp We had discussions about topics 3.44 .540 .768 .907 v18ep We were involved in group discussion. 3.44 .560 .760 .907 v18fp We did practical acitvities. 3.37 .641 .727 .908 v18gp We did investigations/experiments to test out our ideas 3.19 .749 .697 .911 v18hp We designed our own investigations/experiments. 3.12 .774 .742 .908 v14ip We discuss our teaching practice. 3.39 .605 .714 .909 Cronbachs α=.991

N=931

cpd_int: Significance of CPD Item 𝒙𝒙� s 𝒓𝒓𝒊𝒊𝒊𝒊 a v13bp I enjoyed the CPD. 3.30 .587 .638 .751 v18cp The CPD made me change my teaching practise. 2.99 .667 .611 .767 v18cp I would like to have the opportunity to attend more CPDs. 3.20 .629 .664 .737 v18np I got new input for my teaching. 3.23 .551 .587 .775 Cronbachs α=.806

N=936

Documents

Report about the formative and the summative evaluation … · Report about the formative and the summative evaluation of the ... summative evaluation of the project . ... We used