School - eranet.co.za 2008_Rumep Summative report...5.2 Mr Adonis Hoho senior secondary school ... ACKNOWLEDGEMENT ... maths Teachers have poor content knowledge,

Confidential report for restricted circulation

SUMMATIVE EVALUATION © QPIE PAGE i

QPiE Quality Projects in

Education c.c. C.K. 987224523

RUMEP BEd PROJECT

Summative Evaluation March 2009

by

Quality Projects in Education Hobden,P. & Hobden,S.

commissioned by Zenex Foundation

CONTACT DETAILS Postal Address: 12 David McLean Dr. Westville 3630 Telephone: Home :031 2662216 Cell :082 5474031 Facsimile: 086 6574067 Email: [email protected] CONTACT PERSONS: Prof Paul Hobden, BSc, MEd, PhD Dr Sally Hobden, BSc, MEd, PhD

MISSION STATEMENT To contribute to the advancement of education and in particular science and

mathematics education through delivery of high quality products informed through experience in the field, awareness of research findings and

collaboration with participants.

mailto:[email protected]�


SUMMATIVE EVALUATION © QPIE PAGE ii

CONTENTS SECTION 1 : EXECUTIVE SUMMARY 1

1.1 Professional development........................................................................... 2

1.2 Mathematics content tests ......................................................................... 2

1.3 Attitude tests .............................................................................................. 3

1.4 Classroom environment.............................................................................. 4

1.5 Case studies ................................................................................................ 5

1.6 Commentary on the findings...................................................................... 6

SECTION 2 : THE INTERVENTION 9

2.1 Findings ...................................................................................................... 9

2.2 Evidence of an exemplary Programme...................................................... 9

SECTION 3 : TESTING OF GRADE 10 LEARNERS 16

3.1 Mathematics content tests ....................................................................... 16

SECTION 4: ATTITUDES AND SURVEYS 23

4.1 Attitude tests ............................................................................................ 23

4.2 Classroom environment survey ............................................................... 29

SECTION 5 : CASE STUDIES OF THE PROJECT TEACHER 37

5.2 Mr Adonis Hoho senior secondary school ............................................... 40

5.3 Mr Z Cafu Amajingqi Secondary High School ....................................... 44

5.4 Mr Gontshi Nozalisile secondary school.................................................. 50

5.5 Mr Mellet Templeton high school ............................................................ 55

5.6 Mrs Mntwini Ngangelizwe secondary school .......................................... 60

5.7 Mrs Ndarane Masizakhe secondary school ............................................. 65

5.8 Mr Ndzala Gobizembe senior secondary school/ Jonguhla high ........... 70

5.9 Mrs Nika Josi Marele high school ........................................................... 75

5.10 Ms Sanqela Ntabenkonyana high school................................................. 80

5.11 Mr Williams Elukhanyisweni ................................................................. 86

5.12 Mr Willlie Eyabantu ................................................................................ 91


SUMMATIVE EVALUATION © QPIE PAGE iii

SECTION 6: SOME COMMENTARY ON THE EVALUATION 97

6.1 Why does improved competency of the teacher not result in improved learner

performance ............................................................................................... 97

6.2 What can be done in a similar situation to improve uptake. ................. 98

6.3 Is there any value added by further evaluation...................................... 99

6.4 Intervention and evaluation design......................................................... 99

6.5 Last words............................................................................................... 100

ACKNOWLEDGEMENT This report was compiled by staff of QPiE. The report was written by Paul Hobden and Sally Hobden, with professional assistance from Thokazani Mkwanazi and Thelma Rosenberg. Thanks must be given to all staff at RUMEP who assisted by facilitating access to the schools and provided the many documents and reports; to the teachers who allowed their learners to be tested and to fill in the survey forms. Special thanks also to Mrs Mfani Poho from Fort Hare University who coordinated the fieldworkers for the mathematics testing component of the project. Distribution of report It is impossible to provide detail descriptions of the project useful for formative evaluation and to keep the identity of the teachers or schools anonymous. In addition, we have used photographs to provide the reader with as much of the relevant context as possible. In addition, because tests results hold the danger of being potentially detrimental to the confidence and self-image of both teachers and learners, especially in poor communities we strongly recommend that results should not be disseminated to schools or teachers. Consequently, we ask that this report be restricted and is not circulated.

Prof Paul Hobden, BSc. MEd. PhD. Evaluation co-ordinator.


©QPiE RUMEP BEd Project Summative evaluation Mar2009 1

SECTION 1 EXECUTIVE SUMMARY

The basic purpose of the evaluation was to provide information to Zenex Foundation and RUMEP about the extent to which the three year project had made a positive contribution to the quality of Mathematics teaching in grades 10 and to document what could be learned from the process. The RUMEP Fort Beaufort Mathematics Project (known as the “Project”) had identified among other the following factors as contributing to the problem of low achievement in mathematics: Teachers were under or unqualified teachers and consequently lacked both the content knowledge and pedagogic skills required; Teachers/schools had a shortage of materials and resources to deliver the curriculum properly; Teachers lacked an adequate support structure for professional development within the schools. The above factors were believed to impact negatively on learner performance in mathematics. RUMEP solution was to address this problem through the provision of a B.Ed programme to mathematics teachers in high schools. Given the problem was seen by RUMEP as not content knowledge only, but related to skills, support and resources, the project elements they designed also included; provision of teaching resource materials; assisting teacher to set up peer support networks; providing intensive in classroom support and mentoring to teachers; providing workshop based and residential teaching for the B.Ed.

Learners get poor results

and low participation in

maths

Teachers have poor content knowledge,pedagogic skills,assessment techniques.

BEd Inset Improve skills

Improved classroom teaching

Shortage of resources to

deliver curriculum

Classroom support

Provide Resources

Improved results

Higher uptake of

Maths

Current situation

Underlying reason

Intervention Direct Impact

Indirect impact

Lack of support for teachers and new curriculum

The evaluation approach taken may best be described as realist (Pawson & Tilley, 1997) with a focus on an intervention in context producing outcomes. Consequently it followed a mixed method research approach using both qualitative e.g. case studies and quantitative data e.g. comparison groups. It also had elements of participatory evaluation (Fetterman, 1996) as it was required to be responsive to the interests and concerns of the stakeholders and provide opportunities for improvement during the project. The report is organised into five main sections. The first is this executive summary which provides an overview of the findings. The second is the section describing the professional development programme offered by RUMEP for the project teachers, thirdly a report on the overall project findings for mathematics achievement. This was followed by a chapter dealing with attitudes to mathematics and the classroom environment. Case studies of the Grade 10 teachers are then provided and the final section looks at the overall findings, suggests reasons to explain them and finally some suggestions to guide future projects.



1.1 PROFESSIONAL DEVELOPMENT

The evaluation team visited RUMEP during two of the contact sessions when teachers were present and we were able to view teaching and learning. We had a number of other visits to RUMEP during which we accessed documents and teacher portfolios and discussed the professional development course. It is our finding that the professional development programme offered by RUMEP met all expectations laid out in the proposal accepted by the Zenex foundation. All components as described were implemented such as the face to face teaching, continuing afternoon sessions, local school based support, and provision of resources. It is also our finding that the programme was informed by best practice outlined in the educational literature and was only limited by the time and funds made available. Overall we consider the programme to have been exemplary within the constraints of time and funding.

Is there evidence that the project intervention provided the opportunities for teachers to become competent mathematics teachers? We feel that the professional development component of the project was a complete success in that it provided a structured programme which maximised teachers’ opportunities to develop themselves to become effective mathematics teachers taking their context into account.

1.2 MATHEMATICS CONTENT TESTS

A GET basic skills mathematics test was administered by fieldworkers employed by the evaluators in February 2007 and 2008, to the Grade 10 learners taught mathematics by RUMEP B.Ed. project teachers in twelve different schools, and to the Grade 10 mathematics learners at six similar schools to provide a control group for future comparisons. Testing was conducted at the end of 2006, 2007 and 2008. This test contained the same basic skills items and an additional section testing the Grade 10 mathematics curriculum content. With respect to progress from 2006 to 2007, the project schools showed, on average, some improvement in both sections of the test, while the control schools showed an average decrease in performance. The progress made during 2007 was slight. Comparing the project and control schools, it was noted that the mean scores for both the Grade 10 content and basic GET mathematics sections were statistically significantly higher in the project schools than in the control schools, despite the fact that the mean scores for the basic skills test at the beginning of the year showed no statistically significant difference. This pointed to better learning in the project schools in 2007. The learners at the project and control schools were comparable at the beginning of 2008 as evidenced by the mean scores on the basic GET skills test which showed no significant difference. Neither the October basic GET skills test nor the grade 10 content test revealed any significant difference between the project and control schools by the end of the year. There is no evidence from the maths testing that learning was better in the project schools in 2008.



In general, the overall level of achievement in the Grade 10 maths content has continued to decline. In almost all cases, the mean scores on the basic GET skills section improves from the beginning to the end of the year. This is obviously a good thing, but the concern is that it is achieved at the expense of progress with the current grade work. It remains a cause for concern and puzzlement, that despite the exemplary professional development programme offered, and the efforts of the teachers whom we have observed to be teaching adequately, the achievement in our Mathematics tests remains so poor. We have reviewed the test repeatedly and remain convinced that the level of difficulty is low, and the items test the GET mathematics skills at very basic level, and the Grade 10 section contains the simplest examples of grade appropriate work.

Has the project intervention made a difference to the learners’ mathematics marks? The improvement in the mathematics was only seen in the basic GET skills but the scores were still very low. While this indicates some progress, it is too little too late, and learners are falling behind with Grade appropriate work. There is no evidence from the maths testing that achievement was better in the project schools in 2008 compared to the control schools..

1.3 ATTITUDE TESTS

It is well known in mathematics education that motivation to learn mathematics and a productive disposition towards the subject is an important feature in successful mathematics learning. The 28 items on the attitude test were put into themes and a composite positively worded statement attached to each theme. The combined score on all themes was taken as measure of overall attitude towards mathematics. As in 2006, the greatest agreement in 2007 was accorded to the statements relating to the importance and high status of mathematics, and the least with the perception of mathematical competence. The differences in attitude on each of the scales was computed for all the matched learners and very little change was evident from February to November. The changes, although small, did however direct attention to aspects of each of the attitudes in each school. The overall attitude of the grade 10 learners in the project schools declined over the year, mainly due to the decline in perception of competence in mathematics, understandable in light of the poor scores. In contrast, the overall attitude of the grade 10 learners in the control schools increased due to increased scores on perception of competence and confidence – difficult to understand in view of the extremely poor performance on our tests. The learners at the project and control schools began 2008 with very similar mean attitudes to mathematics – again typically high agreement that maths was useful and a high status subject , and low mean agreement that they were competent at the subject and confident when doing maths. The learners at the project schools showed significantly higher mean perceptions of teacher affirmation which is encouraging. No statistically significant difference was measured on any scale between February and October, indicating that the mean attitudes of learners was unchanged throughout the year.



More nuanced data was obtained by disaggregating data according to school, and this is discussed in Section 4 under the case study of each teacher. Some teachers consistently had their learners indicate good agreement with the scale my teacher affirms me, which is a positive finding. The stark discordance often noted between perception of competence (high) and the actual marks obtained in the Grade 10 testing (very low) is difficult to understand. It is however, consistent with the findings of TIMSS 2003, that in general, the attitudes of South African learners, the vast majority of whom are African, indicated that they enjoy and value mathematics and science (Reddy, 2006). Reddy cautions that we "must consider that these may be socially desirable responses, and one would have to probe further to determine the 'real' attitudes of the learners" (p. 95).

Learner attitudes do not seem to change easily and the pattern of high agreement that maths is a high status and useful subject, and low agreement with statements related to perceived competence and confidence persist. The teacher affirmation scale seems to be an indicator of a caring teacher, and a scale sensitive to teacher absence and change.

1.4 CLASSROOM ENVIRONMENT

With the introduction of the new curriculum came changes in the way we would expect teachers to interact with learners and how they would arrange their classrooms for learning. The classroom environment survey was used to try and determine if the project teachers were changing their practices to align themselves more with the desired teaching and learning practices asked for in the new curriculum. The course they were attending emphasised these changes and modelled many of the desired practices in the contact sessions e.g. group work, cooperative environment etc. It would be expected that if this component of the project intervention was effective there should appear some differences between the control schools and the project schools and that this change should be picked up by the classroom environment survey. At the beginning there was little difference between the project and control schools except for the one category of teacher support. At end of the second year there was definite statistically significant positive changes in the mathematics classroom environment taught by project teachers compared to the previous year (they improved as time passed) and to the control schools (the intervention is making a difference). These changes occurred in nearly all of the categories and indicated that many if not most of the project teachers were moving toward creating positive learning environments. Unfortunately these gains did not continue into 2008 when the teachers had finished the course and graduated. In many cases the learners reported less positive learning environment. However, in general over the whole three years of the evaluation, environments did in general improve, and the project teachers created more positive environments than the control teachers. This was further supported by the fact that when project teachers left a school or were away for a long period that the environments became more negative.

Overall those teachers who participated on the programme created more positive learning environments aligned to the new curriculum than those who were in the control schools.



1.5 CASE STUDIES

It needs to be recognised that the real engine for change in educational programmes is the process of differently resourced teachers making constrained choices amongst a range of opportunities provided (Pawson & Tilley, 1997). For this reason a lot of emphasis was placed on studying teachers in their school contexts. A descriptive summary of all the project teachers is provided in the report. Below is a table summarising some of the findings for individual teachers. It is a summary of the changes in the learners’ mathematics achievement, attitude and perceptions of mathematics classroom environment over the three years. For each case study an example of a school visit and observation is provided together with all the contextual factors which we feel impact on the ability to provide effective teaching and learning in that school. These descriptions provide some context to the conditions of the schools and the problems associated with transferring skills to the classroom realities. These case studies also illustrate that aggregating data can often hide important findings from view. We found that there were many different consequences to the professional development training received. For example some teachers did not improve their teaching. It was obvious that the BEd was for personal development and they were not interested or able to translate this into changed practice in the school classroom. Others were able to do this and were examples of dedicated teachers implementing their new found knowledge yet there were only limited improvements in the learners’ mathematical marks. What the case study data pointed to were the reasons why this might have happened. The data also illustrated the point that it is difficult to isolate a normal case as each teacher operated in a different school context and had different personal circumstances that led them for example to seek employment elsewhere using the new degree as the enabling factor for promotion. We did find some common factors that stood out as determinants of translating professional development into improved classroom practice. Firstly these were associated with personal determination i.e. a teachers motivation and drive to make it work and secondly constraints created by the school in which the teacher operated.

Is there evidence that the quality of the project teachers’ classroom teaching has improved?

In general we found that the majority of teachers were able to translate their professional development skills from the degree course into the classroom (such that they could be considered competent qualified mathematics teachers) but this did not necessarily result in improved mathematics achievement. There was also a wide variety of factors which influenced teachers transfer of skills to the classroom most importantly being personal factors and constraints arising from the school context.



1.6 COMMENTARY ON THE FINDINGS

i) Why does improved competency of the teacher not result in improved learner performance?

No direct link could be found to suggest that competence of the project mathematics teachers would result in the improvement of the mathematics grade 10 scores. It is our contention that basics skills are the major stumbling block. The learners entering grade 10 have very low basic skills in mathematics that despite competent teaching they are not able to access the grade 10 level mathematics and consequently show very little improvement over the year despite having a competent teacher. This was one of the major logic links made in the programme design. It was hypothesised that if the teachers were provided with professional development such that they were competent in both the mathematical skills and the pedagogical skills that improvements would be seen in the achievement of their learners. We did not find any causal or correlational evidence to support this general statement from the tests we administered to the project and control groups. We do not believe the approach was a waste of resources. Besides being competent mathematics teachers, some teachers achieved personal success through promotion or contributed in other areas to building capacity within the system. The programme was built around the reasonable premise that for learner achievement to improve, every teacher in the system must be trained and be able to implement in the classroom. We believe this programme was able to achieve this. But we now see that this was not the critical step or only stumbling block preventing good learner results. Why has there not been a significant improvement? The major reason was the lack of basic skills of learners moving into grade 10 with minimal improvement during the year. We also believe the following all played a part also militating against good results: The learner lack of motivation mentioned by many teachers; School factors, such as the focus on Grade 12s to detriment of other grades, and high individual teacher workloads; Teacher factors include the administrative duties that come with senior posts in the school which cut down on time available for classroom teaching and the lack of enthusiasm to change from their colleagues. ii) What can be done in a similar situation to improve uptake?

It is our considered opinion that a serious intervention is required before or during grade 10 to make sure learners doing grade 10 have basic mathematical skills.

Previous programmes and interventions have focussed on schools and teachers. We feel that it is now time to focus on the next part of the complex system being the learners, and motivate them to learn mathematics and provide opportunities to remediate basic skills. These suggestions have resource and management implications and are likely to be resisted as impossible and financially expensive to implement but to continue is to go against all the data collected over the last ten years showing little progress in mathematical competency in our rural schools. Unfortunately we are wasting resources if we allow the current situation to continue. i) Generating interest: We feel some programmes for the school vacations outside of schools in which maths is put forward as fun, exciting and valuable needs to be put in place. This should not be curriculum based but rather focus on teaching learners basic



reasoning skills for mathematics in the context of number games, maths puzzle solving, maths logic etc. Hopefully this will improve learners mathematical functioning and get them interested in spending time on mathematics. It should be restricted to learners in GET phase and if it has a competitive element should be based on teams. The object is not to celebrate the clever maths learners but to get the “normal” learners interested and motivated.. ii) Parallel curriculum: We also feel that it is time the department and teachers are made aware of the fact that teaching higher levels of mathematics to children who do not have the basics is totally unproductive. Our main suggestion is to develop a programme of “alongside learner support”, to remediate for basic skills whilst carrying on with grade appropriate work. This requires careful analysis of the basic skills required in each section and timeous “teenage appropriate” activities to refresh or introduce the basic concepts that will be required. This would require extra time – it cannot replace the normal curricula maths. The problem of inadequate basics is pervasive across all contexts including urban and township school. iii) Further research.

We believe that it is very important to continue with the evaluation but with a major focus on the teachers and the senior certificate results and with less emphasis on the testing of the grade 10s. The evaluation has been extended by a year to obtain further information about the long term effects of the intervention. Given the current trend, we do not believe at this stage that the testing of the grade 10s for mathematics achievement will show an improvement. However, it will tell us if the low base level of mathematics is stable and inherent in schools of this type. It will reinforce the suggestion that a basic skills intervention is required. Another focus will be the long term accumulative effect of the full cohort of teachers in the schools on the senior certificate results. Hopefully, there will be an accumulative effect and some improvement in the mathematics results of the district will be found when the complete cohort of teachers is considered. For example a learner could get a number of project teachers over the last three years of schooling resulting in a change from the norm. These examinations are also high stakes and learners will study hard and hopefully benefit more from what they were taught in grade 10. Consequently we should see an improvement over the years. Thirdly it will be interesting to determine what happens to the teachers who participated in the project. We will be able to see if they remain in the mathematics classroom or if they simply use the degree status to further personal ambitions and get promoted out the classroom. This could require further thinking around how to keep good teachers in the classroom. iv) How can the evaluation be improved?

It is our feeling that future interventions should be very closely examined so that i) they are not set up based on faulty logic models, ii) more effort be placed on comparison groups or alternative evaluation strategies and iii) independent learner testing be closely aligned with school testing.



Logic models: The intervention was based on a logic model which was found to be suspect. It was predicted that if the teachers were well trained and supported in schools that learner achievement would improve. Unfortunately there are too many other factors in play to support a causal link even if the results had improved. It is our feeling that future interventions should be very closely examined so that they are not set up based on suspect logic models. To infer causation, there has to be a logical link between the “dose” and the “response” but it has to be accepted that often there is not a linear relationship or there is an indirect link only (Davidson, 2005). In this study we were tracking learner marks and there is actually no accepted model or relationship between teacher training and learner achievement. It is a contested area within the literature. Comparison groups: Evaluation designs that have good comparison groups are all but impossible to set up. Unfortunately there is convincing evidence (US Department of Education, 2003) that “in a sizeable number of cases the inaccuracy produced by the design is large enough to result in erroneous overall conclusions about whether the intervention is effective, ineffective, or harmful.” (p.3). For example in this study the comparison group of classrooms were chosen from surrounding schools. Because by definition the project has no influence over the control schools, many changes occur during the course of the three year study. In the end the evaluators were not sure that the control schools were in fact a reliable comparison any more (which assumes no changes to school and teachers over the three years). The point being made is that serious thought needs to be put into the value of comparison designs and resources required to set them up effectively. Either a lot more resources allocated in choosing and monitoring controls or alternatively more reliance on a blend of strategies (Davidson, 2005) with less emphasis and status on quasi experimental designs. Testing: The evaluation design included testing of learners at the beginning of the year and again at the end. Low results were obtained that were different to those of the internal school examination. We suspect that other factors besides test design are influencing results such as test preparedness and motivation. We believe preparation for testing is a very important factor. If evaluators are to continue testing in schools some system of cooperation needs to be worked out whereby learners are able to prepare for the testing. One example would be to include questions in the normal testing programme of the school or to conduct the tests immediately after school examinations. At present we might not be measuring the learners true potential. Consequently when we report levels of achievement of 10 % for grade 10 we are not sure if this is the learners actual potential or one achieved through disinterested participation in what they perceive as a low stakes test. This issue needs some discussion among the role players to improve the validity and reliability of the testing programmes. Perhaps contracts with schools who participate, to integrate the evaluation tests into the school programme and allocate times as part of normal examinations. Last words

Intervening in schools to bring about change is a complex business. We are seeing that there are a number of factors that all need to be addressed at the same time. This evaluation has informed us that having a competent teacher in a functioning school is necessary but not sufficient for mathematics achievement to improve significantly. Unfortunately the solution lies in addressing all three at the same time being school management, teacher competency and learner preparedness.



SECTION 2– THE INTERVENTION

2.1 FINDINGS

In a recent report on Professional learning (Wei, 2009) it was acknowledged that professional development does not always lead to professional learning. Professional learning is defined as a product of activities that increase teachers’ knowledge and change their instructional practice in ways that support student learning. The report indicated that there are a number of factors that need to be considered for the development to be successful. We will consider just two as examples. Firstly the programme must focus on concrete tasks of teaching, assessment, observation and reflection rather than abstract discussions of teaching. This was the case in this project. Attending lectures and examining learner reflective portfolios provided sufficient evidence to determine the obvious practical focus of the learning modules. Secondly the report indicates that the best chance of student achievement improving occurs when teachers were engaged in sustained, collaborative professional development that focuses on deepening teachers content knowledge and instructional practices. This programme took place over three years, had built in activities that encouraged collaboration such as weekly local support meetings and formation of professional support groups. What we are reporting is that in our professional opinion , the professional development intervention of the BEd for the teachers was aligned with best practise and while not perfect, was exemplary given the time and resources allocated to it.

Evaluation question: Is there evidence that the project intervention provided the opportunities for teachers to become competent mathematics teachers? We feel that the professional development component of the project was exemplary in that it provided a structured programme which maximised teachers’ opportunities to develop themselves to become effective mathematics teachers taking their contexts into account.

2.2 EVIDENCE OF AN EXEMPLARY PROGRAMME

We have described in previous evaluation reports (Implementation, 2006: Interim, 2007) some of the contact sessions we observed and the support activities. We also indicated that a few minor issues needed sorting out to improve the programme. We have followed up on these and RUMEP has responded e.g. designing improved tests etc. What follows is a description of our final visit to RUMEP in which all the evidence of the teachers work and service provider inputs were displayed. The Bachelor of Education (In-service) had run its course by the end of 2007 and so the visit to RUMEP on Monday 25 August 2008 was to interview the staff and to scrutinise the evidence presented. The evaluators were shown to a room that contained all the evidence, beautifully presented and organized.The following aspects of the project intervention will be discussed: resources; formal course materials and student work; school support visits and testing,



Teaching resource materials

The resources provided were carefully documented. From evidence presented, the record showed, for example, that at Ntabenkonyana Secondary School, 240 sets of maths instruments were provided, 50 calculators, 87 Grade 12 textbooks in 2005, and 8 assorted single copies of maths textbooks. In 2006; another set of about 8 different textbooks, a blackboard compass and protractor, a further 60 maths sets, and 8 copies of a Grade 11/12 textbook; In 2007; a further 48 maths sets, and two more teacher copies of textbooks. These materials were taken and signed for by the teachers.At the school visits, the evaluation teams asked to see these materials and this is reported in each case study. There was a small display of some of the materials that the teachers had been given over the three years. This included:

Basic stationery such as pencils and rulers, scissors and mathematics sets

Blackboard compass and protractor Fraction pieces and geometry shapes

template RUMEP materials) Pads of dotty paper, shapes, 100

squares and Cartesian coordinate systems

Playing cards and dice for classroom activities

Scientific calculator Wooden geoboard constructed in

technology module Maths dictionary and multilingual

concept dictionary for Mathematics and Science concepts

RUMEP teacher resource – Rethinking the Teaching and Learning of Mathematics

The textbooks given to teachers included standard textbooks, revision series, problem solving and puzzle books e.g. Thought Provokers by Doug Rohrer In addition, as part of the Language of Teaching and Learning (LOLT) initiative, teachers were given books on maths and science for a class library. These were not textbooks but more popular type books and also puzzle books and puzzle stories.

Figure 2.1 Resources provided



Formal course materials

The documentation related to the formal teaching of the module was provided, together with a few posters of analysis done as ongoing self-monitoring of the programme. In addition to a General Information file containing all the course information and module outlines, three big lever arch files were provided with module materials given to students. These are typically small booklets on single topics such as trigonometry, data handling, transformation geometry and are a combination of actual curriculum content and some teaching ideas. We have most of these materials on file.

Figure 2.2 RUMEP documentation provided

In summary, the modules presented over the duration of the programme were:

Table 2.1 RUMEP BEd (Inservice) Programme modules

Module Examiner(s) Content Maths for Teaching 3

Tom Penlington & Thomas Haywood School level maths content

Curriculum Studies in Mathematics 3 Rose Spannenberg

Based on their own teaching portfolios. Value and discussion of examples of learner work etc.

Professional Development in Mathematics 3

Tom Penlington Errors, misconceptions and slips. Some theory and then lots of student to discuss errors and remediation strategies

Classroom based research module Thami Mahlobo

Project done by teachers in their classrooms and reported at a mini “How I teach” conference and by means of a written report.

Professional development in Mathematics

Various staff

Workshop presentations. Teachers had to do 2 workshops, one of which was attended by an assessor and write reports on both.

Assessment file: A file was presented containing marks and assessment tasks for the three years. We took a copy of the university academic record (2005, 2006) and separate sheet of 2007 marks. This could potentially be used to correlate academic performance of the teacher with school based performance and learner progress. File of course evaluations: This contained the RUMEP BEd (in service) course evaluations and asked about development in teaching practice, classroom management, assessment, lesson planning and if beliefs and attitudes to maths teaching and learning had changed in any way. They were also asked about what had been learnt from conducting workshops and invited to comment on any negative aspects of the course and provide suggestions. Comments were mainly positive, for example: You RUMEP staff are superb. You have got all the skills to empower teachers and to groom them to



grow professionally. The negative comments were few but concerned the perceived high workload especially the crowded third year, and that some lecturers were perceived to have treated the teachers disrespectfully. Lesson Videos: On his school visits Mr Haywood videoed lessons of the teachers and then played them back to the teacher and engaged in a discussion. These videos were then used in teacher workshops where teachers were asked to critique each others and their own lessons. We were given the dvds of the lessons which we viewed and used in cases where the teacher was not able to show us a lesson. Given that these were used during the sessions they were again evidence of making teachers reflect on their practices and as such are excellent resources. Examples of teacher work

Teacher Portfolios Four 2007 portfolios were presented for us to look at. These files were retained by RUMEP as examples for the new students. The rest of the portfolios were returned to the teachers and we asked to see them in the schools. In most cases teachers had these portfolios and were using them in their planning in 2008 indicating that these were not just theoretical exercises but rather the creation of a useful resource. Two portfolios are described here as examples. Ms Manengele This is not one of focus teachers, but started the year at a school together with Mrs Sanqele who is a focus teacher. Her work was presented in two substantial files and scored 93% for her assessment, so this work can be considered exemplary.

Figure 2.3 Assessment rubric showing 93% achievement

The work was arranged by term and the photograph alongside shows the Table of Contents. This is a collection of very valuable documents that give good overall picture of the functioning of schools. Reading the reflections on the work done, it is clear that the schools are subject to many interruptions which greatly hinders progress.

e.g. 12 March to 16 March I could not finish what I had planned this week because of the short week, on Wednesday there was a match, Thursday were sport features and Friday was the HIV Aids awareness. 19 March – 23 March This week was very short, there was a public holiday on Tuesday and there was AIDS activity on Thursday and Friday SADTU activity. Therefore, we did not do a lot.



Examples of marked learner was provided, showing for example that formative comments had been made regarding the size of equal parts in fractions. Photographs were provided of the learners busy with practical activities related to ratio. It was impressive to see the response that had been planned in reaction to what was read in the learner reflections and how remediation or interventions were planned. Mrs Mntwini This is one of the focus teachers. She wrote of being very overloaded and stressed.

At school I do not have a chance to be free, sometimes during break time I had to go to a science class to prepare for experiments the next day. At present I am teaching Technology Grade 8 & 9; LO Grades 8, 9 & 10; Physical Science Grade 10, 11 & 12; and mathematics Grade 10.

Her lesson planning is quite brief which is to be understood in light of the fact that this is just but one of 9 different classes taught. She also records the stress of the external evaluators arriving on 7 February 2007! She has examples of marked work without indication of what was wrong. There is evidence that work done in the modules was transferred to the classroom. For example, the method of finding the area of a circle activity was seen in one of the modules. She also has an extensive case study of a learner that she was helping in particular. This was an excellent reflective activity for professional development. Integrated Planning files Two of these files were presented. These files include a rationale, and go from long term planning down to individual lessons, assessment activities and evidence of learner work. The photo in Figure 2.9 shows the Table of Contents. These files are clear evidence that the key issues of planning were thoroughly discussed with the teachers, and that the teachers have had supervised practice in planning.



Mathematics Workshops Part of the course requirements was to present a workshop for colleagues in their school or areas. Several teacher reports of their workshops were made available to us. The reports displayed included (as examples): Mrs Lungiswa Banga : Workshop on data Handling held at Ngcabasa Primary Hall on 19 September 2007. The teacher provided the programme, workshop evaluations and photographic evidence. A RUMEP lecturer attended and evaluated the workshop. Mr Petros Mellet: Surface Area and Volume (conducted in Afrikaans) held at Templeton High School on 17 October. This workshop was attended by teachers from surrounding primary schools as well as high schools. The multilingual concept dictionary (a RUMEP resource) was used to provide teachers with translations of the various geometric terms. Teacher profiles file This file contained a short biography of each of the 10 remaining project teachers, their teaching philosophy, and in some cases examples of their planning. We were given copies of their biographies to be used as context during our school visits. Benchmark pre and Post Tests

Following our input in July when we pointed to the lack of progression in the RUMEP testing, the staff have produced new tests for 2008. There are tests from Grade 8 to Grade 12. The first 10 items are multiple choice and test basic skills (below grade level). The answers to these are indicated on an answer sheet on the inside cover of the booklet. The remaining questions were free response type more appropriate to the grade level. Copies of the Grade 10 test were provided. Graphs of the testing results, as well as the matric results in the area for the past few years were provided.



2.3 COMMENT

Overall it can be seen that we found lots of evidence to support our assertion that the professional development programme in the form of the BEd modules, the resources and support were exemplary within the context of time and funds available. Why this was the case at RUMEP was obvious to the evaluators who between them have over 60 years of teacher professional development experience. Firstly, it was obvious from our many visits that the staff complement at RUMEP was stable. The same people worked on the programme for the three years. They were staff who were committed to careers in professional development with many years of combined experience as opposed to many projects we have seen in other interventions, in which a few contract staff are brought on board to service a project and come and go depending on funding. Secondly, feel that a major strength was that the staff who provided the input were also the staff who conducted the workshops, evaluated the teachers and visited them in the schools. This continuity cannot be underestimated particularly when dealing with adult learners. Thirdly, the staff had strong relationships with the local teachers and schools. For example, built into the programme were the need of teachers to run workshops for colleagues which helped spread the “good news”. Taken together with many school visits it was obvious that the staff took seriously the concept of basing their work in the realities of the schools.



SECTION 3– TESTING OF GRADE 10 LEARNERS

The 2008 testing followed a pre-post test design and it was important that the same learners were tested in both February and October and care had be to taken to keep all the data from a particular learner together. This was achieved by assigning learners who wrote the February test reference numbers, and putting that same number on their October scripts. Ideally each learner would have scores for mathematics content in February and October, mathematics attitude in February and October, and classroom environment in October. In practice the matching was not perfect – many learners did not write again in October and there were many “new” learners. For the purposes of comparison, and to include only learners who had been at the school all year, most of the analysis of the Maths tests was confined to the matched learners.

3.1 MATHEMATICS CONTENT TESTS

Construction of the test

The February test consisted of basic multiple choice questions covering the mathematics content of the GET phase. The scores on this test provided an indication of the mathematical knowledge and skills of the learners entering Grade 10, and a point of comparison between the project and control schools. The November test had two sections, the first a repeat of the basic GET multiple choice questions, and the second part consisted of questions designed to test, at a minimum level, the competency of the learners in the outcomes of the FET Grade 10 curriculum. This test was the same as had been administered in 2006 and 2007to ensure consistency and the correct grade level even though the scores have been very low. Administration of the test

We are fortunate to have secured the help of the same group of fieldworkers led by Mrs Poho for the three years of this evaluation., They administered the tests in both February and November 2008, and are now well known in the schools. In an attempt to avoid the problems encountered in previous years when we tried to do the testing around exam time, in 2008 the testing was done in October. Data reduction

This was done as previously, and is described in the initial report. In short, the multiple choice question responses were coded numerically and analysed in SPSS, and the Grade 10 content questions were again marked by university students and a teacher under the supervision of Mr Mkhwanazi. Analysis of the test

The following scores were computed for each learner who was matched : February basic GET mathematics competence score November basic GET mathematics competence score November Grade 10 mathematics competence score

In addition, sub scores for the separate learning outcomes were computed. This allowed for finer analysis and often showed up sections of work that were either well taught and learnt or had not been done at all.



Comparison between project schools and control schools

Basic GET mathematics skills The mean score of all the matched learners (N=383) at the beginning of the year for the basic skills test was 31.1 %. There was no statistical difference (p = 0.625) between the mean score of the project schools (30.8% ) and the mean score of the control schools (31.4%). This indicates that the groups of Grade 10 learners at the project and control schools began, on average) with equivalent basic GET maths skills.

ImitshizJosi MakJonguhlaElukhanyNompenduHohoInyibibaAmajingqNgangeliNozalisiKama HigMasizakhJabavuEyabantuNtabenkoTempletoHector P

Mea

n F

eb

Ba

sic

sk

ills

tota

l %

60

50

40

30

20

10

0

27.331.0 30.8

25.0

31.928.2

24.7

37.2

30.5 29.3

37.134.8

26.028.0

33.7

26.929.2

At this level, learners would have demonstrated competence in half the GET basic skills tested.

Blue bars denote project schools

Figure 3.1 Mean scores on February GET basic skills test, by school

The basic skills were tested again at the end of the year as part of the final test and the mean improved slightly to 34,9% which was a statistically significant (p=0.00) difference. The mean score in the basic skills increased from 31.4% to 34.3% in the control schools, and from 30.8% to 35.4% in the project schools. The October mean basic skills tests score of the project schools was however, not statistically significantly better (p = 0.4) than the mean score of the control schools. Grade 10 mathematics The 50 mark section of the test covering basic Grade 10 content was again very poorly answered. The mean score for all 444 learners who wrote the test was 6%, and no learner got above 40%. This is shown graphically in Figure 3.2.



1009080706050403020100

Fre

qu

en

cy

400

300

200

100

0

Mean =5.97�Std. Dev. =5.551�

N =444

Figure 3.2 Mean score on Grade 10 mathematics content of all grade 10 learners tested (project and control schools)

The following data which is disaggregated by school, is computed for the matched cases only so that we have comparable learners in the project and control schools.

ImitshizNgangeliJosi MakMasizakhAmajingqHohoJonguhlaNompenduElukhanyEyabantuTempletoJabavuKama HigInyibibaNtabenkoNozalisiHector P

Mea

n G

rad

e 10

co

nte

nt

tota

l %

30

25

20

15

10

5

0

4.93.6

5.93.2

5.46.7

1.7

11.9

5.35.4

10.411.8

2.54.6 4.35.2

4.0

Blue bars denote project schools

Figure 3.3 Mean score on November Grade 10 mathematics content test disaggregated by school

The mean score of the learners in project schools (6,2%) is not statistically significantly different to the mean score of the learners in the control schools (6,6%). The mean score at all schools is extremely low – fact the raw scores show that many learners scored just 0.5 mark on the entire test, indicating that the grade 10 work was completely unfamiliar to them. Figure 3.4 shows the distribution of scores for each of the three maths tests, for project and control schools. It is particularly interesting to note the schools that appear as higher achieving outliers in each case – Ntabenkonyana has six learners who are doing



much better than the average for the project schools which must be a credit to Ms Sanqela.

RUMEP Project

Control schoolProject school

100

80

60

40

20

0

Nozalisi

TempletoNtabenko

Ntabenko

Hector P

Hector P

NtabenkoNtabenkoNtabenko

Ntabenko

Hector P

Hector P

Hector PHector PHector P

Hector P

Hector P

Hector P

Hector P

Hector P

Jabavu

Ntabenko

Hector PHector P

Hector P

Jabavu

Grade 10 content total %Basic skills total %Feb Basic skills total %

Figure 3.4 Comparison of distribution of scores on Grade 10 mathematics content, and basic GET skills tests for project and control schools

Comparison of February and November basic GET mathematics skills scores

As mentioned above, and illustrated in Figure 3.5 the mean scores in the GET basic skills improved slightly in both project and control schools. This is of no practical significance since the scores are still very low, and the main indicator of adequate progress is the achievement on the Grade 10 level test.

RUMEP ProjectControl schoolProject school

Mea

n

60

50

40

30

20

10

0

6.66.2

34.335.431.630.8

Grade 10 content total %Basic skills total %Feb Basic skills total %



Figure 3.5 Comparison of mean scores for Feb and Oct GET Basic Skills and Grade 10 content for Project and Control schools

The comparison between February and October basic skills mean scores for the different schools is shown in Figure 3.6. The error bars (showing the 95% confidence level) show that the difference noted is always within the overlap of the error bars, and so is not statistically significant.

SchoolNompenduMasizakhKama HigJosi MakJabavuInyibibaImitshizHector PEyabantuTempletoNtabenkoNozalisiNgangeliJonguhlaHohoElukhanyAmajingq

Mea

n

60

50

40

30

20

10

0

Basic skills total %Feb Basic skills total %

Figure 3.6 Comparison of mean scores in February and November GET basic skills tests, disaggregated by school

Summary

Progress from 2006 to 2008 This is summarised in Table 3.1. The project schools which had been showing, on average, some improvement in both sections of the test, have fallen back somewhat while the control schools are showing an average increase in performance. Of great concern is the failure of the mean for the Grade 10 content to rise above 10%.

Table 3.1 Comparison of testing results in November 2006 to October 2008

Project schools Control schools Basic skills Grade 10

maths Basic skills Grade 10

maths 2006 31,2 7,3 31,1 7,7

2007 35,3 8,3 30,0 5,6

2008 35,4 6,2 31,6 6,6

Progress during 2008 The mean scores of the matched grade 10 project school learners in the basic GET mathematics section of the October test showed slight improvement from 30,8% in February 2007 to 35,4% in October 2007. The mean score of the matched control school



learners increased from 31,6% in February 2007 to 34,3% in October 2007.. When the statistical variation is considered these are very small differences. End of 2008 comparisons The learners at the project and control schools were comparable at the beginning of 2008 as evidenced by the mean scores on the basic GET skills test which showed no significant difference. Neither the October basic GET skills test nor the grade 10 content test revealed any significant difference between the project and control schools by the end of the year. There is no evidence from the maths testing that learning was better in the project schools. Better performing schools Mr Gontshi’s small class at Nozalasile (10 matched learners) continues to do relatively well and Ms Sanqela’s class at Ntabenkonyana (46 matched learners) has six learners appearing as outliers in Figure 3.4 indicating relatively good achievement. COMMENTARY

It remains a cause for concern and puzzlement, that despite the exemplary professional development programme offered, and the efforts of the teachers whom we have observed to be teaching adequately, the achievement on our Mathematics tests remains so poor. We have reviewed the test repeatedly and remain convinced that the level of difficulty is low, and the items test the GET mathematics skills at very basic level, and the Grade 10 section contains the simplest examples of grade appropriate work. What explanation can we offer? Basic skills: A close look at some of the learner work indicates that basic skills are lacking to the extent that new work cannot easily be accessed. Marks are not so much lost by careless errors, nor by pockets of unfamiliar work as often occurs, but by a complete lack of very basic algebraic skills such as expansion of bracket, multiplication of terms with powers and complete unfamiliarity with trigonometry and graphical work. We are aware that our test might be viewed by learners as low stakes, and that they had not prepared for the test but this does not account for lack of routine mathematical skills. Poor background often hinders the mathematical progress of learners even when they move to well resourced schools – other projects have seen learners(selected for their potential) struggling to achieve even 30% in Mathematics at grade 10 level. School factors: From the school visits it was apparent that schools are under tremendous pressure form the DoE officials to improve the Grade 12 results which are seen as a very public indicator of success. This seems to shift the time and energies of the staff to the Grade 12s to the detriment of teaching lower down in the school. For example, when a teacher had been away on leave for six months and returns to find little work had been done by her replacement, she admitted that all her time was spent on a bid to get the Grade 12s ready for the final exam. We have noticed dwindling learner numbers, particularly at the more remote schools. This leads to increased workloads for teachers as the number of teachers per school is dropped and one teacher has to deal with all the Mathematics (and sometimes also the Physical Science) in the school. Having up to seven or eight different classes to prepare for is a daunting task.



Teacher factors: Understandably, the teachers applied for promotion on the strength of their new qualification, and several seem to have been appointed to management posts. This is a positive development for them personally, but often moves them from classroom teaching, or loads them with administrative duties that squeeze out the classroom work. Their colleagues are not always prepared to take on the “RUMEP way

Evaluation question: Has the project intervention made a difference to the learners’ mathematics marks? The improvement in the mathematics was only seen in the basic GET skills but the scores were still very low. While this indicates some progress, it is too little too late, and learners are falling behind with Grade appropriate work. There is no evidence from the maths testing that achievement was better in the project schools in 2008. Individual teachers, notably two who have been a school for a while and who have a university science background are making better progress with their learners than the norm.



SECTION 4– ATTITUDES AND SURVEYS

4.1 ATTITUDE TESTS

Motivation to learn mathematics and a productive disposition towards the subject is an important feature in successful mathematics learning. An important role for the teacher is to promote this productive disposition and so an increase in positive attitude towards mathematics would be a positive signal. For this reason, the attitudes of the Grade 10 learners taught by the eight project teachers were measured, along with the attitudes of Grade 10 learners at nine control schools, were measured in both February and October. This allowed for initial comparison at project and control schools, and for changes in attitude to be computed. Construction of questionnaire

The attitude test (an adaptation of the short form of the well known Fennema-Sherman Mathematics Attitude Scale) used in 2006 was used again to ensure consistency. The construction of this test was fully described in the initial report on this evaluation. Administration of the questionnaire

As in 2006 and 2007, the fieldworkers gave this test on the same day as the learners did the mathematics content test, and the classroom environment survey. Data reduction

The February attitude questionnaires were numbered to match the mathematics tests which were pre-numbered, giving the possibility of investigating the relationship between the maths test score and the attitude score of each learner. The November attitude tests were numbered similarly, with those learners present in February retaining their numbers for November. As previously, the responses to each item were coded form -2 for strongly disagree to +2 for strongly agree, with adjustment for negatively phrased items. Data reduction and analysis of the questionnaire

The explanation of the data reduction given in the initial report is repeated here to assist in interpretation of the discussion that follows. The 28 items in the questionnaire were put into five themes as shown below. Negatively phrased items were reverse coded and the wording reversed to positive phrasing as indicated in the brackets. The numbers refer to the question number on the questionnaire. Perception of ability - I see myself as competent at maths

2 I am not good at maths (I am good at maths) 4 Most subjects I can manage, but I make a lot of mistakes in maths (I do not

make a lot of mistakes in maths) 23 For some reason even though I study, maths seems unusually hard for me

(Maths is not unusually hard for me) 27 I think I could do advanced mathematics after school



Perception of confidence - I feel comfortable and secure doing maths 1 Generally, I have felt secure about attempting mathematics. 5 I usually have been relaxed in maths class. 7 I get upset when I think of trying math problems (Thinking of maths does

not upset me) 20 Mathematics makes me feel confused (I am not confused by maths) 26 Mathematics usually makes me feel uncomfortable and nervous (Maths

does not make me nervous) 28 When I get a math problem that I can’t immediately solve, I keep on trying

until I have the solution. Status of maths - Maths is a high status subject

3 I don’t like people to think I am clever in maths (I like people to think I am clever in maths)

8 I’d be proud to be the best learner in maths 18 If I got the highest mark in maths I would keep it a secret (If I got the

highest mark I would not keep it secret) 19 It would make me less popular if I was really good at maths (Being good at

maths does not make me less popular) 21 Being seen as clever in mathematics would be a great thing 16 I am happy to get good marks in mathematics 17 It would be really great to win a prize in mathematics

Usefulness of maths - Maths is very useful 11 I’ll need to really understand mathematics for my future work 9 I study mathematics because I know how useful it is 12 I will use mathematics in many ways as an adult 13 Mathematics has no relevance to my life (Maths has relevance in my life) 14 I see mathematics as a subject I will not use much in daily life as an adult (I

will use maths in daily life) 15 It is worth the effort to study hard in maths 24 Knowing mathematics will help me earn a living.

Role of teacher - My teacher affirms me 6 My maths teachers have been interested in my progress in mathematics 10 Maths teachers have made me feel I have the ability to go on in mathematics 22 My teachers think I’m the kind of person who could do well in mathematics. 25 I have found it hard to win the respect of maths teachers (I have found it

easy to win the respect of my maths teacher) The mean scores for each of the five scales above were computed for each learner, as well as an overall mean which was computed by averaging the means of each scale (so as not to overemphasise a scale that comprised more items). Consequently, there were five sub-scores for each learner and an overall attitude score. Comparison between project schools and control schools

In order to ascertain whether the learners at the project and control schools were equivalent at the beginning of 2008, the mean scores on each attitude scale were computed and graphed for the project and control schools (see Figure 3.7). The mean scores were very similar with the only scale showing a statistically significant difference (p = 0.02) being the stronger agreement in the project schools with the composite statement my teacher affirms me. The two groups of learners had greatest mean agreement with the statements related to the high status and usefulness of maths, and low mean agreement with the statement related to their perceptions of their maths competence.



The mean attitude scores from the October testing were computed using only the matched learners as they were known to have been at the school for the duration of the year. A very similar pattern was noted (see Figure 3.8) and no attitude scale showed a difference at the 95% confidence level between the project schools and the control schools.


Mea

n

2.00

1.50

1.00

0.50

0.00

0.690.66

0.430.58

1.031.05

0.830.69

0.620.49

0.230.23

I have a positive attitude to maths

My teacher affirms meMaths is very usefulMaths is high status subject

I feel comfortable and secure doing maths

I see myself as competent at maths

Strongly agree

Agree

Figure 4.1 Mean attitude scores for learners in project and control schools in February


Mea

n

2.00

1.50

1.00

0.50

0.00

0.750.75

0.520.67

1.171.14

0.900.81

0.600.53

0.240.37



I feel comfortable and securedoing maths

I see myself as competent atmathsStrongly agree

Agree



Figure 4.2 Mean attitude scores for matched learners in project and control schools in February

The change in score for each scale was computed for each learner. The change could fall in the range -4 to +4, indicating a shift from strongly agreeing with a statement (+2) to strongly disagreeing with a statement (-2) or vice versa. Table shows the very small mean changes in attitude in the matched learners overall at project and control schools. The differences, (typically around 0.05) represent only about 1% of the possible change. Overall, the mean attitude scores for Project and control schools has remained stable, with no significant change from February to October.

Table 4.1 Change in mean attitude scores of matched learners from February to October 2008 in project and control schools

147 .0289 .66629 .05495

174 .0216 .66869 .05069

122 .0027 .69101 .06256

157 -.0435 .64443 .05143

136 .0326 .52997 .04544

171 .0309 .49280 .03769

125 .0811 .52500 .04696

168 .0629 .55045 .04247

148 .0541 .65630 .05395

174 .0603 .75835 .05749

76 .0616 .31787 .03646

114 -.0016 .35861 .03359

RUMEP ProjectProject school

Control school

Project school

Control school

Project school

Control school

Project school

Control school

Project school

Control school

Project school

Control school

Change in perception ofability

Change in confidence

Change of perceivedstatus of maths

Change in perception ofusefulness

Change in perceivedteacher affirmation

Change in overall attitude

N Mean Std. DeviationStd. Error

Mean

In the following section, the data is disaggregated by school to see if there are more notable changes in attitudes at some schools. Comparisons between project schools

When the data is disaggregated by school, the changes from February to November are more pronounced. This is because the changes were frequently in different directions at the different schools, and so when the data is combined the net changes are small.



SchoolTempletoNtabenkoNozalisiNgangeliJonguhlaHohoElukhanyAmajingq

Mea

n

2.00

1.50

1.00

0.50

0.00

-0.50

RUMEP Project: Project school

I have a positive attitude to mathsI have a positive attitude to mathsMy teacher affirms meMy teacher affirms meMaths is very usefulMaths is very usefulMaths is high status subjectMaths is high status subjectI feel comfortable and secure doing mathsI feel comfortable and secure doing mathsI see myself as competent at mathsI see myself as competent at mathsStrongly agree

Agree

Figure 4.3 Mean attitude scores for project schools in February and October 2008, disaggregated by school

Read from left to right, the bars in Figure 3.9 show the mean agreement with each attitude scale firstly in February and then in October. The data has been displayed this way because showing increase and/or decrease in mean score without reference to the actual mean score can be misleading. For example, the perception of competence score at Ntabenkonyana dropped from February to October, but remains the second highest mean score. We can offer no explanation for the high and increased perception of competence and confidence at Amagingqi where the learners scored a mean of 4% on the Grade 10 content test. The individual schools results are discussed later in the section of the report where the results of testing for the classes of each project teacher are discussed Comparison between control schools

The control schools show a similar pattern of little change in overall attitude. Imitshiza, a very poorly performing school with a mean of just 1.7% on the Grade 10 content test showed a marked increase in perceived mathematics competence! At Josi Marela which was made a control in 2008 due to the protracted absence of Mrs Nika during the year showed a marked decrease in the perception of teacher affirmation, and an increase in confidence. Hector Peterson which is the highest achieving school tested, shows a relatively high mean overall attitude to mathematics.



SchoolNompenduMasizakhKama HigJosi MakJabavuInyibibaImitshizHector PEyabantu

Mea

n

2.0

1.5

1.0

0.5

0.0

-0.5

RUMEP Project: Control school

I have a positive attitude to mathsI have a positive attitude to mathsMy teacher affirms meMy teacher affirms meMaths is very usefulMaths is very usefulMaths is high status subjectMaths is high status subjectI feel comfortable and secure doing mathsI feel comfortable and secure doing mathsI see myself as competent at mathsI see myself as competent at mathsStrongly agree

Agree

Figure 4.4 Mean attitude scores for control schools in February and October 2008, disaggregated by school

Summary

The learners at the project and control schools began 2008 with very similar mean attitudes to mathematics – again typically high agreement that maths was useful and a high status subject , and low mean agreement that they were competent at the subject and confident when doing maths. The learners at the project schools showed significantly higher mean perceptions of teacher affirmation which is encouraging. No statistically significant difference was measured on any scale between February and October, indicating that the mean attitudes of learners was unchanged throughout the year. More nuanced data was obtained by disaggregating data according to school, and this is discussed in Section 4 under the case study of each teacher. Some teachers consistently had their learners indicate good agreement with the scale my teacher affirms me, which is a positive finding. The stark discordance often noted between perception of competence (high) and the actual marks obtained in the Grade 10 testing (very low) is difficult to understand. It is however, consistent with the findings of TIMSS 2003, that in general, the attitudes of South African learners, the vast majority of whom are African, indicated that they enjoy and value mathematics and science (Reddy, 2006). Reddy cautions that we "must consider that these may be socially desirable responses, and one would have to probe further to determine the 'real' attitudes of the learners" (p. 95).

Learner attitudes do not seem to change easily and the pattern of high agreement that maths is a high status and useful subject, and low agreement with statements related to perceived competence and confidence persist. The teacher affirmation scale seems to be an indicator of a caring teacher, and a scale sensitive to teacher absence and change.



4.2 CLASSROOM ENVIRONMENT SURVEY

The WIHIC questionnaire

As part of our pluristic approach to evaluation, a number of instruments were used to gather data. One that we used to provide evidence of the teaching environment was the classroom environment survey. This was administered at the end of each year to see the learners’ perceptions of the learning environment created by the project teacher. It was hypothesised that it would help find out if change was taking place across the grade 10 mathematics classes as the years passed and the teacher became comfortable with the new curriculum style and the classroom environment required. In the new curriculum, learner-centred instruction is advocated over traditional teacher-centred instruction. Three of the new emphases are: firstly, there is a move away from individual learning to more cooperative group work; secondly, there is a move from teacher talk/ learner passivity to teacher as facilitator and learner actively involved; thirdly, inquiry in maths where learners are expected to be more involved in investigations and active learning. Examples of this new focus are found throughout the curriculum documents (Department of Education, 2005). The question that arose was whether these changes were actually appearing in the classrooms after receiving professional development ? Given the above it was decided to use a classroom environment questionnaire. Fortunately, over the last 20 years a number of instruments have been developed, used extensively and become well established particularly in science and mathematics education (Fraser, 1998). These studies have consistently demonstrated that they can be used to determine students perceptions of their classroom environments in a variety of contexts such as Australia, Japan, Korea and India ( Zandvliet & Fraser, 1998) and South Africa (Aldridge, Laugksch, Seopa & Fraser, 2006). Consequently it was decided to use the WIHIC questionnaire rather than design a new instrument. It measures high school students perceptions of their environments across a range of scales such as teacher support and involvement, scales which could be expected to change given the emphasis placed on them in professional development. The questionnaire can be found described in many publications such as that by Fraser (1998) and the two papers mentioned above. For the purposes of this report the scales are mentioned in brief and an example provided. The six scales are Student Cohesiveness (SC), Teacher Support (TS), Involvement (IV), Task Orientation (TO), Co-operation (CO) and Equity (EQ) and are briefly described below:

SC Student cohesiveness (extent to which students know, help, and are supportive of one another)

TS Teacher support (extent to which the teacher helps, befriends, trusts, and shows interest in students)

IN Involvement (extent to which students have attentive interest, participate in discussions, perform additional work, and enjoy the class)

TO Task orientation (extend to which it is important to complete activities planned and to stay on the subject matter, high for classrooms with high stakes exams)

CO Co-operation (extent to which students co-operate rather than compete with one another on learning tasks)

E Equity (extent to which students are treated equally by the teacher)

For example the IV scale deals with perceptions of extent to which the learner has attentive interest and participates in classroom discussions. One of the eight items in this scale is “I ask the teacher questions”. Learners can then respond on a Likert scale



of Almost never 1, Seldom 2, Sometimes 3, Often 4, Almost always 5. In the current study the survey was given to all learners in grade 10 who were taught by the teachers on the programme. This was done at the end of grade 10 in 2006 and again in 2007 and 2008. This allowed comparisons to be made. Note that these are not the same learners but rather the classroom environment perceived by different cohorts of learners of each of the teachers on the programme. The control schools were also given the survey. Analysis and interpretation of the questionnaire

Below appears an analysis of the differences between the project schools and the control schools (Table 4.2). What do the mean scores represent? Each scale of the survey had eight items. Adding a learner’s response for eight items on a scale such as Teacher Support gives a total score between 8 (all eight items scored 1) and 40 (all items scored 5). These were then averaged for the cohort. Consequently, for the scales a mean of 24 would represent response of mostly “sometimes” . A score of 32 would represent mostly “often”. while a score of 28 would represent somewhere between “sometimes” and “often”. Looking at the table one can see that most means lie in this range between sometimes and often.

Table 4.2 Description of the means for control and project schools by year on the WIHIC scales

Year Sample SCT TST INT TOT COT EQT 2006 Project Mean 28.44 25.69 26.69 33.68 27.55 29.85 N 297 288 294 302 288 307 Std. Deviation 4.503 5.570 4.777 4.867 5.221 5.703 Control Mean 28.00 24.60 25.80 34.10 27.69 30.46 N 208 202 212 206 201 212 Std. Deviation 4.763 5.535 5.508 4.978 5.597 6.030

2007 Mean 30.67 27.62 27.67 33.25 28.50 30.39 N 235 228 230 235 235 235

Project

Std. Deviation 4.372 5.592 5.061 5.003 5.348 5.642 Control Mean 29.18 24.51 26.46 32.69 28.75 29.64 N 184 183 181 183 185 184 Std. Deviation 4.824 6.332 5.351 5.107 5.210 5.991

2008 Project Mean 30.02 27.28 27.25 31.74 28.19 30.53 N 213 213 213 198 196 197 Std. Deviation 4.233 5.097 4.760 6.050 5.571 6.049 Control Mean 29.46 26.49 26.54 32.55 27.28 28.45 N 228 228 228 217 216 228 Std. Deviation 4.307 5.108 5.045 5.609 5.710 9.139

In previous years, an analysis was done to see if there were differences between the control and project schools. In the first year 2006 there was a statistically significant difference between groups only for teacher support which was seen as higher in the project schools. Given that the testing was done after a year of the project this was an expected result (see Implementation report) . In the final year of the project 2007, a number of positive differences were recorded and it looked as if the teachers were establishing significantly more positive environments for the learning of mathematics than their colleagues in the control schools (see Progress report). It appears that in



2008 these differences disappeared and the original gains have decreased. However, they are still in general better than at the start and better than the control.

8

16

24

32

40

SCT TST INT TOT COT EQT

Project 2006

Project 2007

Project 2008

Figure 4.5 Change in project schools by year on the WIHIC scales Reference to the above figure shows that in five of the six categories there was a decrease in 2008. The analysis (ANOVA) reported in Table 4.3 shows only a significant difference for Equity which unfortunately on closer inspection is not practically significant. The difference only appeared significant because the control decreased. This is a disappointing result as it was expected that the project teachers would continue to grow and have even more positive environments than the control teachers as they became accustomed to using their new skills.

Table 4.3 Comparison of the means (ANOVA) for the control and project schools for 2008 on the WIHIC scales

Sum of Squares df

Mean Square F Sig.

SCT * Sample Between Groups

35.445 1 35.445 1.943 .164

Within Groups 8009.444 439 18.245 TST * Sample Between

Groups

67.993 1 67.993 2.611 .107

Within Groups 11431.640 439 26.040 INT * Sample Between

Groups

56.099 1 56.099 2.328 .128

Within Groups 10580.532 439 24.101 TOT * Sample Between

Groups

68.024 1 68.024 2.006 .157

Within Groups 14007.504 413 33.916 COT * Sample Between

Groups

86.238 1 86.238 2.707 .101

Within Groups 13061.966 410 31.858



EQT * Sample Between Groups

455.550 1 455.550 7.374 .007

Within Groups 26131.566 423 61.777

There are many reasons that can be put forward that might explain this result. The first is that the department of education has had many training courses for all teachers and provided lots of literature which emphasized the need to get learners involved and provide more teacher support etc. the very characteristics that the BEd course also encouraged. Secondly, the control schools were not stable. While we wanted the same teacher to remain teaching Grade 10 mathematics this did not happen. Examination of the table below of the six control schools shows that only two of the teachers remained the same and for three schools there was a different teacher in grade 10 each year. That would definitely contribute to the control school environments changing.

Table 4.4 Change of teacher in grade 10 class each year in control schools.

School Year (n= learners tested)

2006 2007 2008 A Teacher 1 46 2 34 3 56 B Teacher 1 41 2 29 3 28 C Teacher 1 30 2 22 3 16 D Teacher 1 23 2 33 46 E Teacher 1 52 52 20 F Teacher 1 38 18 25

Looking at the control school data over time it can be seen that some of the categories improved over the time while others decreased. It appears to be arbitrary and no pattern can be seen.



8

16

24

32

40


Control 2006

Control 2007

Control 2008

Figure 4.6 Change in control schools over the three years of the intervention in the WIHIC scale

Because of this it was decided to take an average of the control school environments and use that when discussing the individual case studies as opposed to referring to the control for each year. It is assumed that a control teacher in 2006 is equivalent to a control teacher in 2008. While this will not always hold, it does even out the problems associated with different teachers in different years. Signs of positive change

There are two positives that we can take from the classroom environment survey and they are discussed below. i) Overall project vs control classrooms Firstly if the data is aggregated into two groups for the full three years i.e. classrooms in which a project teacher taught and classrooms which were considered control i.e. non-BEd teacher, then the project classrooms have a more positive environment than the controls over the three years. In addition where comparisons are made with other classroom environments of grade 10 classrooms in other parts of the country (obtained in other research projects i.e. Hobden, 2008) these rural classrooms have equivalent and in some cases a more positive environment for the learning of mathematics than other better resourced schools.

Table 4.5 Statistical comparison by year between project and control schools for each category using t-tests

Scale SCT TST INT TOT COT EQT

All project classes 29.6 26.8 27.2 33.0 28.0 30.2



All control classes 28.9 25.3 26.3 33.1 27.9 29.5

National sample 31.7 26.0 25.5 29.9 31.0 30.0

8

16

24

32

40


Scales

Lev

el o

f ag

reem

ent

All project classes

All control classes

National sample

Figure 4.7 Comparison of project, control and a national sample for three years.

Table 4.6 Comparison of the means (ANOVA) for the control and project schools over three years on the WIHIC scales

Sum of Squares df

Mean Square F Sig.

SCT * Sample

Between Groups 170.042 1 170.042 8.160 .004

Within Groups 28403.485

1363 20.839

TST * Sample

Between Groups 732.230 1 732.230 23.371 .000


1340 31.331

INT * Sample

Between Groups 271.705 1 271.705 10.561 .001


1356 25.727

TOT * Sample

Between Groups 3.622 1 3.622 .129 .720


1339 28.073

COT * Sample

Between Groups 9.029 1 9.029 .304 .582


1319 29.709

EQT * Sample

Between Groups 174.244 1 174.244 4.068 .044




1361 42.829

It would be expected that if this component of the project intervention was effective there should appear some more differences between the normal schools and the project schools and that this change should be picked up by the classroom environment survey. There is no statistical difference when year cohorts are compared. However, when the results for the three years are aggregated, there is definite statistically significant positive changes in the mathematics classroom environment taught by project teachers over the three years of the evaluation. It can be seen from the graph above that in five of the categories the project teachers have a more positive environment and four are statistically significant. ii) Individual case studies Some interesting results are also obtained when the individual teachers are examined. Each of the cases will be discussed in detail in the appropriate chapter dealing with the case studies. Here only illustrative examples have been be given to emphasise that some of the project teachers do make a difference. What was generally found was that when a project teacher left the school and another teacher took over and taught the learners either completing the year or the following year, the classroom environment was reported as being less positive. Below are two examples where it can be seen that the green line for 2008 drops below others in most cases..

Changes and comparisons of Teacher Profile

16

20

24

28

32

36

40


Scales

Mean Initial year 2006

Intermediate 2007

Final year 2008 (Maternity)

Control schools

Changes and comparisons of Teacher Profile

16

20

24

28

32

36

40


Scales

Mean

Initial year 2006

Intermediate 2007

Final year 2008 (Left July)

Control schools

Figure 4.8 Change in classroom environment when project teachers leave

At the beginning there was little difference between the project and control schools except for the one category of teacher support. At end of the second year there was definite statistically significant positive changes in the mathematics classroom environment taught by project teachers compared to the previous year (they improved as time passed) and to the control schools (the intervention is making a difference). These changes occurred in nearly all of the categories and indicated that many if not most of the project teachers were moving toward creating positive learning environments. Unfortunately these gains did not continue into 2008 when the teachers had finished the course and graduated. In many cases the learners reported less positive learning environment. However, in general over the whole three years of the evaluation, environments did in general improve, and the project teachers created more positive environments than the



control teachers. This was further supported by the fact that when project teachers left a school or were away for a long period that the environments became more negative. Together with the overall aggregated scores this is in our opinion convincing evidence that project teachers do make a difference. However, we have found that there are significant statistical differences between the project teachers in how they have changed their classroom environments for teaching. These differences are illustrated under the next chapter which looks at the case studies of each of the project teachers in the evaluation.

Overall those teachers who participated on the programme created more positive learning environments aligned to the new curriculum than those who were in the control schools..



SECTION 5: CASE STUDIES OF THE PROJECT TEACHER

Case studies

A descriptive summary of all the project teachers is provided in the report. Below is a table summarising some of the findings for individual teachers. It is a summary of the changes in the learners’ mathematics achievement, attitude and perceptions of mathematics classroom environment over the three years. For each case study an example of the latest 2008 school visit and observation is provided together with all the contextual factors which we feel impact on the ability to provide effective teaching and learning in that school. These descriptions provide some context to the conditions of the schools and the problems associated with transferring skills to the classroom realities. These case studies also illustrate that aggregating data can often hide important findings from view. We found that there were many different consequences to the professional development training received. For example some teachers did not improve their teaching. It was obvious that the BEd was for personal development and they were not interested or able to translate this into changed practice in the school classroom. Others were able to do this and were examples of dedicated teachers implementing their new found knowledge yet there were no significant improvements in the learners’ mathematical marks. What the case study data pointed to were the reasons why this might have happened. The data also illustrated the point that it is difficult to isolate a normal case as each teacher operated in a different school context and had different personal circumstances that led them for example to seek employment elsewhere using the new degree as the enabling factor for promotion. We did find some common factors that stood out as determinants of translating professional development into improved classroom practice. Firstly these were associated with personal determination i.e. a teachers motivation and drive to make it work and secondly constraints created by the school in which the teacher operated.

Is there evidence that the quality of the project teachers’ classroom teaching has improved?

In general we found that the majority of teachers were able to translate their professional development skills from the degree course into the classroom (such that they could be considered competent qualified mathematics teachers) but this did not necessarily result in improved mathematics achievement. There was also a wide variety of factors which influenced teachers transfer of skills to the classroom most importantly being personal factors and constraints arising from the school context.

While the BEd professional development project for Mathematics involved over thirty teachers, the grade 10 teachers were chosen as the sample to study in this evaluation. The reasons for this are presented in the Terms of Reference provided by the funder for the evaluation. Below is a table indicating what happened to the teachers over the



three years. It can be seen that only six of the original 12 continued to teach grade 10 uninterrupted in 2008, the first year after the completion of the BEd course. A couple had a change in personal circumstances and were unable to teach for extended periods and three departed to other schools and one no longer teachers grade 10.

Table 5.1 Overview of grade 10 teachers involvement with project

Teacher School 2006 2007 2008

1 Amagingqi

Cafu Cafu Cafu

2 Elukhanyisweni

Williams Williams Williams

3 Eyabantu Willie Willie Willie Not teaching Grade 10

4 Ngangelizwe

Mntwini Mntwini Mntwini

5 Nozalisile

Gontshi Gontshi Gontshi

6 Ntabenkonyana

Sanqele Sanqele Sanqele

7 Templeton

Mellet Mellet MelletPromoted and doing further study

8 Josi Marele Nika Maternity leave

Nika

Nika Personal tragedies so away until 3rd term

9 Mazizakhe Ndarane Away 4 months after accident

Ndarane

Ndarane Maternity leave until 3rd term

10 Hoho Adonis Adonis Left school Replaced by Slatsha who was also

project teacher but he left mid 2008 Gobizembe Ndlaza Ndlaza Left school and went to

Jonguhlanga 11

12 Phandulwazi Mali Mali

Left in Feb Not at school. Removed from study as no useful data.

For each of the case studies, a description of the latest school visit is provided to give some context. This is followed by a summary of the mathematics results and a discussion of the attitudes to mathematics and the classroom environment created by the teacher. An overall summary of progress is given for each teacher and this is summarised in following table. Key for following table: Average performance : declining No change improving Please note that all changes are represented not only those that are statistically significant.



Table 5.2 Overall progress of Grade 10 Project Teachers over 3 years

School Mathematics Basic skills improvement Grade 10 content

Attitude to Mathematics

Class environment

Amajingqi Mr Cafu

Between 1,3 and 5% increase over each year.

Stable at around 5% which is below project &control mean

Down on all scales in 2008 and overall attitude lower than control schools

Down on four scales but still above baseline &control schools

Elukhanyisweni Mr Williams

Shown increase each year ranging from 4% to 10%

Showed good increase to 10.3 % but down in 2008 to 5.3%

Overall mean scores stable around project mean, with slight variations in some scales

Mean scores stable but remaining above average for project schools

Eyabantu Mr Willie

Mean scores in basic skills did change in any year

Mean scores stable at around 5% which is below project & control mean

Overall mean scores stable but low, with drop in teacher affirmation

Mean scores stable but low and comparable to control schools

Gobizembe Mr Ndzala


Showed increase of 3% to 8.5 % but down in 2008 to 4.6%

Stable overall, but teacher affirmation up and confidence down.

Stable, at a fairly positive level.

Hoho Mr Adonis

Between 1 and 2% increase over each year.

Stable from 2006 to 2007 then dropped by 6% to 4.3% in 2008

Attitudes were improved 2006 to 2007 and then dropped in 2008 when Mr Adonis (& Mr Slatsha) left.

Dropped from 2006 to 2007 and then dropped again in 2008 when Mr Adonis (& Mr Slatsha) left

Josi Marele Mrs Nika


Showed slight increase to 7 % but down in 2008 to 3.2%

Showed increase in 2007 but sharp decrease in 2008 esp teacher affirmation

Showed increase across all categories in 2007 but all down in 2008.

Masizakhe Mrs Ndarane

Mean scores in basic skills did not change significantly in any year

Stable at about 7.5% but down in 2008 to 3.6%

Stable overall, but perception of competence is up

Showed stability across all categories to 2007 but down in 2008 esp co-operation.

Ngangelizwe Mrs Mntwini


Gone from being stable at about 6% down to lowest project school at only 2.5%

Continuing downward trend on all scales to below baseline and control school levels

After initial improvement down on all scales but teacher support to below baseline & controls

Nozalisile Mr Gontshi


Showed big increase in 2007, down to 12%: above average in 2008 and best project school

Increased on all scales from 2006 to 2008.

Small learner numbers but increases each year

Ntabenkonyana Ms Sanqela

Shown increase each year of about 5%

Slight but steady increase each year to 10.4% mean with 6 good outliers

Increased on all scales from 2006 to 2008, notably confidence & teacher affirmation.

Only teacher to show consistent improvement in all categories

Phandulwazi Mr Mali

NA NA NA NA

Templeton Mr Mellet

Shown increase each year of about 4%

Gone down each year from a high of 15% down to 5.3%

Stable from 2006 to 2007; down on all scales in 2008 esp teacher affirmation

Initially up but 2008 down on every scale to below baseline &control schools



5.1 MR ADONIS HOHO SENIOR SECONDARY SCHOOL

Context

Note: For all cases the first part of the case describes the latest visit of the evaluators. Earlier visits are described in previous report. Hoho Secondary is a relatively small school located about 3km off the main road. The gravel road leading to this school is manageable even by a small car but can pose some problems on rainy days. The school is electrified with about 10 classrooms and an old unused classroom meant for educators to use as they wish. The principals office is sometimes used by other staff members, as observed on the day of the visit in February 2007. There is no library in the school and safety of resources cannot be guaranteed given the makeshift fence that is securing the school.

Mr Adonis moved in 2008 to another school where he was employed to teach Physical Science. He was therefore not included in the 2008 analysis, but Hoho Senior Secondary School was retained as a project school as Mr Slatsha (also a BEd (inservice) teacher) taught there until mid 2008. There are thus testing results for 2008, but no classroom observation or interview data. MATHEMATICS TESTING RESULTS

Mr Adonis' Grade 10 learners performed relatively well in the grade 10 maths content section of the test, achieving the third highest mean of the project schools in 2007, and maintaining the level of achievement in 2006 .The highest score of a Grade 10 learner in Mr Adonis’ class was 27% in 2007. In 2008, when Mr Adonis had left the school, the marks in the grade 10 content test dropped down to a mean of under 5% and no learner scored higher than 10%. The achievement in basic skills which was showing a steady increase, has also fallen back in 2008. This is summarised in Table 4.1.



Table 5.3 Grade 10 Mathematics results 2006-2008, Hoho SSS

Basic skills Mean %

Gr 10 maths Mean %

2006 November n = 47 29.3 10.4

February (all) n = 33 33.0

November (all) n = 30 34.2 10.8

February (matched) 33.9 2007

November (matched) n = 26 36.2


November (all) n = 48 28.5 4.3


November (matched) n = 47 28.9 4.3

Attitude test

year200820072006

Mea

n

1.50

1.00

0.50

0.00

0.54

0.730.67

0.48

0.79

0.65

0.900.83

0.95

0.58

0.810.83

0.32

0.68

0.54

0.27

0.370.40

school: Hoho




I see myself as competent atmaths

Figure 5.1 Mean scores on attitude tests 2006 – 2008, Hoho SSS

The learners at Hoho school were ranked sixth out of the twelve project schools in 2006 for their overall attitude towards mathematics. What stood out at his school was the relatively strong self confidence of the learners who showed the greatest agreement with the statement that they perceive themselves as competent at maths. The learners at Hoho school on average showed an increase in confidence and perception of teacher affirmation through the 2007 year which pointed to a caring teacher. There was an overall decline in positive attitude in 2008, most notably on confidence and teacher affirmation. This can possibly be ascribed to the change in teachers in that year.



Classroom environment test

The table below indicates the learners’ perception of the classroom environment they experienced when taught by this grade 10 teacher. In each case the category score can be compared to the overall score obtained from all the control school classrooms. It must be remembered that these are different cohorts of learners passing through the project teacher’s classroom in 2006 , 2007 and a new cohort in 2008. The table provides the actual scores and the graph provides a visual picture of the changes and comparisons that can be made. A score of 8 represents almost never”, 16 = “seldom”, 24 = “sometimes”, 32 = “often”, and 40 = “almost always”. For each category a statistical test was carried out to determine if the changes were significant and these are reported where relevant. These comments apply for all the teacher cases that follow and for brevity are not repeated.

Examination of the data displays indicates that in the case of Mr Adonis his classroom environment has not shown an improvement in any category while he was there. What is of significance is that it declined even further when another “project” teacher took over for 6 months. , The scores are virtually the same as the control school means. The negative class environment for 2008 can be explained by the fact that the learners would have had Mr Slatsha followed by other teachers after July when he also left which could have influenced their choices. It is assumed that the learners are showing their dissatisfaction at losing their maths teacher!

Table 5.4 Grade 10 Mathematics results 2006-2008, Hoho SSS


Initial year 2006 30.8 29.5 27.6 34.2 29.9 30.8

Intermediate 2007 29.4 28.3 26.3 31.3 29.1 29.4

Final year 2008 (Left July) 28.8 25.8 26.2 29.1 26.6 29.2

Control schools 28.9 25.3 26.3 33.1 27.9 29.5

16

20

24

28

32

36


Environment scales

Lev

el o

f ag

reem

ent

Initial year 2006

Intermediate 2007

Final year 2008 (Left July)

Control schools

Figure 5.2 Adonis: Comparison of learning environment 2006 to 2008 with control schools



Overall summary

Mr Adonis’ learners seemed to be showing very slight improvement in mathematics scores and attitude to mathematics from 2007. Since he left the school in 2008, the scores have declined. This cannot be assumed to be a causal relationship and we did not visit the school so cannot comment on contextual factors which might be influencing learner achievement and attitudes. What can be said in this case is that the project did not have a lasting positive influence on the teaching of mathematics at the school, as the slight gains initially noted were reversed once the two project teachers had left the school.



5.2 MR Z CAFU AMAJINGQI SECONDARY HIGH SCHOOL

Context in 2008

School environment Equipped with a splendid three storey face brick buildings, a detached yet smart and spacious administration and office building, Amajingqi Secondary is indeed a beacon in a township-like village situated about 2km from Adelaide when coming from Alice. The school has enough space for parking as well as some unused land for any additional extensions or upgrading. Despite its good appearance the school has difficulties. In 2007, the principal Mr. Monese said they had set up a team to "lay foundation – no basics from feeder schools". This is the "only black high school for farm feeder schools" which are 45 – 50 km from town and most learners stay on their own as the parents and guardians work on the farm. He said that there was ‘not that good support from the community” – there was a high burglary rate and the fence had been cut to allow their livestock to graze in school grounds.

Surprisingly the administration building was deserted with only the secretary on site and the notice boards hardly displayed current communication or the school timetable. We later discovered during the interview with Mr Cafu that the school had lost 4 mathematics teachers since the beginning of the third term forcing the school timetable to be juggled again, an exercise that proved difficult and had just been finalised on the day of our visit. What was even more glaring this time around was a complete absence of cars in the top car park as before, an empty staffroom, empty block of classrooms in the ground floor of one of the three-storey buildings and a steady stream of learners towards the gate of the school which was neither manned nor locked. Having waited for about 25mins to talk to Mr Cafu, who was allegedly in class waiting for a relief teacher, we managed to see him in his empty classroom. These were signs that something was amiss at this school in spite of the good buildings, electricity and fencing as well as specialist rooms such as laboratories, the home economics room and the library. Upon probing further, we discovered that the staff morale was at its lowest having been excessively blamed for the poor state of affairs at the school in spite of zero support in general and threatened with disciplinary action. A bosberaad had been promised in the following week and each teacher was expected to present a subject improvement plan. In fact the department had even threatened to change the SMT if there was no improvement or plans to revive the school.



Maths classroom environment The classroom we interviewed Mr Cafu in had about 20 desks arranged in 5 groups of four. A display of a few commercially made charts and posters from RUMEP was visible with very little in terms of teacher-made or learner-designed charts or posters. The teacher’s table has some papers and documents as well as some textbooks. The class had a secured cupboard. Lesson Observation

Planning: Mr Cafu indicated to us that he had been on sick leave since Thursday 21 August 2008 and we later discovered that this may be stress related following an extraordinary visit to the school by the Eastern Cape Department of Education (ECDoE) officials who lambasted the school management team (SMT) for this dysfunctional school and the subsequent poor matric results. As part of the school management team, Mr Cafu shouldered the blame and especially on behalf of his department, which unfortunately had lost many staff members recently. As a result, no lesson had been planned since 21 August 2008 and even when scrutinizing the grade 12 teacher file, no evidence of planning was observed in spite of noting the ECDoE school moderation instrument with scheduled dates. No comment can be made on the learners, classroom management, presentation and the level of content taught in the absence of an observed lesson. Documentation

Even though the letter advising all project participants and their principals of our visit had been received on time, Mr Cafu appeared unprepared for our visit in terms of the required documentation. The teacher file (i.e. Grade 12 file) had gaps with inadequate evidence of proper planning and application by the educator. No grade 10 mark list was available so instead we examined the grade 12 mark list. Some attempt was made by Mr Cafu to use a variety of assessment strategies in generating continuous assessment (CASS) marks for the learners by giving them 2 tests in March, 1 investigation with three tasks, 2 assignments and 1 common examination set by the ECDoE. The marks themselves were dreadful ranging from 2% to 6% in Test 1, 10% to 28% in Assignment 1 and 0% to 58% in the June Exam. The only improvement noted was in the marks for Assignment 2 that ranged from 42% to 76%. The provincial guidelines on the implementation of the NCS together with the Subject Assessment guidelines for 2007 was kept in the file but nothing else was filed for 2008. The only departmental circular observed was in the staffroom giving workshop dates for July and August 2008.



There were glaring omissions in terms of coverage given the time of the year. Topics that should have been covered in previous grades had been only been taught in grade 12 such as linear equation, linear inequalities and factorization. Notably, very little had been covered in Learning Outcomes 1 and 4 with some basic algebraic fractions and patterns (which should have been done in grade 11 at least) as well as measures of dispersion (earmarked for grade 10 really). The work covered for Learning Outcomes 2 and 3 was very limited with nothing on functions and calculus at this late stage. The learner work seen was of a relatively satisfactory standard with some factual inaccuracies that had been marked as incorrect. The teacher tended to set routine exercises and used sample marking with big ticks across a page with no comments whatsoever which is disturbing given the number of learners in this mathematics class. Routine type of questions predominantly pitched at a lower level of thinking were used. The only challenging work noted was on some Data Handling questions. RUMEP supplied the school with 8 textbooks, 4 workbooks, a board compass, a board protractor for teachers as well as 20 calculators, 142 Maths Sets, 12 Study and Master grade 10 textbooks and 45 Effective Teaching grade 12 books. Some of these had been lost but a reasonable number was still available at the school. Not many original worksheets were available except photocopies of exemplar papers, books, and some activity sheets from RUMEP. The Interview

Even though Mr Cafu was in distress he acknowledged the value added by the BEd in his teaching since he is now able to run workshops with GET mathematics teachers confidently. He feels that the course has equipped him with the necessary skills to interpret the policy documents, deal with learner misconceptions far better than before and has become more reflective. Over and above this, he has gained new knowledge especially with Probability and statistics. Mr Cafu attends the Monday afternoon workshops that RUMEP runs and has benefited a lot in topics that he was not comfortable with at the beginning. There has been some departmental intervention with a 3-day workshop in a term and some follow-up Saturdays and Holiday workshops for poorly performing schools, which is an improvement from the past in terms of support, provided to schools. He claims to be using several resources, mainly from RUMEP. The school has performed very poorly over the last four years and this has progressively worsened with less than 20% of the candidates passing matric. (In fact, a departmental document found at another school, indicates a pass rate of 10% for this school and shows it to be the worst performing school out of 46 in the district). The enrolment has declined from about 1200 learners in 1992 to less than 600 learners in 2008. Consequently, the department declared the Deputy Principal, Mr Cafu himself (as HOD) and 3 other teachers redundant in their current posts. Even though Mr Cafu appreciates what the project has done for him personally, his circumstances were too depressing and he had very little to comment on in terms of the success of the project. Instead he felt that learners are not practising on their own at home, do not attempt their homework and he has struggled to mark their work regularly. TESTING RESULTS

In 2008 the match was again very small – the school seems to be in a muddle and under pressure from the department. We tested 27 learners in February and 26 learners in October and only 6 were common. This follows the pattern noted in 2007 and could reflect shifting learner population or widespread absenteeism, or possible movement between teachers on the same grade neither of which point to a well functioning school.



Mathematics content test Learners at Amajingqi performed very poorly relative to the other project schools, obtaining the third lowest mean for the basic skills test, and the lowest mean (less than 5%) for the Grade 10 content section in 2006. The mean percentage score for the algebra section was less than 1% meaning that these learners had no idea at all about very basic algebraic procedures. This trend continued in 2007. In 2008 , the 6 matched learners at this school performed very badly achieving a mean of only 4% on the Grade 10 maths content section of the test, and a highest score of 6%. If all 26 learners who wrote in October are considered, the mean is even lower at 3.8%. This is of great concern given that most of the grade 10 learners (93 out of 140) were doing Mathematical Literacy and so these tested are presumably the more able mathematics learners. The basics GET skills mean mark improved slightly but remained under 35%.

Table 5.5 Grade 10 Mathematics results 2006-2008, Amajingqi SSS

Basic skills

Mean % Gr 10 maths

Mean % Basic skills Gr 10 maths

2006

November n = 20 28.0 4.9



February (matched) 27.5

2007



October (all) n = 26 30.5 3.8


2008


Attitude test The learners at Amajingqi also had a relatively low positive attitude towards maths when compared to learners at the other project schools in 2006. Despite the very poor mathematical performance of these learners, the mean score for a perception of competence in maths was relatively high. The 2007 testing was problematic and no November attitude data was obtained for this school. The 2008 analysis was done on the matched learners which made a very small group in this case and so the data needs to be interpreted with caution. Once again, given the very poor performance in the maths test, it is hard to understand why the 6 matched learners at this school showed the highest level of perceived competence among the project schools and also a marked increase in confidence.



20082006

Mea

n

1.50

1.00

0.50

0.00

0.44

0.60

0.370.38

0.79

1.03

0.660.72

0.28

0.46

0.150.19

school: Amajinq


My teacher affirms me

Maths is very useful

Maths is high status subject

I feel comfortable andsecure doing maths


Figure 5.3 Mean scores on attitude tests 2006 – 2008, Amajingqi SSS

Classroom environment test Mr Cafu has remained at the same school for the three years. While the classroom environment has decreased across four scales in 2008, he continues to be above the control schools and the baseline at the start of the project. It appears as if he is definitely creating a good learning environment for his learners despite the troubles in the school.

Table 5.6 Grade 10 Mathematics results 2006-2008, Amajingqi SSS

Scale SCT TST INT TOT COT EQT Initial year 2006

27.7 25.3 26.7 34.4 27.1 29.5 Intermediate 2007

33.0 30.8 31.54 37.07 28.67 32.4 Final year 2008

30.8 27.7 28.2 33.5 29.6 32.8

Control schools 28.9 25.3 26.3 33.1 27.9 29.5



16

20

24

28

32

36

40


Environment scales

Lev

el o

f ag

reem

ent

Initial year 2006

Intermediate 2007

Final year 2008

Control schools

Figure 5.4 Cafu: Comparison of learning environment 2006 to 2008 with control schools

Overall summary

Amajingqi Secondary is a school that faces serious challenges with a dwindling number of staff from 31 in January 2008 to 22 in August 2008 with a possibility of further redundancy declarations. The staff shortage has resulted in Mr Cafu teaching two Grade 10 maths classes as well as five more classes in different grades and subjects. This is clearly too big a load for one person who is also expected to manage a department successfully while providing academic leadership to his colleagues as well. This is an interesting case as Mr Cafu has managed to maintain a good (above control school level) learning environment in his classes despite troubles of morale etc. within the school. However, this has not translated into improved Grade 10 mathematics results, although the basic skills have improved from February to October/November in each year of testing. This affects the attitude of his learners whose perception of their maths competence and confidence is justifiably low. His learners, are not average, close to agreeing that he affirms them in their mathematical endeavours. It is an indicator that there are other factors that are preventing the marks improving. They do not depend only on a well qualified teacher and supportive learning environment. It would appear that Mr Cafu has the ability to produce a good learning environment but other factors interfere resulting in him not applying what he has learnt from the project.



5.3 MR GONTSHI NOZALISILE SECONDARY SCHOOL

Context based on 2008 visit

Mr Gontshi started teaching Biology and Agricultural Science with a Bachelor of Science degree but no education qualification. He began teaching Grade 10 Maths from about 2000. He did not qualify for the B.Ed (inservice) course due his lack of teaching qualification, and eventually (by special arrangement with Zenex) was included in the school support part although he was dong a PGCE through Rhodes. He abandoned the PGCE and continues to teach unqualified.

The school consists of two rows of classrooms – at the end of the lower row, is the burnt down storeroom (burnt down by two learners). All the books and computers were destroyed. Some lessons take place outside due to the burnt building. The Principal indicated that there is trouble with the community about the appointment of the administrative assistant.

Maths classroom environment There are learners desks and chairs and a chalkboard in the class. No RUMEP charts seen on walls; there were hand written summaries done by teacher on “Functions” (algebraic graphs only) stuck on the walls near the chalkboard.

Lesson Observation

Only recently compiled files were available – Mr. Gontshi’s other file burnt. There was no lesson plan presented for the lesson observed – only the activity used in lesson, and the homework (1 example only!) was orally dictated to the learners. Class management was not a problem in this small class of only 9 learners who were very attentive and worked well on in their small groupings on the worksheet used. Presentation: The lesson presentation consisted of a short introduction and then each group was given the same worksheet. Each group was told to work on different example from A – D on worksheet – told they would do a group presentation on this at the end of



lesson. No introductory example on chalkboard; no heading; no listing of key concepts on chalkboard; overall there was no development of the content at all. Mr. Gontshi walked from group to group work checking and explaining when learners asked questions. Reflection about various axes was the focus of the learner activity for this lesson, but Mr Gontshi’s instructions asked the learners to find the “translation code” as done in the introductory example when recapping – but learners are dealing with reflections! This led to a fair bit of confusion. The learners were neatly dressed in full school uniform. They engaged well with the activity – the groups were small so all learners could see worksheet and all got involved in working out solution. Learners discussed mostly in Xhosa and their written explanations of solution were done in good English. The learners did show some difficulty in the oral explanations during the group presentations later. Documentation

Mr Gonthsi had a teaching file filled with notes on various topics – mostly Gr. 12 work. He has the DoE forms for marks but not filled in. He said that he had obtained copies of tests, etc from learners after the fire. He has a departmental circular for marks. With respect to curriculum coverage, it was noted that some parts had been covered in detail with numerous examples e.g May – 23 July work on algebraic graphs (linear and quadratic); while others seem to be one day sections e.g 20 July: financial maths; 4 August: substitution in algebraic equations; 5 August went onto exponents; 7 August back to functions. At this stage Mr. Gontshi had covered all assessment standards for Grade 10 in L.O.1 (in 1st term with financial mathematics going into 2nd term); had only done the sketching algebraic graphs from L.O.2 (in 2nd term) and had covered the coordinate geometry content (as shown in learner books: 11 August & 19 August). He was busy with transformations at the time of the evaluation visit. No work had been done on L.O.4 and very little content covered from L.O. 2. There was a reasonable amount of work in the learners’ books, regularly marked but with few explanations of why things were wrong. All learners appeared to have textbooks. The Interview

On being asked abouthte RUMEP programme the reply was “Amazing”. He said it had given him much confidence in most of mathematics topics and given him ideas of ways of approaching certain areas of curriculum. Even though he was not part of the RUMEP project, he was still happy to be guided by the RUMEP visitors. Thomas is still visiting and gives materials from the previous years. He attends the DoE workshops – these



were only for Grade 12 though. Many of the RUMEP resources, for example all the calculators, were burnt and have not been replaced. The school has a programme with Gr. 8 & 9 learners focussing on basic maths – this Grade 10 group is the first group coming through this programme. For this reason, Mr Gontshi said he expected better results from this group of 11 Grade 10 learners. He feels that should the project not succeed, it will be due to teachers not being confident and rushing through (or overlooking) certain sections. Mr Gontshi involved in a district workshop for Grades 10 -12 in the following week: he will assist with trigonometry – trying to bridge these gaps for learners. TESTING RESULTS

Mathematics content test Mr. Gontshi's Grade 10 learners appear to have made very good progress throughout 2007. The three matched learners improved in basic skills from a mean of 33% to a mean of 60% and their Grade 10 content mean of 31.7% was way above the average at any other school tested. The results were not so impressive in 2008 and while the basic skills improved somewhat, the Grade 10 content dropped markedly, but is still well above the mean for the project schools. This was in fact the best performing project school.

Table 5.7 Grade 10 Mathematics results 2006-2008, Nozalisile SSS

Basic skills Mean %

Gr 10 maths Mean %

2006 November n = 23 35 8.9




2007



November (all) n = 10 42 11.8


2008

November (matched) n = 10 42 11.8

Attitude test Mr Gontshi’s learners express high mean positive attitudes to mathematics. Over the ears, the perception of competence has increased, and the mean confidence level in 2008 is high. Consideration of Figure 5.5 shows that the attitudes are becoming more positive over the years which is a pleasing result.



200820072006

Mea

n

1.50

1.00

0.50

0.00

1.02

0.80

0.68

0.850.83

0.65

1.441.36

1.201.111.14

0.86 0.85

0.17

0.55

0.75

0.38

0.09

school: Nozalisi





Figure 5.5 Mean scores on attitude tests 2006 – 2008, Nozalisile SSS

Classroom environment test Comparisons between the different years is difficult for Mr Gontshi due to the fluctuation in numbers of learners in his mathematics class. For example they changed from 23 in 2006 to 3 in 2007 and 10 in 2008. However, it does appear that in 2008 he has managed to create an improved classroom environment with his ten learners. Overall it has improved from the previous years particularly in teacher support (which was low in 2007 despite just 3 learners!) and is well above the controls.

Table 5.8 Grade 10 Mathematics results 2006-2008, Nozalisile SSS


30.4 26.9 27.8 33.4 29.5 31.0

Intermediate 2007 32.0 20.0 23.3 29.0 26.3 32.3

Final year 2008 31.7 28.7 31.0 33.8 30.9 32.6

Control schools 28.9 25.3 26.3 33.1 27.9 29.5



16

20

24

28

32

36


Environment scales

Lev

el o

f ag

reem

ent

Initial year 2006

Intermediate 2007

Final year 2008

Control schools

Figure 5.6 Gontshi: Comparison of learning environment 2006 to 2008 with control schools

Overall summary

Nozalisile school looks in a ‘sad’ state with the burnt staffroom where all books and resources were stored. Learners are all well dressed in full school uniform and overall it seems to be a functioning school despite the initial outward appearance. Mr. Gontshi teaches Grade 10 & 12 mathematics and Grade 10 -12 Agricultural Science. He seems to be over-commited in his teaching responsibilities to the detriment of his own studies – he abandoned his PGCE due to the work and travelling it entailed. Although he has no teaching qualification, the principal is very happy with Mr. Gontshi’s diligence and commitment. Mr. Gontshi is involved in Winter School and on Saturdays. The principal conceded that he had a big load and that some had now been removed. Mr Gontshi has good content knowledge from his initial degree and although he did not have the benefit of a formal programme he did have the RUMEP school support and went to the workshops on occasion. Overall he appears to be one of the success stories of the project as his classroom environment is perceived very positively by learners, they have good attitudes and relative to others his marks albeit by a small amount are improving. Since he did not do the formal programme, we can conjecture that his success (best performing project school by our measures) and his exceptionally good utilisation of the RUMEP materials (RUMEP fieldworker report) is due to the classroom support he received, and his strong academic background.



5.4 MR MELLET TEMPLETON HIGH SCHOOL

Context in 2008

Templeton High is a well established school with excellent infrastructure. The school is from Gr. R – 12. Overall it is a well managed school with good administrative support. The principal was not at school on the day of the evaluation visit – at a departmental meeting.

Maths classroom environment Mathematics is taught in the Science lab so it is difficult to have group work (RUMEP staff have also recommended using another room for mathematics teaching). There is a chalkboard, an OHP and several wall charts, some of which have been created by the teacher himself. For example, he has made summary charts of geometric transformations, the parabola graph with key mathematical terms explained in 3 different languages. The adjoining stockroom is used for storage of mathematics materials as well. Lesson Observation

Mr. Mellet taught a lesson on algebraic functions. The topic was y=ax2+q (“Influence of changing the value of q on translation”). A typed lesson plan and accompanying worksheet for the lesson were presented. Each learner had their own worksheet and square paper was provided for each learner as well. Individual work was encouraged in this lesson, particularly when using resources– Mr. Mellet does not allow borrowing; he goes to laboratory stockroom to get extra rulers and colour pencils. He teaches in a clear strong voice, but generally does not insist on learners’ full attention before discussing examples. The class of 30 (out of 38) was quite noisy, but all were busy discussing the work. Mr. Mellet walked between benches, checking on progress and discussing with learners – overall, a rather hurried pace in this lesson. Generally, Mr. Mellet seemed stretched trying to respond to the many calls of “Meneer”; “Pappa” ; “Jy gaan te vinnig” and some calling “Ek is klaar”. When he is at the chalkboard, Mr. Mellet does try to get whole class attention for important points of discussion, but habitually accepts chorus answers – in this lesson, he never questioned individual learners. Mr. Mellet switched between Afrikaans and English throughout lesson, but taught mostly in English. The headings and subheadings for the lesson were written in English and Afrikaans.



The Maths content was appropriate for the relevant assessment standard in Grade 10. With his own understanding of the NCS improving, Mr. Mellet made a very good effort to integrate across learning outcomes in Maths – he had taught geometric transformations ahead of this lesson this year. He continually emphasised important concepts like domain, range, maximum and minimum values. Self-made wall charts with important concepts summarised are in three languages: English, Afrikaans and iXhosa. On this day there were 30 of the 38 learners present. They were all well dressed, polite and seemed very relaxed in Mr. Mellet’s presence. They were a rather noisy class but all actively involved working off the prepared worksheets in this lesson. They are clearly at different levels and quite demanding of Mr. Mellet’s attention. Documentation

Only long term planning was available – compared to 2006 and 2007 when short term planning was visible in terms of weekly plans. The long term plan was devised by Mr. Mellet. Spreadsheet of marks covering all assessment instruments were available. Poor performance of Grade 10’s: top CASS mark 47%; top exam mark 48%. Subject policy document and assessment guidelines for grades 10 and 11 in Mathematics were presented and a circular on departmental workshops for the year and a circular on the criteria for setting and assessing projects in Mathematics were evident. Some effort had been made to keep up with the pace of the curriculum with evidence that work from L.O.s 1, 2 and 3 had been covered. Learners have filled about half of a double quire book, but no L.O. 4 as yet. Work was not substantial at all - the basics are being covered and challenging work was being avoided. .The context painted was one of very weak learners, not ready and capable of Grade 10 work and with deep-rooted deficiencies. It seemed that there had been no individual chance for marking so there was only a tick at the end of the class work and no visible corrections by the teacher or learners. The June exam marks recorded show a low level of achievement. Learners buy their own Classroom Mathematics textbook and only about 20 have textbooks. The learners had RUMEP printed Cartesian planes for transformations and graphs. The Interview

Mr. Mellet was very welcoming and extremely positive about the programme. The resources and the support received from RUMEP stand out for him – he still uses the resources and some of the work done on the programme. The programme has made him “open” for learners’ problems and to identify misconceptions. He is more sensitive to the pace of the learners – this also has an influence on other subjects he teaches. Reflection is now a habit of mind. He is also now open to change his teaching programme e.g. he taught geometric transformations (L.O.3) before the parabola this year.



Mr Mellet visits RUMEP often (he is doing B. Ed Hons now) and sees staff and is given resources e.g. Thomas has given him notes on his pen drive and Mr. Mellet uses these in his lessons. He attends worksops every 2nd Monday from 3pm – 5pm in Fort Beaufort.. Mr. Mellet was one of three teachers sponsored by Zenex to attend the AMESA conference. He uses the small set of 10 Study and Master textbooks(from RUMEP) for groupwork in the class as very few learners have the Classroom Mathematics textbook. Mr mellet indicated that the learner achievement was much worse this year – he gave a baseline test and only 3 learners from 38 got 30%; others from a group of 80 Grade 10s got less than 30%. Some learners have moved to the maths Literacy class. He feels many learners are not ready for Grade 10 (age was still a condonation factor in Grade 9 in 2007; no longer so from 2008) If the project fails to improve quality of mathematics teaching and learning, Mr. Mellet said it would be due to the quality and attitude of learners in Grade 10. He said they seemed to lack an inner drive to excel (maybe due to socio-economic conditions) and the condonation on age (learners proceed without much effort). Also there is a lack of parents’ support. 2008 Testing results

Mathematics content test Mr Mellet’s class achieved the second highest mean score in the Grade 10 content section in 2007 although the mean was down from 2006. The highest score for a learner was 23% but individual high scores in LO1 and LO4 were in the low 40s. The basic skills improved slightly from February to November 2007. The situation in 2008 was disappointing. The mean score on the Grade 10 content section was just over 5% but the highest individual score was 32% which is considerably above average.

Table 5.9 Grade 10 Mathematics results 2006-2008, Templeton HS

Basic skills Mean %

Gr 10 maths Mean %

2006 November n = 12 47.1 14.8


November (all) n = 16 45.6 11.2


2007


2008 February (all) n = 42 31.7

November (all) n = 17 40.0 5.35



Attitude test The mean agreement in Mr Mellet’s Grade 10 class with the composite statement I have a positive attitude to maths was the highest mean agreement of all the project schools in 2006 reaching the agreement level. In 2007 there was high agreement on the teacher



affirmation scale, and an improved perception of mathematical competence. This positive trend was reversed in 2008 and the Grade 10 cohort of learners taught by Mr Mellet that year agreed far less, on average, with all scales.

year200820072006

Mea

n

2.00

1.50

1.00

0.50

0.00

0.68

0.951.00

0.45

1.301.19

1.381.41

1.73

0.860.94

1.22

0.29

0.700.73

0.06

0.30

0 02

school: Templeto

I have a positive attitude to mathsMy teacher affirms meMaths is very usefulMaths is high status subject



Figure 5.7 Mean scores on attitude tests 2006 – 2008, Templeton HS

Classroom environment test Mr Mellet appeared to be one of the success stories when his class was examined in 2007. Everything was improving and learners had a very positive attitude toward the classroom environment he created. However, in 2008 this appears to have changed. The environment on most scales has dropped and is now below that of the control schools. When his personal circumstances are considered it becomes easy to suggest why this has happened and is discussed below.

Table 5.10 Grade 10 Mathematics results 2006-2008, Mellet


Environment in 2006 29.6 28.5 30.6 34.6 28.5 30.0

Environment in 2007 33.3 31.6 29.6 34.4 31.8 34.3

Environment in 2008 29.3 22.5 24.7 32.9 28.1 28.0

Control schools 28.9 25.3 26.3 33.1 27.9 29.5



16

20

24

28

32

36


Environment scales

Lev

el o

f ag

reem

ent

Environment in 2006

Environment in 2007

Environment in 2008

Control schools

Figure 5.8 Mellet: Comparison of learning environment 2006 to 2008 with control schools

Overall summary

Mr. Mellet teaches one Grade 10 Mathematics class; Grade 11 ML and Grade 12 ML and Physical Science for Grades 10 – 12. He is furthering his studies (BEd Hon) and has also been promoted to Deputy Principal at Templeton. While he states that he has become more confident in providing leadership in mathematics, his administrative workload has increased and this is likely affecting the time he has to spend on dealing with learners’ difficulties. He values the “RUMEP” way but circumstances seem to be overwhelming him. It seems obvious that he benefited a lot from the project and his mathematics teaching was improving by all our measures. The result was that he was given more administrative responsibilities in the school and also decided to study further. As a consequence we hypothesise that he has not had the time to focus on his teaching and consequently learners are reporting less positive environment and his learners marks are suffering. To us this appears to be a case of the project having the desired effect of improving teacher’s capacity ‘but in the long run personal goals and school pressures have taken the person away from mathematics teaching.



5.5 MRS MNTWINI NGANGELIZWE SECONDARY SCHOOL

Context

School environment Ngangelizwe is a relatively small school with only three blocks of buildings. The school utilises five classrooms only for teaching purposes plus a computer room due to their enrolment, which stands at 190 supported by a teacher complement of 6 including the acting principal, Mrs Mntwini. All the teachers travel in daily – along same route we followed from Breakfastvlei turn-off. Transport leaves school daily at 2.30pm. The principal’s office, which we used for the interview, is also used as a storeroom for high-tech equipment (such as new computers) and this is where the secretary is also housed.

The school has a computer room with 18 computers. On the day, we observed computer sessions facilitated by 2 tutors who informed us that teachers send learners to the computer room according to a set timetable daily. Incidentally, there were three new computers still awaiting connection in the principal’s office. Maths classroom environment Having confirmed our visit with Mrs Mntwini telephonically the day before, we anticipated a rather unproductive visit from her response. Indeed when we arrived, we were reminded that she was rushing off to an interview for a principal’s post and as such could only talk to us for half an hour. Consequently, no mathematics class was observed at all. The photograph shows her classroom with other learners. Documentation

No teacher file was provided as evidence of planning for the year. However, based on the written summary from Thomas about each participants, it appears that the Grade 12 teacher at Ngangelizwe is unhappy at the pace at which these learners are being taught and work they cover at the end to an extent of hoping to claim back the Grade 10 mathematics class from Mrs Mntwini, as she is always busy with management



issues at the expense of the learners. No grade 10 marks provided to evaluators. Only the grade 12 mark schedules were displayed in the principal’s notice board possibly given that the CASS moderation workshops were running in the area that week. As a norm, the departmental circulars, workshop dates, school meeting dates and timetables are displayed in the principal’s office at this school and it appears that teachers do have access to them given that this office is used by the secretary as well. According to Mrs Mntwini, she has taught conversions of decimals (all types) to fraction form, approximation of surds to some rational number and an investigation of number patterns leading to generalizations in LO 1. In LO 2 she has taught factorization, solving linear inequalities (no mention of linear equations here) and graphing a linear function. For LO 3 she mentioned Polygons, Measurements (Scale Drawing in particular) and has just started basic Analytical Geometry. Data Handling and Probability in LO 4 will be tackled in term4. Based on her admission and from what was mentioned earlier by Thomas, these learners are far behind their work. Mrs Mntwini claims that learners are not motivated, homework not done in most cases, and some are not even attending the extra classes she runs for 1½ hrs on Mondays, Tuesdays and Thursdays. The class work in the learner books is marked regularly but in some cases incorrect responses were marked as correct such as in number patterns where we saw 1 – 3 > 2; 19 – 19 > 2; …; 150 – 201 - > 2 leading to a confusing general rule y = 2n + 1 (all of this was accepted as correct with huge ticks across the books). By and large straightforward exercises were common with some interesting investigations now and then, for example in Space and Shape where the learners investigated types of triangles and area. RUMEP supplied the school with books and calculators which are kept in a safe place and only used when required. The teacher uses a lot of photocopied RUMEP activity sheets and textbook exercises. Nothing original was presented to us. The Interview

Mrs Mntwini still values the course and its impact in improving teaching in general. However she appears distracted in her mathematics teaching by the additional administrative duties coupled with her excessive workload of teaching other subjects and grades (six in total). She acknowledged that in the past she taught what she wanted to teach and what she was capable of teaching without taking due consideration of the curriculum. Since she joined the project, she now works closely with the NCS document and the Assessment Standards in particular. In her view, the pace setters from RUMEP are also useful and more current and relevant than the ones from the department Mrs Mntwini made no mention of school visits by Thomas but in the RUMEP portfolio Thomas had noted some difficulty in reaching her ever since her role changed in the school. However she did admit that RUMEP had done all they could to support them over the years. The same cannot be said for the DoE though in her view who appear to concentrate on grade 12’s through CASS moderation workshops, discussing exam paper structures, etc and not dealing with approaches to teaching specific topics in mathematics for example.



The RUMEP resources are kept in a strong room. Learners take textbooks home, some are lost, parents informed but generally they are replaced. She uses a lot of worksheets from RUMEP in her teaching. The learner results not pleasing at all but everyone is working round the clock to improve them. A meeting with parents was planned for the following week to discuss how to motivate learners to attend extra classes. Project success Mrs Mntwini was of the opinion that the project had been successful in motivating participating teachers but work needed to be done in project schools to encourage more teachers to get involved so that everyone can talk the same language within these schools. As it is she is the only one in the project and the grade 8 & 9 teachers are not interested in her ideas and suggestions. TESTING RESULTS

Mathematics content test

Table 5.11 Grade 10 Mathematics results 2006-2008, Ngangelizwe SS

Basic skills Mean %

Gr 10 maths Mean %

2006 November n = 39 27.1 5.1




2007


2008 February (all) n = 34 28.4




Mrs. Mntwini's Grade 10 learners in 2007 performed very poorly in the mathematics test, achieving a mean score of just under 6% for the Grade 10 content (little improvement on the 5% in 2006), and around 30% for the basic skills section. The highest score for the Grade 10 content section was 13% and the mean for LO4 (data handling and probability) a mere 1%. These were some of lowest results among the project schools. In 2008, Mrs Mtwini’s learners obtained the fourth highest project school mean score in the basic skills in February, and yet the lowest mean score in the Grade 10 test. They did however improve markedly in the basic skill over the year. This could suggest they were doing “below grade” work. In the interview Mrs Mtwini indicated that at the school no one takes Mathematical Literacy because of a shortage of teachers at the school. This would result in very weak learners trying to do the mathematics when they would be better suited to the less academic option. Attitude test Mrs. Mntwini's Grade 10 learners showed among the least positive attitude towards mathematics among all the schools tested in 2007 and the mean of the matched learners decreased on all scales. The downward trend continued in 2008 and the agreement with all scales was very low. There remains a relatively strong agreement though that maths is a high status subject and useful.



year200820072006

Mea

n

1.50

1.00

0.50

0.00

-0.50

0.300.360.39

0.300.310.40

0.73

0.52

0.81

0.620.500.50

-0.05

0.310.27

-0.01-0.09

-0.26

school: Ngangeli





Figure 5.9 Mean scores on attitude tests 2006 – 2008, Ngangelizwe SS

Classroom environment test Mrs Mtwini classroom environment surveys are difficult to understand. There has been significant improvement of three scales over the baseline. For example the teacher support reported by the learners each year has improved. This is very positive. However, she has declined over the years on three scales and in most cases is below the control scores. For example many learners report that they are not treated equally by the teacher and not task orientated. This is conflicting with the teacher support reported. However because five of her scales are below those of control schools there is cause for concern. Overall other than the increasing teacher support, it appears that her learners do not perceive the learning environment as positive.

Table 5.12 Classroom environment 2006-2008, Mtwini


26.5 26.4 25.9 30.2 25.7 28.2

Intermediate 2007 28.4 27.3 26.7 30.9 26.9 27.7

Final year 2008 28.1 28.3 25.4 28.0 26.5 27.2

Control schools

28.9 25.3 26.3 33.1 27.9 29.5



16

20

24

28

32

36


Environment scales

Lev

el o

f ag

reem

ent

Initial year 2006

Intermediate 2007

Final year 2008

Control schools

Figure 5.10 Mntwini: Comparison of learning environment 2006 to 2008 with control schools

Overall summary

Ngangelizwe appears to be a school confronted with many challenges that include relatively small enrolment, shortage of teaching personnel, excessive teacher workloads, poor results across the grades, apathetic learners and a non-existent Senior Management Team. The fact that the acting principal also teaches Technology in grades 8 and 9, Physical Science in grades 10 to 12 and Mathematics in grades 10 is an indication of how dire the situation is here. The poor results at the end can be attributed to a completely overwhelmed set of teachers trying to do their best under difficult situations. The standard of Grade 10 mathematics at this school is extremely poor and the overall attitude of the learners towards mathematics among the lowest of all schools tested. This appears to be a case of a teacher who through reasons that are not clear has been unable to translate her project involvement into positive changes in the classroom and improvement of mathematics results.



5.6 MRS NDARANE MASIZAKHE SECONDARY SCHOOL

Context

Masizakhe is a deep rural school set in the hills outside Fort Beaufort. The actual school is a neat freshly painted building and the principal’s office is well equipped with a computer and photocopy machine.

The school seems to be on a decline with shrinking pupil enrolment and poor Grade 12 results. A departmental initiative to provide transport into the town for school has led to many learners electing to travel each day rather than to stay in their small village. Despite the evidence of new grave yards near this village, Ms Ndarane does not attribute the troubles at the school to the HIV AIDS epidemic. A departmental circular on the will reports the news that this school was ranked 44 out of 46 schools in the Fort Beaufort district in the Grade 12 exams with just 3 of the 18 learners who wrote managing to pass – a pass rate of 17%. In accordance with a research decision, this was considered a control school for the 2008 testing since the project teacher had been away from the school for the first two terms. Maths classroom environment The classroom was clean and spacious but seemed bleak due to the small number od desks and learners (just 7). There was a broken wooden cupboard, and two charts of graphs on the wall. The room is well lit with fluorescent lighting.



Lesson Observation

The lesson was on quadratic graphs. There was a simple lesson plan but it was directly related to the activity being done. The class was well managed ( easy task with just seven) and the learners were seated in two groups. Each group was given a different graph to draw to facilitate later comparisons. The learners were given RUMEP axes to use to draw the graphs, and the actual worksheets were taken from the Malati materials. Mrs Ndarane presented the lesson in a clear voice and had a pleasant classroom manner. She moved between the groups to keep them on track. She made good use of the chalkboard to build up a summary of the work as she went along. The Maths content of this lesson was directly and explicitly related to the NCS assessment standard. The learners were drawing a series of quadratic graphs to see the effect of changing values of the coefficient of the leading term. Mrs Ndarane did well to link this new work back to the more familiar straight line graphs, and to pick up the need to plot extra points to check the correctness of graphs. I thought she showed good classroom skills. The learners needed a lot of prompting to see the pattern, but she did well to summarise the results. There were just seven learners, and only two girls. One of the boys was very small and appeared to be under age. Checking the data collected in the testing showed that there is indeed a boy of 14 in this class. Typically a 14 year old is in Grade 8 so this is unusual. Documentation

Mrs Ndarane was on maternity leave for most of the first two terms and she returned to find that her replacement had made very little progress through the curriculum. There are no marks in the mark book prior to her return in mid May. The curriculum documents are in the storeroom along with copies of textbooks. Departmental circulars are displayed on the office wall. The learner books indicate quite clearly how the year has unfolded for these learners. The first

part of the books has very sparse work and very low level – very basic algebra that seems about Grade 8 level. As soon as Mrs Ndarane returns (15 May), the amount of work increases dramatically with work done almost each day, and more appropriate to Grade 10 level. The work is regularly marked and errors sometimes clearly pointed out. Learners appear to work from textbooks and not worksheets. The Interview

Mrs Ndarane spoke to us after the lesson. Although the formal RUMEP course is over, the materials are still in use. As with other project teachers her 2007 planning file has been carried over to 2008 and she is mostly working from that. She told us how the other staff members are also using the materials she got from RUMEP – the Maths



teachers are using the content, the English teacher is using the idea of getting learners to write reflections ands the lesson plan template provided by Thomas has been taken by the principal as an example. The BEd programme has made a difference to her teaching in that she was given a Maths class (which she really wanted) on the strength of being in the programme. She has no prior Maths teaching experience to compare with. Mrs Ndarane assured us that the RUMEP resources provided over the years of the BEd are safely in the school somewhere. The textbooks were seen in a stockroom but not eh equipment. When asked to comment on the marks expected from his Grade 10 learners in our testing later in the year Mrs Ndarane said “I am afraid..” She realises how far behind the learners are and in July she had to set a Grade 9 revision test instead of a Grade 10 test. (In the event the mean score was 3.6% with the highest mark being 6% and the lowest 0%). When asked about factors that could impede the success of the project, Mrs Ndarane mentioned the lack of motivation of the learners who were not availing themselves of the extra classes provided in town for them, and frequent absenteeism. A departmental imitative to bus learners into town had effected the enrolment at the school. She would like to extend help and encouragement to the neighbouring primary school as no maths is happening there. This results in leanres coming to the secondary school with very little maths background. 2008 Testing results


Table 5.13 Grade 10 Mathematics results 2006-2008, Masizakhe SS

Basic skills Mean %

Gr 10 maths Mean %

2006 November n = 9 31.7 7.2




2007


2008 February (all) n = 8 29.4




Mrs Ndarane's 2006 and 2007 learners performed at about the average level for the schools tested in both the Grade 10 content section of the mathematics test, and the



basic skills section. In 2008,, the performance on the Grade 10 work was low as she had expected after her absence earlier in the year. Attitude test

year200820072006

Mea

n

1.50

1.00

0.50

0.00

0.67

0.54

0.68 0.650.63

0.83

0.97

0.870.840.74

0.520.53 0.53

0.630.56

0.300.22

-0.03

school: Masizakh





Figure 5.11 Mean scores on attitude tests 2006 – 2008, Masizakhe SS

The mean agreement in Mrs Ndarane's class with the composite statement I have a positive attitude to maths has decreased from 2006 to 2008. The increase in perception of mathematical competence over the years is not justified by the testing scores. Classroom environment test Based on the small 2007 improvements to the classroom environment it was expected that her scores would remain stable or increase slightly in 2008. This was not to be as she was not present for the first six months. When examining the graphs, it is of concern that the scales of teacher support and learner involvement have decreased to levels below the baseline in 2006. These are fundamental scales and indicate that the classroom does not have the environment encouraged by the new curriculum guidelines. In addition the scale of cooperative classroom has decreased dramatically even below the controls which is in direct contradiction to what is suppose to be happening. These changes can be explained by the fact that a different teacher took over while she was away on personal leave or alternatively the learners did not have a teacher. What is positive is that the classroom environment remains more positive in five of the six scales when compared to the control schools.

Table 5.14 Classroom environment scales 2006-2008, Ndarane


31.0 28.2 28.4 33.3 26.0 32.8

Intermediate 2007 30.9 27.9 30.2 33.9 28.0 30.5

Final year 2008 (Maternity) 31.8 27.2 26.8 34.2 21.8 30.6

Control schools 28.9 25.3 26.3 33.1 27.9 29.5



16

20

24

28

32

36


Environment scales

Lev

el o

f ag

reem

ent

Initial year 2006

Intermediate 2007

Final year 2008 (Maternity)

Control schools

Figure 5.12 Ndarane: Comparison of learning environment 2006 to 2008 with control schools

Overall summary

The learners at Masizakhe school are performing below average for the project schools and show weakening positive attitudes to mathematics. This is a reversal of the positive sentiments expressed by learners in 2006. The evaluation teams did not visit the schools during the period between February and November 2007 to observe teaching and so we can offer no explanation for this. The downward trend continued in 2008, influenced no doubt by Mrs Ndaranes’s absence on maternity leave and the dwindling numbers at the schools. She seems committed to working hard to get the learners back on track. It appears that Mrs Ndarane was gaining a small amount from her involvement in the project although this did not translate into significant improvement of mathematics marks. Overall the classroom environment data for this case suggests that when a teacher is not present the classroom environment reported by the learners becomes more negative. While not convincing evidence it does indicate that the project teachers do make a positive difference.



5.7 MR NDZALA GOBIZEMBE SENIOR SECONDARY SCHOOL/ JONGUHLA / MIZAMO HIGH SCHOOL

Context

Mr Ndzala has had an unsettled professional life since we last visited him in February 2007 when he was at Gobizembe High School. He had difficulties securing his salary and moved to a school in Kidds Beach early in 2008. By the time we visited in September 2008, he had moved again to a school in Mdantsane. He secured a permanent post here under the scheme to provide one extra maths teacher to each school. He was however, allocated Physical Science classes to teach. He had only been at this school for 6 weeks when we visited. The previous Science teacher left in February so there was no official handover to Mr Ndzala, who was landed with learners with no portfolios and very behind in their work.

Mizamo High School seems quite new and is a modern design with separate blocks of three classrooms in a cottage style with verandahs and covered entrances. The classrooms are bright with good lighting and lots of windows. We met the acting principal (there has been no appointed principal for three years) who consulted the timetable on the wall to see what time the next lesson would begin, and found it should have started 10 minutes ago! There are problems at this school and the DoE visited recently and came down hard on everyone. The level of discipline and learner control on the day seemed very low. In the classroom block opposite where we were, learners were running in and out of the classrooms unsupervised until a teacher came across from another block and proceeded to cane all the learners who did not run off home in time! Maths classroom environment The lesson we observed was specially arranged (Mr Ndzala does not have a Grade 10 mathematics class) and the 13 learners were seated two to desk. There were no spare desks indicating that the desks might have been brought in for this lesson. Lesson Observation

It must be noted that this was a “once-off” lesson for our benefit as Mr Ndzala was not teaching mathematics at that school. Consequently he had no long term planning nor learner work to show us, not even for his Science classes. (He said that his planning at previous schools had been left in the handover tot the new teacher there). He had planned a section on Co-ordinate Geometry envisaging 12 periods of 45 minutes. There



was, however, no clear plan for this particular lesson which resulted in far too much information for a single lesson, and a lack of focus. Classroom management was good (helped by small class size) and the required resources (maths sets) were available for use. Learners worked in pairs and Mr Ndzala gave good feedback and support as he moved form desk to desk. The lesson was well presented with a good real life introduction to the co-ordinate system (the GPS function on cell phones) and reference to the historical work of Descartes. He has a clear teaching voice and interacted well with the class. The mathematical content of the lesson was problematic. The initial work on locating and identifying points on the Cartesian Plane (using RUMEP grids) is well below Grade 10 level and the pace was very slow. This work was done in pairs where as simple revision it should have been a quick individual activity requiring little discussion. Then there was a huge leap to discussing the distance formula which depends on Pythagoras’ Theorem. Learners denied knowledge of this theorem (covered in GET) and lengthy and involved explanations were given completely losing the learners. While this is somewhat excused as it was an unknown class, better foresight and planning would have had contingency plans in case learners lacked the required background knowledge. The learners in the class we observed were neatly dressed, well behaved and spoke English adequately. Documentation

Mr Ndzala had no documentation to show us. When pressed, he claimed that his files, textbooks etc were either at home for safekeeping, or had been left in one of his previous schools. The Interview

Mr Ndzala spoke to us after the lesson. He has appreciated the support of the RUMEP staff and says it is the only thing that kept him in the school and motivated during his struggle to get paid. He attributes the good Grade 12 results achieved at Gobizembe in 2007 to the help he had in teaching those learners in Grade 10. Valuable skills in lesson planning and designing activities had been learnt in the BEd course. He feels that a big change for him has been an appreciation of the value of yearly planning, taking into account the departmental guidelines. Mr Ndzala spoke of his disappointment at the lack of support from the DoE, and particularly their failure to address long standing tensions and problems in the school. He also feels the lack of collegial support in the school. He goes to the RUMEP cluster meetings for help and support and has asked Thomas Haywood to visit him at the Kidds Beach school. The RUMEP resources provided over the years of the BEd are scattered due his frequent moves, but Mr Ndzala seems to know where they are. For example, the blackboard instruments are in a store room at the Kidds beach school and he plans to leave them there in safety until he is settled at a school. Some books were left at Gobizembe, he has others at home, especially the education ones. Mr Ndzala was not able to comment on the marks expected from his Grade 10 learners in our testing as he was no longer at the school where testing was taking place.



When asked about factors that could impede the success of the project, Ndzala mentioned the lack of support form the DoE, and the lack of enthusiasm of the teachers. He also mentioned other barriers to learning such as English as a medium of instruction. Mr Ndzala suggested that the project be extended to include the GET phase as he feels that the establishment of good mathematical background is critical. 2008 Testing results

The 2008 testing was done at the Kidds Beach school, Jonguhla, as we wished to track the Grade 10 learners taught by Mr Ndzala. In the event he had gone by the third term, but we tested the same learners again as they had half a year of his teaching, and little new is done in the fourth term. Mathematics content test

Table 5.15 Grade 10 Mathematics results 2006-2008, Gobizembe/Jonguhla

Basic skills

Mean % Gr 10 maths

Mean %

2006 November n = 23 31.3 5.6





2008 February (all) n = 19 26.1

Jonguhla November (all) n = 10 31.5 4.6



The mean score on the Grade 10 mathematics content test achieved by the learners at Gobizembe school was 8.5% which was the ranked fifth out of the 11 project schools and showed some improvement from the 2006 mean. There was quite considerable improvement in the mean scores in the basic skills section. Mr Ndzala’s Grade 10 learners at Jonguhla improves somewhat in the basic skills through the year, but the Grade 10 content section was poorly done. This is not a point of comparison with the previous two years being a different school, and Mr Ndzala only there for 6 months. Attitude test Mr Ndzala’s Grade 10 learners seem to show reasonably positive attitudes to mathematics at both schools we tested. The high agreement with the composite statement my teacher affirms me at his new school is encouraging as it indicates that he was creating a positive environment at that school.



year200820072006

Mea

n

1.50

1.00

0.50

0.00

0.59

0.85

0.60

0.97

0.560.57

1.00

1.27

1.06

0.71

0.900.85

0.40

0.71

0.600.53

0.41

-0.04

school: Gobizemb





Figure 5.13 Mean scores on attitude tests 2006 – 2008, Gobizembe/Jonguhla

Classroom environment test It is difficult to make definitive interpretations on the classroom environment in this case due to his movements from one school to the next where conditions are so different. What is positive is that despite moving to a new school and only being with the learners for 6 months he was able to produce a positive learning environment above average in three scales being Cohesiveness of the class, teacher support and learner involvement. Three scales decreased significantly and we can only presume this was due to the learners not having him present. These were task orientation i.e. learners not focussing on the work, doing homework, preparing for exams etc. cooperation i.e. competition increased and finally equity i.e. the learners did not feel equally treated. Given the movement of the teacher during the year there can be little certainty about interpretations unfortunately.

Table 5.16 Classroom learning environment 2006-2008, Ndzala


Initial year 2006 (Gobazim) 28.6 26.4 27.2 34.2 27.3 31.2

Intermediate 2007 (Gobazim) 29.1 27.4 28.2 33.9 30.1 31.9

Final year 2008 (Jonguh) 30.8 29.1 28.3 27.9 27.4 28.1

Control schools 28.9 25.3 26.3 33.1 27.9 29.5



16

20

24

28

32

36


Environment scales

Lev

el o

f ag

reem

ent

Initial year 2006 (Gobazim)

Intermediate 2007 (Gobazim)

Final year 2008 (Jonguh)

Control schools

Figure 5.14 Ndzala: Comparison of learning environment 2006 to 2008 with control schools

Overall summary

The notion of a teacher settled in a school and working year by year with a stable set of learners has not materialised for Mr Ndzala. He appears to have been at the mercy of departmental inefficiencies and has had to struggle to get permanent paid employment. After teaching six months at a school he was finally paid his current salary but not the arrears. An initial legal appeal did not succeed and he is still paying off the lawyer’s fees. At his new schools, he inherits problems and told us he was embarrassed at the poor standard at his present school. He hopes to move from the school as soon as possible. Little can be learnt from this case except that he was doing well and improving learning environment and mathematics results until he moved to the other schools. It is our opinion that he is a success case of the project and given a good stable school will make a good contribution.



5.8 MRS NIKA JOSI MARELA HIGH SCHOOL

Context

Josi Morela has an impressive face brick external structure for a school situated in deep rural surroundings but has limited space for playing fields or expansion. The school is equipped with an adequate staffroom, sizable classrooms that can accommodate 30 to 50 learners, is electrified and has a fence meant as a deterrent for stray animals. The principal’s office, secretary’s spacious office, a staff room and toilet facilities for teachers and visitors are all situated in the administration building, the first one as you enter the school gates. Mrs Beza, the principal of the school, welcomed and spoke to us briefly in her office while we waited for Mrs Nika. Unfortunately, the principals’ opinion of Mrs Nika was negative as she was more relieved than worried at the prospects of her leaving for Hoho Secondary in the near future. What we observed was that the notice board in that office displayed lots of departmental circulars and notices, SGB committee members and meeting dates, teachers responsible for classes in 2008 as well as the school timetable. This was a sign of a functional school whose management is keeping tabs of what is going on in education in general. Peace and tranquillity prevailed at this school and the atmosphere was suitable for teaching and learning in general. However, the enrolment has been decreasing annually due to transport problems and the location of the school. This has led to deplorable and unbearable workloads for staff, which tend to scare teachers away from such working environment. Maths classroom environment Mrs Nika still uses the Home Economics room for her Mathematics class, as it is reasonably spacious with 28 desks and 34 chairs. Even though there was no OHP, the chalkboard is substantial for a class of that size and there is enough storage capacity for both the teacher and learners alike. Unfortunately, the posters and charts that we had noted in 2007 seem to have disappeared this time around rendering the classroom bare. Adjacent to this is a room where mostly textbooks, calculators and other teaching materials from RUMEP are stored and this is where the teacher spends most of her time. Lesson Observation

Notably Mrs Nika does not seem to be following the ECDoE guidelines or work schedule for grade 10. Even though there was evidence of some long-term planning, questions



were raised as to when it was prepared given the lack of linkages in the plan. In fact lesson plans had been done up to 22 August in a surprisingly similar format. Notably she had planned for LO1 to LO3 till now but surprisingly started off with LO 2 (i.e. Functions and Algebra) without any Numbers and Relationships concepts. Indeed learners are bound to battle understanding the Algebra if they have not mastered, let alone, taught numbers. The lesson observed on the day had a lesson plan that provided insufficient detail. The four learners, who sat as a group, were visibly nervous and uncomfortable with strangers in their class and the teacher forgot to explain the reasons for our visit. We later discovered that these learners were in fact unfamiliar to her as her class was later on in the day. Mrs Nika preferred the traditional teacher driven approach and subsequently failed to engage the learners effectively in the lesson. In spite of trying to use an activity sheet, aimed at consolidating the multiplication of a binomial by a trinomial, the learners were left more confused than before. At the end too much time, about 75 minutes, was spent on this type of work and could not complete her lesson. The learners were attentive and occasionally asked questions, predominantly in isiXhosa, while the teacher responded in English mainly but used isiXhosa for clarification. What concerned us though was that she was doing at the end of August what was meant for term 1 of the grade 10 mathematics curriculum. The work was appropriate for grade 10 but misplaced at this stage of their development. Even the problems that were chosen were not structured to become progressively more challenging from relatively simple ones. The emphasis was on the rules that had to be applied correctly no matter what. Unfortunately, the learners used them incorrectly, as no explanation was provided for them being wrong in the first place. In fact, there were some glaring errors, such as -3 x -4 = +7 and -7 + -5 = -2 that were left hanging as errors without any corrective measures. Documentation

The teacher file contained some long-term planning for grade 10 mathematics class. Only a copy of the 2005 Learning Programme was included in the file and nothing on the new curriculum statements. It is strange to understand the absence of these critical documents given that Mrs Nika is the only person teaching mathematics from Grades 8 to 12 as well as Technology in Grade 9. The curriculum coverage was very thin in general, which is an indicator of how poorly prepared her learners are at this school. However, this is not surprising given the history and the personal problems that Mrs Nika had to endure in last three years.



The amount of work covered was shocking given the time of the year. Upon scrutinising the learner books, we noticed that one learner had written 12 sets of homework since the beginning of the year with nothing written for January and February, only one days work in April and June. It was clear that the grubby state of the learners’ books had been cause for concern and Mrs Nika had got the learners to copy out their work into new, clean and covered exercise books. This included copying the previous errors! The grade 10 work was very easy and just below the expected norm, whilst the grade 11 work though appropriate, seemed to have been done as a whole class with very little individual practice. In fact, the June Examination paper written by the grade 10 learners was pretty much work covered in Grade 9 with one or two grade 10 related topics included. RUMEP has supplied the school with textbooks, calculators, posters and other teaching materials. However, since the room they are using is not secure, these have been stored in a safe room. No new calculators provided in 2008 but a few from the previous years were lost while she was away from school. No worksheets were spotted in the grade 11 learner books on display. The Interview

Mrs Nika was happy with the project for changing her approach to teaching. She further acknowledged that the BEd course has taught her the value of proper preparation. However, she was disappointed at not completing her BEd due to personal circumstances beyond her control. She is more determined than ever to continue until she succeeds. RUMEP staff have played their role supporting them to an extent of even covering up for a lack of support from the department of education. Most of the RUMEP resources are claimed to be stored for security reasons but some have disappeared. The school has performed very poorly in the recent past with only two learners out of 17 passing mathematics in grade 12. They tend to pass other grades but battle when it comes to matric, which is not surprising given the limited quantity and quality of work covered in the lower grades. Mrs Nika attributed the poor performance in grade 12 on the shortage of time given her absence and damage done by her unqualified replacement over two years. Even though she is thankful for the project, she feels that the authorities must deal with workload issues; otherwise, they cannot practice what they learnt effectively. Even the learners have questioned the wisdom of her teaching mathematics to all grades as they can see she is stretched. She needs time and space to give the best to each grade. 2008 Testing results

Mathematics content test The learners in Mrs Nika ‘s class performed poorly on the grade 10 content test in 2007 with the highest individual score being 17%. The mean mark was even worse in 2008, with the best of the five learners scoring just 4%. On a more positive note, the 10% increase in mean score on the basic skills section was pleasing, and an even greater increase was noted in 2008.



Table 5.17 Grade 10 Mathematics results 2006-2008, Josi Marela

Basic skills Mean %

Gr 10 maths Mean %

2006 November n = 16 30.6 5.5





2008 February (all) n = 6 25.8

November (all) n = 5 39 3.2

February (matched) 25

November (matched) n = 5 39 3.2

Attitude test The mean attitudes of the Mrs Nika’s Grade 10 learners in 2007 were notably more positive in 2007 than in 2006, particularly the teacher affirmation scale. This pleasing trend did not persist into 2008, arguably due the absence of Mrs Nika in the first two terms. On the contrary, all scales dropped to the extent of there being mean disagreement with the composite statement my teacher affirms me, as indicated by the negative bar in Figure 5.15.

year200820072006

Mea

n

1.50

1.00

0.50

0.00

0.55

0.79

0.48

-0.10

0.75

0.27

0.910.900.99

0.57

0.96

0.60 0.58

0.73

0.55

0.13

0.43

0.30

school: Josi Mar





Figure 5.15 Mean scores on attitude tests 2006 – 2008, Josi Marela

Classroom environment test Mrs Nika classroom environment improved significantly across all scales in 2007 . Unfortunately this was not maintained during 2008 and all scales decreased. It can only be assumed that being absent from the class for six months made a significant difference. However, most scores remain above the control average. The only one significantly below is competition which dropped a lot even below the control. This



indicates a competitive spirit being encouraged in the class as opposed to a more cooperative learning environment.

Table 5.18 Classroom learning environment 2006-2008, Nika


26.9 23.7 25.0 30.7 26.8 27.6

Intermediate 2007 33.2 29.8 29.4 34.3 29.8 31.8

Final year 2008 (Away 6 months) 30.0 27.6 28.4 32.0 24.4 30.4

Control schools 28.9 25.3 26.3 33.1 27.9 29.5

16

20

24

28

32

36


Environment scales

Lev

el o

f ag

reem

ent

Initial year 2006

Intermediate 2007

Final year 2008 (Aw ay 6months)Control schools

Figure 5.16 Nika: Comparison of learning environment 2006 to 2008 with control schools

Overall summary

The standard of Grade 10 mathematics at this school remains very poor and below the mean for both the project schools and the entire group of learners tested. The basic skills though have improved significantly in the course of each year of the testing and were considerably above average at the end of 2008.. The trend of slight improvement in overall attitude of the learners towards mathematics and significant improvement in a number of important categories in the classroom environment survey was reversed and these scores dropped in 2008. Mrs Nika represents a positive success for the project as there was an improvement in all the learner scores in 2007 when she was at the school for the full year. She has been able to improve learner results in basic mathematics significantly, but not in the grade 10 work. The variation in the class environment scales and attitude scales can easily be explained by the 6 month break and again emphasises that project teachers do make a difference.



5.9 MS SANQELA NTABENKONYANA HIGH SCHOOL

Context

Ntabenkonyana is a typical rural school with some surprisingly striking features that can only be identified once you are on the premises. The school has four blocks of classrooms currently in use, a staffroom and an administration area doubling up as a secretary’s office and a separate room for the principal. Access to the Principal’s office is via an average size room that acts as a waiting room and a secretary’s office. The Principal’s office is fairly small and sometimes doubles up as a storage room for new furniture such as printers and photocopiers while allowing visitors privacy with the principal. On the day of our visit, we observed that the secretary was being trained on the use of some computer software that had been installed in the school for administrative purposes.

The school is electrified, has a fence that can only act as a guideline for the community to distinguish the school premises but is not a good enough deterrent for vandals and thieves. Another feature observed this time around was the shade ports that had been erected. The school gave us a sense of being open to the community, accommodative with a healthy atmosphere in general. Educators were quite friendly and welcoming to visitors. Even the chairperson of the governing body welcomed our visit as he had attended a health forum held at and fully supported by the school which is a health promoting school in the area. Maths classroom environment The classroom observed was a fairly large one, with two double-sized chalkboards adjacent to one another and visible to all learners in the classroom. The classroom has a lockable cupboard and can accommodate about 50 learners easily with enough desks and chairs (even though 2 chairs had to be borrowed from another room on the day). The classroom walls are filled with RUMEP posters as well as some interesting and illuminating learner-prepared charts on graphs, reflections, number sense development, etc.



Lesson Observation

The lesson plan for the day was prepared using a short-term planning template (apparently a standardised document in most of the schools in the area). Even though no daily forecast was observed, the educator knew very well what her daily routine would be from our discussion later on. In this lesson plan there were clear guidelines as to what was to be achieved, how it would be achieved, learner activities to be done, role of the educator and some assessment tasks for the class, which was very gratifying. Mrs Sanqela managed the class with distinction in spite of the large number of students coupled with the many groups she had to oversee. Learners were visibly excited about the lesson judging by the general ambience in the class. A friendly atmosphere existed throughout the lesson and the educator was constantly probing learners to engage in the class activities and discussion. Enough resources such as a pair of scissors, rubber, Cartesian plane and an activity sheet had been planned for each group of four learners. These were brought by the teacher, distributed to groups and later collected from groups by individual group leaders. The lesson introduction was appropriate and attempted to review (with success) other transformations that had been learnt leading to some generalisations. The outcomes of the lesson were clearly articulated with relevant activities planned and completed at specific intervals. Learners were continually involved and absorbed in some tasks while guided by the educator and supported in some cases by the group leaders. The educator displayed good presentation skills in general and was supportive to learners. The only hiccup was time management at the end. The lesson on Translations was introduced and contextualised using the general meaning of the term itself and its mathematical equivalence. Different triangles had to be cut by learners from the photocopied sheets and learners had to make different drawings guided by the questions from the educator. It was pleasing to observe the educator rephrasing questions in an attempt to assist those learners who had problems with the task at hand. Proper explanations were provided regularly during the lesson and accurate terminology was used as much as possible. On the day, 48 learners were in class arranged in twelve groups of four learners each. These were smart, well-nourished and visibly motivated learners. Communication was mainly in English with regular code switching for clarity from both the educator and learners themselves. The noise level in class was understandably high given the number of students, group work and the nature of tasks that the learners were engaged in. Documentation

The teacher filecontained lesson preparation that was done on a “short term planning template” with limited details of how teaching will be done. Samples of learners work was also available as evidence of what ha transpired in the year. The grade 10 mark lists indicated 54 as an initial intake in class and various assessment tasks had been used to generate the mark profiles of the learners such as tests (written in terms 1 & 3), investigations (given in terms 1 & 2), an assignment



(given in term 3) and an examination (written in term 2). This was a clear indication that different assessment strategies were being employed by Ms Sanqela in her teaching. These mark lists further indicated a wide range of marks from 0% to 81% for these assessment tasks. The NCS curriculum documents were kept in the School Preparation file and they had been obtained from RUMEP as well as the Eastern Cape Department of Education. Dept circulars: By and large the departmental circulars, meeting schedules, workshop plans and departmental instructions were displayed in the secretary’s office adjoining the principal’s office. These could be easily accessed by all educators and other stakeholders in the school. A fair coverage of the grade 10 curriculum was observed from both the learner books shown to us and the curriculum coverage form completed by the educators. In fact the only learning outcome (LO) that had very limited coverage was Functions and Algebra as only linear equations and factorization had been done in term 1. The learner books had LO 1 in the front and then LOS 3 & 4 at the back. From the books presented it is clear that sufficient work is set by the educator with ticks as indicators of the marking process. Even though no comments were made in the learner books themselves as feedback to learners, sufficiently detailed comments were made on the 3 learner portfolio tasks presented. In the main consolidation exercises were prevalent with some investigations here and there supported by some interesting written reflections by the learners themselves. RUMEP did supply the school with books for learners and the educator has been using some printed worksheets from RUMEP as well as some originally designed ones. The Interview

Mrs Sanqela seems to have embraced the learner-centred approach to teaching and learning and the investigative style of teaching is promoted in the process. She was openly excited at the prospect of this visit by the evaluators and was thoroughly

prepared for us. As a teacher she has grown enormously especially with her ability to plan for better teaching and her understanding of the assessment standards. She feels she can now use relevant teaching strategies to inspire her learners and enhance their understanding of the mathematics. These teaching strategies have been particularly valuable and have assisted her in handling new topics in the NCS with confidence. One of her strengths, which she attributes to the project, is leadership.

Ms Sanqela was grateful for the support and intervention of RUMEP staff saying that without them she would not have gained the confidence she now has as a mathematics educator and would not have known about other teaching strategies which have helped her learners understand mathematics better. Very little support was mentioned from the ECDoE side except for one 2-day workshop with lots covered by the presenter but little or no engagements from the participants. She uses a lot of teacher materials designed by the RUMEP staff as well as some activity sheets handed to them by RUMEP. Even though the grade 12 mathematics results are pleasing according to the school management team, there is little evidence to support this claim. In fact they acknowledge that in the rural area, learners only become serious with their studies



when they reach grade 12 with prospects of leaving school to further their studies or use matric to find some employment. The project has unearthed a conscientious leader in Ms Sanqela who has a passion for mathematics, teaching and learners. Her dedication and effort has paid dividends and she is now heading the mathematics department. One of her roles, in the words of the deputy principal, was taking newly employed mathematics teachers under her arms through orientation, guidance, support and development in general. This is something she has performed with success given the enabling environment in the school. 2008 Testing results


Table 5.19 Grade 10 Mathematics results 2006-2008, Ntabenkonyana

Basic skills Mean %

Gr 10 maths Mean %

2006 November n = 41 33.7 8.1




2007


2008 February (all) n = 54 35.3

November (all) n = 49 38.1 10.3



Ms Sanqela's 2007 Grade 10 learners were ranked sixth (the middle position) out of the project schools for the Grade 10 maths content section. The highest individual mark was 21%. The basic skills section of the test also showed average achievement. The marks continued to improve in 2008. When the distribution of marks was graphed in a boxplot (see Figure 3.4) it is noticeable that most of the outliers (those that do statistically better than the norm) come from Mrs Sanqela’s school. The highest mark on the Grade 10 section of the test was 28% which is relatively very good, and indicates some familiarity with the work. Attitude test The mean agreement in Ms Sanqela's Grade 10 classes with all the composite statements in the attitude questionnaire improve consistently form 2006 to 2008. This is illustrated well in the graphical representation in Figure 5.17.



year200820072006

Mea

n

1.50

1.00

0.50

0.00

1.02

0.810.75

0.87

0.630.62

1.41

1.251.25

1.030.96

1.03

0.85

0.47

0.62

0.51

0.33

0.14

school: Ntabenko





Figure 5.17 Mean scores on attitude tests 2006 – 2008, Ntabenkonyana

Classroom environment test Ms Sanqela’s class environment improved in six categories from 2006 to 2007. She has continued this trend in 2008 improving on all categories and is the only teacher in the group who has done this. It is obvious that she is implementing what she has learnt on the project and creating a positive learning environment for her learners.

Table 5.20 Classroom learning environment 2006-2008, Sanqela


27.2 24.9 25.9 35.0 27.1 28.4

Intermediate 2007 31.7 27.8 27.2 34.8 30.1 30.1

Final year 2008 31.4 28.6 28.9 35.2 30.1 32.7

Control schools 28.9 25.3 26.3 33.1 27.9 29.5



16

20

24

28

32

36

40


Environment scales

Lev

el o

f ag

reem

ent

Initial year 2006

Intermediate 2007

Final year 2008

Control schools

Figure 5.18 Sanqela: Graphical comparison of learning environment 2006 to 2008 with control schools

Overall summary

Ntabenkonyana is a school with a lot of promise and hope and can be classified as a success story given the overall environment that exists in the school. Perhaps the involvement of more educators in staff development projects and the embracing of such projects by the management team encourage teachers to work harder in improving their teaching and hopefully the results. Mrs Sanqela is the only teacher in our sample that showed consistent improvement on all measures over all three years. Mrs Sanqela is one of the success stories of the project and has shown improvement in her teaching of mathematics. However, this has not been turned into large gains in her learners marks. This case is a first indication of how a trained and conscientious teacher working in a stable environment does not necessarily result in learner marks improving to acceptable passing levels.



5.10 MR WILLIAMS ELUKHANYISWENI

Context

The school, in deep rural area, is generally well-resourced – no problem with learner textbooks and classroom equipment. The library however is not operative and is used for storing old books.

Gardens were planted by “Greening the nation” project but now neglected – lack of water given as reason. Maths classroom environment The classroom had sufficient desks and chairs for the 23 learners present, a teacher’s table, chalkboard and one storage cupboard. No RUMEP Maths posters on the walls; there were some Science posters on the walls in Grade 10 classroom. Mr. Williams stated that the RUMEP Maths posters were in the Grade 11 and 12 classes.

Lesson Observation

Planning: The topic was: Drawing line graphs using the table method: investigating the effect of q in qaxy . Mr Williams had a lesson plan and a typed activity sheet ready for the lesson. He could have planned to have 1 sheet for each learner – in this lesson there was one activity sheet and one piece of square (grid) paper for each pair (or among 3 learners in one case). Learners were seated in 5 groups: and had the necessary stationery and were given resources (RUMEP materials: mathematics sets & grid paper) when these were needed in the lesson. They worked in pairs at their groups and Mr. Williams monitored group work continually during the lesson.



At the start, the teacher checked previous homework and then the worksheet for the lesson was then handed out – learners told to work in pairs on this. He told the learners that they must alternate the sketching of graphs so that all learners get opportunity to draw graphs (learners seem used to this arrangement and none are idle – others offering advice to the one writing on worksheet and sketching on grid paper). The lesson proceeded with a mix of whole class teaching and paired work. Mr. Williams used a learner’s error as a ‘teaching point’ and asked learners good questions about why these points should be on the line: drew learners attention to the relationship between x and y in the equation xy . He led the learners to see the need for the same scale on both axes. Mr. Williams then showed which other points (not in the table) could also lie on this line. This is good teaching practice. Mr. Williams stressed important concepts and revised relevant background knowledge. He showed good subject content knowledge and planned meaningful learner activities in the lesson but learners could have been given more homework to do. The lesson was taught entirely in English, with all terms stressed in English as well. The learners discussed among themselves in isiXhosa. Documentation

Mr Williams had a teaching file with a list of Grade 10 topics ticked off, mark sheets, tests and investigations for both terms, and the curriculum documents and departmental circulars. In terms of curriculum coverage, up to July only L.O.1 content had been covered. Mr. Williams felt that he had covered the basics in L.O. 1 and expected that he would not have much difficulty with aspects of L.O.2 that linked with exponents and also L.O.3 linked with what he had done in translations in graphs. He stated that he did not think he would finish all the topics but definitely he would touch on all the learning outcomes. He said that the exam was not early that year – not like in previous years when there was an exam in October and then everyone goes home. At least this year he would have more time to cover the content as the learners would stay at school. He said he did not plan to drop L.O. 3 completely – for him he felt that he could integrate topics and cover the other outcomes before the end of the year. Learners typically had about 20 pages formal work and about 30 pages at the back in rough. It seems that learners write questions in the back and answers in the front. He explained that learners work roughly at the back of their books and then rewrite neatly in front of the book after he has marked their efforts at the back. Mr. Williams thought that they did sufficient work – some is not evidenced in learners’ books as it is portfolio work – he marks it, shows it to them and then keeps it. So learners also do lots of work on pieces of paper. Each learner has two tests – both tests done with a test memorandum and marked in test file.

The Interview

Mr. Williams had not received the letter but was very welcoming and had documentation available at the school. The most important thing for him from the programme is that it has improved his knowledge of NCS new content. He stated that the department training was inadequate: sometimes 2 days for 3 – 4 topics in a workshop. So the immersion and the long-term look at the content helped him greatly at RUMEP. The booklets given have



been a support to him. The programme dealt with what he needed in his class. Now he helps other teachers in his cluster – they meet and discuss topics in more detail. Before the programme, he had no planning and he is now doing lesson plans, “although not 100%”. B.Ed course also helped him with long-term planning, which is now a departmental requirement too. When the DoE demanded that teachers have lesson plans, the teachers came to him for advice and he gave them the RUMEP template. Thomas still comes to the school – had been there 4 or 5 times (according to Mr. Williams). Mr. Williams said he is always present at the Monday sessions of support – they meet and help each other. Mr Williams said the RUMEP Maths posters were up in Grade 11 and 12 classes; used mathematics sets and square paper from given resources in the lesson. Mr. Williams said the English books were taken by the English teacher to use in class but then there was a problem that the cupboards did not have locks. Now they are kept in the library and taken for use in the period. The first calculators that he received he gave to the learners but then they did not bring all back. He said they were left with 4 calculators from that set and then he got the second set and – he now requires a deposit of R60 but there has been no take-up for this idea. The calculators are now with Mr. Williams and he does not use them in class. He expects the Grade 10 learners to do better in the 2008 testing – they did not perform well in June. Mr. Williams said there was sometimes a problem with the Grade 9 teacher, even though they were interacting with these teachers and he was expecting that they had done the work only to find out that they did not. . He felt he would cover content even though behind at this stage. Learners have their own textbooks to work from. Why are learners failing (based on June results)? A huge sigh! Mr. Williams he did not know why as he tried his best – he said there was a problem with absenteeism and some learners lacked the basics. When asked if there was any recovery plan to assist with basics, he said that he could not as he did not live in the area. If he did not leave at the time that the “contract transport” left at 2.15 pm he would not get transport and get to town by 7 pm if he was lucky because at 4 pm all the taxis are coming back – they are no longer going to town. . Should the project not succeed, the blame will be on the teachers who are not teaching what they are supposed to be. Alice district will not cooperate with the Department . They had a month of disruptions where teachers were toy-toying – would come to school at 8 am and knock off at 10 am to go to the District office to protest – this had taken about a month and a half. Had only been cooperating with the Department of Education for about a month or so – these “external factors” could also contribute to the lack of success. 2008 Testing results


Table 5.21 Grade 10 Mathematics results 2006-2008, Elukhanyisweni SSS

Basic skills Mean %

Gr 10 maths Mean %

2006 November n = 30 32.5 6.9

February (all) n = 33 35.9 2007

November (all) n = 31 39.2 10.27





2008 February (all) n = 18 26.9




Mr Williams’ learners showed progress in 2007 and the mean score in the grade 10 content section for LO1 (Number and Operations, at 21.2% was considerably better than any other school. The highest individual score for the Grade 10 content (32%) was relatively good as well. The 2008 group of learners did not perform so well with the mean dropping to about 5% and the highest score being 18%. The basic GET skills mean mark did improve quite considerably though. Attitude test

year200820072006

Mea

n

1.50

1.00

0.50

0.00

0.680.700.63 0.64

0.45

0.59

0.95

1.10

0.98

0.61

0.89

0.600.52

0.570.63

0.210.200.22

school: Elukhany

I have a positive attitude to mathsMy teacher affirms meMaths is very usefulMaths is high status subject



Figure 5.19 Mean scores on attitude tests 2006 – 2008, Elukhanyisweni SSS

The overall attitude to Mathematics measured in the successive cohorts of Grade 10 learners have remained fairly stable around the project mean. Figure 5.19 shows that there is some variation in perception of usefulness of maths and its status which however remains high. It is encouraging to note that the teacher affirmation scale has increased agreement in 2008. Classroom environment test From 2006 to 2007 there were some positive changes in his classroom environment. Again in 2008 there were some improvement and some scales that declined. Overall we can say that he has maintained an environment that is above the control schools with little positive growth. One scale decreased being cooperation i.e. his classroom was seen as more competitive in 2008 that before.



Table 5.22 Classroom environment for 2006 to 2008, Williams


27.8 27.5 27.0 33.3 25.3 31.3

Intermediate 2007 31.2 26.9 27.7 32.8 28.7 31.7

Final year 2008 31.1 27.6 27.2 31.4 26.3 31.8

Control schools 28.9 25.3 26.3 33.1 27.9 29.5

16

20

24

28

32

36


Environment scales

Lev

el o

f ag

reem

ent

Initial year 2006

Intermediate 2007

Final year 2008

Control schools

Figure 5.20 Williams: Graphical comparison of learning environment 2006 to 2008 with control schools

Overall summary

The standard of mathematics at this school is low, and after an initial increase has fallen below average for the schools tested. The attitude towards mathematics is stable and about average for the project schools. The classroom environment improved from 2006 to 2007 in a number of categories which was very positive, and remains stable. Mr Williams has not shown any dramatic increases or decreases in his classroom environment scales. However, he has maintained a more positive environment than the control schools. He appears to be a competent teacher who has settled into a particular stable mode of teaching. What is not explained is why the learners are not more competent at mathematics.



5.11 MR WILLIE EYABANTU

Context

This school is fairly close to town in a township area. The actual school was the typical square shape, a grass patch in the middle and with the administrative offices at the one end. This school was struck by a tornado in late 20007 and the roof of one block destroyed which contained the computer room and some classrooms was destroyed. The equipment has been moved to the edges of the room to escape the worst of the weather. A section of the staff room was burnt in 2008 (thought to be learners trying to destroy school records) and Mr Willie claims to have lost his 2008 planning files there. A room in one block had piles of textbooks visible through the window.

The school has been told that they are along way down the list of schools needing repair and must just wait their turn. Maths classroom environment The classroom was bare with a broken ceiling and a broken cupboard. There was no teacher desk but each learner had their own desk. The desks were arranged in groups. No posters etc on the walls. Lesson Observation

Mr Willie does not have a Grade 10 mathematics class in 2008, and so he had arranged to show us a lesson with his Grade 11 class.(he taught them last year in Grade 10). There was no long term planning available, but a lesson plan was provided for the current lesson. It did not seem to match very closely the transformation geometry that was actually being taught, being on reflections whereas enlargements



were being discussed. The lesson presentation was adequate – he has a clear voice and the explanations were good. There was however, no questioning or activities to check for understanding. In fact I noted that although the learners all had exercise books and pens in hand, after 50 minutes they had not written a thing. Mr Willie was using a learner copy of the worksheet to mediate the work so that only a few learners could see where he was pointing. Classroom management was adequate as the learners were well behaved and there were not many resources to manage. The learners were given one worksheet among two, and no text book was used. The mathematical content of the lesson was a bit muddled. The pages used seemed to be lifted from the RUMEP module materials and did not appear to be used in the intended order. For example, in a worksheet given after work on dilations from a fixed point (quite advanced transformation geometry) we were back to providing the co-ordinates of a point which is very basic work from the GET. It does not appear that this lesson was well thought through. The learners in the class we observed were neatly dressed, quiet and attentive. They seemed a bit baffled by the lesson. Looking through a learner’s book revealed the incorrect work done on 29 January and marked and signed by Mr Willie. This shows a lack of conceptual understanding. The procedure used is only applicable for a right angled triangle and the diagram in use does not show a triangle at all. Documentation

Mr Willie had no current documentation to show us. When pressed, he claimed that his planning files had been burnt in the staffroom fire. The Interview

Mr Willie is teaching Grade 9, 11 and 12 Maths. Most of the learners at the school are doing Maths Literacy he estimates about two thirds.



He has all the RUMEP materials but the maths sets and calculators are with the learners. The Grade 12s are supposed to bring them back and he keeps results until the calculator comes back. He has the text books and blackboard instruments and some are kept in admin block for safety. There doesn’t seem to be a good system for safe keeping. He had a cupboard of stuff in the broken classroom – “hey hey hey look at this mess !” when he saw all the stuff tumbling out of the cupboard. The graph paper pads etc. had been kept in the cupboard. Mr Willie cited the change in his attitude as the most positive aspect of the BEd course. He also had been given a lot of options in ways of teaching specific topics so that he can choose a suitable strategy for each group of learners.

He had done his planning of the year and thinks that provides a good guideline. He had the book or file, but now only has the Grade 12 one as the others were lost in the fire. No departmental pace setter has arrived at his school, and the department says it is at the discretion of the school whether to do tests or to have exams. Mr Willie said he had no knowledge on how the current Grade 10s would do in our tests, but he thought the current Grade 11s would have done all right in our Grade 10 last year in 2007. He agreed that if a teacher does the training and gets a BEd it has to be that the learners will do better. The school gets between 60 and 80% pass in Grade 12 – it fluctuates but never below 50% pass. There will be 27 Grade 12 learners writing maths in 2008 and about 50 doing Maths Literacy. Reasons given by Mr Willie for project possibly not leading to improved results included the change in curriculum because some of the “chapters are new to the teachers”. However the new topics were carefully done in the BEd which should help the project teachers. He indicated that the learners have “less than no motivation” and they do not work after school hours. “Nothing –they do absolutely nothing at home. You give them things to do and have to beg them to do something.” Classroom support from Thomas was good. He has been this year. Was the R70k spent per teacher money well spent? Mr Willie said he would broaden the focus more – there are local schools who had no teachers on project and so they are called on the help them at workshops. The time frames were tight they need a study week – he was also doing a MSC (private college) computer course at the same time and that, together with pressure of school work, proved too much. The school was reluctant to give him study leave as there was no other teacher to do the maths in the school. The new maths teacher that the school did get is rumoured to be planning to move again



2008 Testing results


Table 5.23 Grade 10 Mathematics results 2006-2008, Eyabantu SSS

Basic skills Mean %

Gr 10 maths Mean %

2006 November n = 41 26.0 6.24




2007


2008 February (all) n = 33 33




In 2007, the highest individual score on the grade 10 content section was just 12% and the mean for LO4 data handling was 1,1% making it likely that this section was not covered. The basic skills section was poorly done in November (the mean score was lowest of all project schools) and this was one of the few instances where the November mean score was lower than the February mean score. In 2008, when Mr Willie was no longer teaching Grade 10, the mean performance continued to decline on the Basic skills through the year and the Grade 10 section mean score remained around 5% with the highest individual score 14%. Attitude test

year200820072006

Mea

n

1.50

1.00

0.50

0.00

0.590.63

0.55

0.27

0.37

0.50

0.930.951.00

0.790.770.80

0.520.57

0.46

0.32

0.23

0.09

school: Eyabantu





Figure 5.21 Mean scores on attitude tests 2006 – 2008, Eyabantu SSS

Mr Willie was much involved with SADTU activities in 2007 which hindered his academic progress and commitment to the module requirements (RUMEP report) and



so it is not surprising that the attitude of the learners was affected. The mean agreement in Mr Willies’ class with the composite statement I have a positive attitude to maths dropped from February to November 2007, and showed among the lowest agreement of the project schools. Among the matched learners there was a particularly notable drop in agreement in the teacher affirmation scale. In 2008, with another Grade 10 teacher, the teacher affirmation scale is again low, although the perception of competence (although not the actual competence according to our testing) is higher. Classroom environment test In discussing the environment changes from 2006 to 2007 it was suggested that much needed to happen for positive changes to occur. Unfortunately little appears to have happened and the environment remains varied with majority of scales decreasing slightly. It is obvious that there is no concerted effort to positively change the classroom environment established earlier. On average they are very close to the control school means. The fact that another teacher took the grade ten class in 2008 and there was no change shows that Mr Willie did not positively improve the environment.

Table 5.24 Classroom learning environment results 2006-2008, Willie

Scale SCT TST INT TOT COT EQT Initial year 2006 (Willie)

27.6 22.4 25.5 34.7 26.0 29.3

Intermediate 2007 (Willie) 30.3 24.1 25.7 33.3 25.0 28.4

Final year 2008 (Control) 29.0 25.7 26.4 31.9 25.0 29.4

Control schools 28.9 25.3 26.3 33.1 27.9 29.5

16

20

24

28

32

36


Environment scales

Lev

el o

f ag

reem

ent

Initial year 2006

Intermediate 2007

Final year 2008

Control schools

Figure 5.22 Willie: Graphical comparison of learning environment 2006 to 2008 with control schools



Overall summary

Mr Willie has work outstanding and so did not graduate with the rest of the cohort in 2008. By his own admission, he found the workload heavy. It should be noted though that he voluntarily took on union work, and enrolled in a computer course at a private college in a year in which he was trying to complete his B Ed. The learners perform very poorly and have relatively low positive attitudes to mathematics. In the case of Mr Willie there was not a significant improvement from the baseline in 2006 to 2007. Consequently when a new teacher took over there was little change in the classroom environment for teaching mathematics. It is our opinion that he did not transfer his project experience into improving the teaching of maths.



SECTION 6 SOME COMMENTARY ON THE EVALUATION

6.1 Why does improved competency of the teacher not result in improved learner performance?

No direct link could be found to suggest that competence of the project mathematics teachers would result in the improvement of the mathematics grade 10 scores. It is our contention that basics skills are the major stumbling block. The learners entering grade 10 have very low basic skills in mathematics that despite competent teaching they are not able to access the grade 10 level mathematics and consequently show very little improvement over the year despite having a competent teacher. This was one of the major logic links made in the programme design. It was hypothesised that if the teachers were provided with professional development such that they were competent in both the mathematical skills and the pedagogical skills that improvements would be seen in the achievement of their learners. We did not find any causal or correlational evidence to support this general statement from the tests we administered to the project and control groups. However, we do not believe the approach was a waste of resources. As a first step every teacher in the system must be trained and be able to implement in the classroom. We believe this programme was able to achieve this. If this was not the critical step or stumbling block then what is it? Why has there not been a significant improvement? Three factors are suggested. Basic skills: It is our contention that the learners entering grade 10 have very low basic skills in mathematics that despite competent teaching they are not able to access the grade 10 level mathematics and consequently show very little improvement over the year. A close look at some of the learner work indicates that basic skills are lacking to the extent that new work cannot easily be accessed. Marks are not so much lost by careless errors, nor by pockets of unfamiliar work as often occurs, but by a complete lack of very basic algebraic skills such as expansion of bracket, multiplication of terms with powers and complete unfamiliarity with trigonometry and graphical work. We are aware that our test might be viewed by learners as low stakes, and that they had not prepared for the test but this does not account for lack of routine mathematical skills. Poor background often hinders the mathematical progress of learners even when they move to well resourced schools – other projects have seen learners(selected for their potential) struggling to achieve even 30% in Mathematics at grade 10 level.At the beginning of grade 10 we gave the learners a very basic mathematical skills test. On average they scored between 20 and 35% for this test. At year end they were still below 40% in what we consider to be the very basic skills required in order to begin to understand and access the Grade 10 syllabus work. It is no wonder that they scored in the region of 10 to 20% for grade 10 work. The question arises as to why teachers do not implement remedial classes to catch up the backlog. There are many reasons the most common being that they are too busy meeting the requirements of completing the grade 10 curriculum. Another reason is that they do not live near the schools so after school or Saturday programmes have many transport issues associated with them. School factors: From the school visits it was apparent that schools are under tremendous pressure from the DoE officials to improve the Grade 12 results which are seen as a very public indicator of success. This seems to shift the time and energies of the staff to the Grade 12s to the detriment of teaching lower down in the school. For example, when a teacher had been away on leave for six months and returns to find little work had been done by her replacement, she admitted that al her time was spent



on a bid to get the Grade 12s ready for the final exam. We have noticed dwindling learner numbers, particularly at the more remote schools. This leads to increased workloads for teachers as the number of teachers per school is dropped and one teacher has to deal with all the Mathematics (and sometimes also the Physical Science) in the school. Having up to seven or eight different classes to prepare for is a daunting task. Teacher factors: Understandably, the teachers applied for promotion on the strength of their new qualification, and several seem to have been appointed to management posts. This is a positive development for them personally, but often moves them from classroom teaching, or loads them with administrative duties that squeeze out the classroom work. Their colleagues are not always prepared to take on the “RUMEP way

6.2 What can be done in a similar situation to improve uptake.

It is our considered opinion that a serious intervention is required before or during grade 10 to make sure learners doing grade 10 have basic mathematical skills.

There are many reasons put forward by the teachers for the lack of skills. In general, they blame the teachers from the earlier years and the system that allows learners who fail, to progress to the next grade. In some cases you have learners who have never passed a mathematical examination in grade 10. Many schools only have one mathematics class / teacher so there is no discrimination between mathematics and mathematics literacy. A second issue raised by teachers is the lack of a culture of learning with learners not interested in school and doing homework etc. It could be suggested that this is just passing the problem downwards. Grade 12 learners do not pass because they are not prepared so move to focus on grade 10. Now we are suggesting we go further down the grades and next suggestion will be to train the GET teachers. Unfortunately the problem will not go away until schools are functioning, all teachers in all grades are competent and learners are motivated to learn mathematics. We are seeing that all components are required and focus on one does not solve the problem. i) Generating interest: We feel some programmes for the school vacations outside of schools in which maths is put forward as fun, exciting and valuable needs to be put in place. This should not be curriculum based but rather focus on teaching learners basic reasoning skills for mathematics in the context of number games, maths puzzle solving, maths logic etc. Hopefully this will improve learners mathematical functioning and get them interested in spending time on mathematics. It should be restricted to learners in GET phase and if it has a competitive element should be based on teams. The object is not to celebrate the clever maths learners but to get the “normal” learners interested and motivated.. ii) Parallel curriculum: We also feel that it is time the department and teachers are made aware of the fact that teaching higher levels of mathematics to children who do not have the basics is totally unproductive. Our main suggestion is to develop a programme of “alongside learner support”, to remediate for basic skills whilst carrying on with grade appropriate work. This requires careful analysis of the basic skills required in each section and timeous “teenage appropriate” activities to refresh or introduce the basic concepts that will be required. This would require extra time – it



cannot replace the normal curricula maths. The problem of inadequate basics is pervasive across all contexts including urban and township school. All of these suggestions will be resisted as impossible to manage but to continue is to go against all the data collected over the last ten years showing little progress in mathematical competency in rural schools. Unfortunately we are wasting resources if we allow the current situation to continue. Those learners are better using their time if they read a book than attended mathematics class. It’s a foreign language class to them.

6.3 Is there any value added by further evaluation.

We believe that it is very important to continue with the evaluation but with a major focus on the teachers and the senior certificate results with less emphasis on the testing of the grade 10s. The evaluation has been extended by a year to obtain further information about the long term effects of the intervention. Given the current trend, we do not believe at this stage that the testing of the grade 10s for mathematics achievement will show an improvement. However, it needs to be done to confirm the trend establish over the last three years. It will tell us if the low base level of learners entering grade 10 is stable and a characteristic of the schooling system with little chance of improvement in the coming years. If continued poor basics skills are found it will reinforce the suggestion that a basic skills “alongside” intervention is required. Another focus will be the long term accumulative effect of the full cohort of teachers in the schools on the senior certificate results. Hopefully there will be an accumulative effect and some improvement in the mathematics results of the district will be found when the complete cohort of teachers is considered. For example a learner could get a number of project teachers over the last three years of schooling resulting in a change from the norm. It is hoped that this effect will be seen in improved senior certificate results. Thirdly, it will be interesting to determine what happens to the teachers who participated in the project. We will be able to see if they remain in the mathematics classroom or if they simply use the degree status to further personal ambitions and get promoted out the classroom. If this is the case, this could require further thinking around how to keep good teachers in the classroom and how professional development opportunities are offered to teachers.

6.4 Intervention and evaluation design

It is our feeling that future interventions should be very closely examined so that i) they are not set up based on faulty logic models, ii) more effort be placed on comparison groups or alternative evaluation strategies iii) independent learner testing be closely aligned with school testing. Logic models: The intervention was based on a logic model which was found to be suspect. It was predicted that if the teachers were well trained and supported in schools that learner achievement would improve. Unfortunately there are too many other



factors in play to support a causal link even if we had found that the results had improved. It is our feeling that future interventions should be very closely examined so that they are not set up based on faulty logic models. If they are, then we will be looking for outputs that have no direct link or causal relationship to the inputs. Comparison group designs: At the same time evaluation designs that have comparison groups are all but impossible to set up but are encouraged in evaluation designs and reported as if they are unproblematic. For many a randomised group is the gold standard for a study but this is out of the question in most cases as it is impractical and often ethically undesirable. A comparison group is one chosen through some other method. Unfortunately there is convincing evidence (US Department of Education, 2003) that “in a sizeable number of cases the inaccuracy produced by the design is large enough to result in erroneous overall conclusions about whether the intervention is effective, ineffective, or harmful.” (p.3). For example in this study the comparison group of classrooms were chosen from surrounding schools which were very similar in nature in terms of source of learners, school resources and teachers. Unfortunately we had no control over the teachers’ experiences so have no idea whether our control classroom had a teacher who in the middle of the intervention attended a departmental course that could have influenced learners’ achievement in some way. We also found that the teachers in these classrooms changed from year to year as opposed to the project schools where this did not happen as much. The point being made is that serious thought needs to be put into the value of current comparison designs. If planned carefully to take into account of as many factors as possible, the evaluation design becomes prohibitively expensive. If not done, there are chances of erroneous results. Learner testing: The evaluation design included testing of learners at the beginning of the year and again at the end. Low results are obtained that are different to those of the schools. (In the present evaluation we counter this by not working with raw scores but rather improvements only). We suspect that other factors besides test design are influencing results such as test preparedness and motivation. We believe preparation for testing is a very important factor. Many learners do not regularly do homework but when faced with a test do extra preparation. If evaluators are to continue testing in schools some system of cooperation needs to be worked out whereby learners are able to prepare for the testing. One example would be to include questions in the normal testing programme of the school or immediately after school examinations. Again this involves a lot more work on the part of evaluators to ensure test security and has an added cost. However, it is our belief that the general lack of preparation for testing results in learner demotivation when faced with more difficult problems and they opt out of these low stakes evaluator tests. Basically we might not measuring the learners true potential. Consequently when we report levels of achievement of 10 % for grade 10 we are not sure if this is the learners actual potential or one achieved through disinterested participation. This issue needs some discussion among the role players such as evaluators, teachers and funders.

6.5 Last words

Intervening in schools to bring about change is complex. We are seeing that there are a number of factors that all need to be addressed. This evaluation has informed us that having a competent teacher in a functioning school is necessary but not sufficient for mathematics achievement to improve significantly.



REFERENCES Davidson, E.J., (2005). Evaluation Methodology Basics: The nuts and bolts of sound

evaluation. London: Sage. Hobden, P. & Hobden, S. (2007). Implementation evaluation of RUMEP BEd Teacher

Professional Development Project. Johannesburg: Zenex foundation Hobden, P. & Hobden , S. (2008). Interim evaluation of RUMEP BEd Teacher

Professional Development Project. Johannesburg: Zenex foundation Pawson, R. and Tilley, N., (1997). Realistic Evaluation. London: Sage Reddy, V. (2006). Mathematics and Science Achievement at South African Schools in

TIMSS 2003. Cape Town: HSRC Press. US Department of Education. (2003). Identifying and implementing educational

practices supported by rigorous evidence: A user friendly guide. Prepared for the Institute of Educational services by the coalition for evidence based policy. Washington

Wei, R. C., Darling-Hammond, L., Andree, A., Richardson, N., Orphanos, S. (2009).

Professional learning in the learning profession: A status report on teacher development in the United States and abroad. Dallas, TX. National Staff Development Council.

Documents

School - eranet.co.za 2008_Rumep Summative report...5.2 Mr Adonis Hoho senior secondary school ... ACKNOWLEDGEMENT ... maths Teachers have poor content knowledge,