EVALUATION OF THE ENGLISH LANGUAGE PREPARATORY SCHOOL

CURRICULUM AT YILDIRIM BEYAZIT

UNIVERSITY

A THESIS SUBMITTED TO

THE GRADUATE SCHOOL OF SOCIAL SCIENCES

OF

MIDDLE EAST TECHNICAL UNIVERSITY

BY

PETEK ÖZDORUK

IN PARTIAL FULFILLMENT OF THE REQUIREMENTS

FOR

THE DEGREE OF MASTER OF SCIENCE

IN

THE DEPARTMENT OF EDUCATIONAL SCIENCES

AUGUST 2016

Approval of the Graduate School of Social Sciences

________________________

Prof. Dr. Tülin Gençöz

Director

I certify that this thesis satisfies all the requirements as a thesis for the degree of

Master of Science.

________________________

Prof. Dr. Cennet Engin Demir

Head of Department

This is to certify that we have read this thesis and that in our opinion it is fully

adequate, in scope and quality, as a thesis for the degree of Master of Science.

________________________

Prof. Dr. Ahmet Ok

Supervisor

Examining Committee Members

Prof. Dr. M. Levent İNCE (METU, PES) _________________

Prof. Dr. Ahmet OK (METU, EDS) _________________

Assist. Prof. Dr. Gülçin TAN ŞİŞMAN (HU, EDS) _________________

iii

I hereby declare that all information in this document has been obtained

and presented in accordance with academic rules and ethical conduct. I also

declare that, as required by these rules and conduct, I have fully cited and

referenced all material and results that are not original to this work.

Name, Last N ame: Petek ÖZDORUK

Signature :

iv

ABSTRACT

EVALUATION OF THE ENGLISH LANGUAGE PREPARATORY SCHOOL

CURRICULUM AT YILDIRIM BEYAZIT

UNIVERSITY

ÖZDORUK, Petek

M.S., Department of Educational Sciences

Supervisor: Prof. Dr. Ahmet OK

August 2016, 138 pages

The purpose of this study is to evaluate English Preparatory Curriculum of

Yıldırım Beyazıt University School of Foreign Languages from instructors’,

students’, and program and testing office members’ points of view. More

specifically, the curriculum belonging to English Preparatory School was

examined in terms of the context that the English program took place,

instructors’ and students’ perceptions regarding appropriateness of the program

in terms of objectives, content, teaching methods, materials and assessment

procedures in the Preparatory School. The evaluation was conducted by

applying Stufflebeam’s context, input, process and product model (CIPP).

Both quantitative and qualitative data were collected in this study. The

quantitative data were gathered via the questionnaire administrated to all

(n = 243) intermediate level English Preparatory School students. The

qualitative data were collected through individual interviews conducted with the

v

10 practitioner instructors, including 2 Program and Testing Office members as

they are responsible for preparing the assessment materials. Quantitative data

were subject to descriptive statistical analysis while qualitative data were subject

to descriptive content analysis.

Results of the study indicated that Yıldırım Beyazıt University Preparatory

School of English Language School was planned to develop the students’

English skills while it was needed to be improved in order to develop their skills

to follow their departmental courses. The findings also revealed the positive

effect of Independent Learning Center, the physical conditions, materials used in

classrooms, instructors, and the portfolio tasks. On the other hand, the study

showed the problems related to improving students’ listening skills and speaking

skills.

Keywords: Preparatory School English Curriculum, Curriculum Evaluation,

English Language Teaching, Foreign Language

vi

ÖZ

YILDIRIM BEYAZIT ÜNİVERSİTESİ İNGİLİZCE HAZIRLIK OKULU

PROGRAMININ DEĞERLENDİRİLMESİ

ÖZDORUK, Petek

Yüksek Lisans, Eğitim Bilimleri Bölümü

Tez Yöneticisi: Prof. Dr. Ahmet OK

Ağustos 2016, 138 sayfa

Bu çalışmanın amacı, Yıldırım Beyazıt Üniversitesi Yabancı Diller Yüksek

Okulu İngilizce Eğitim programının okutmanların, öğrencilerin ve

programlama-ölçme bölümünde görev yapan okutmanların bakış açısından

değerlendirmektir. Daha ayrıntılı olarak, Yıldırım Beyazı Üniversitesi’nin

İngilizce Hazırlık Okulu eğitim programı okutmanlarının ve öğrencilerinin;

programın amacı, içeriği, öğretme yöntemleri, kullanılan gereçler ve ölçme

yöntemlerinin uygunluğu hakkındaki algıları açısından incelenmiştir.

Değerlendirme çalışması Stufflebeam (1971) tarafından geliştirilen Bağlam,

Girdi, Süreç ve Ürün (CIPP) modeli kullanılarak gerçekleştirilmiştir.

Çalışmada hem nitel hem nicel veriler kullanılmıştır. Nitel veriler orta düzey

İngilizce Hazırlık öğrencilerinin tamamına, 243 katılımcıya uygulanan anketler

aracılığıyla toplanmıştır. Nicel veriler ise 2’si ölçme-değerlendirme biriminde

görev yapan ve ölçme araçlarının geliştirilmesinden sorumlu olan okutmanlar

olmak üzere toplam 10 okutman ile bire bir yapılan görüşmeler aracılığıyla

vii

toplanmıştır. Nitel veriler betimsel içerik çözümlemesi ile nicel veriler ise

betimsel istatistik yöntemleri kullanılarak çözümlenmiştir.

Çalışmanın bulguları Yıldırım Beyazıt Üniversitesi İngilizce Hazırlık

Programı’nın, öğrencilerin İngilizce becerilerini geliştirmek üzere planlandığını

ancak öğrencilerin bölüm derslerini takip edebilmeleri için gerekli olan

becerilerini ilerletme konusunda geliştirilmeye gerek olduğunu ortaya

koymuştur. Çalışmada elde edilen bulgular Bağımsız Öğrenme Merkezi’nin,

fiziki şartların, derslerde kullanılan araç-gereçlerin, okutmanların ve portfolyo

çalışmalarının olumlu etkisi olduğunu göstermiştir. Sonuçlar, diğer bir yandan,

öğrencilerin dinleme ve konuşma becerilerini geliştirmesiyle ilgili problemlerini

de ortaya çıkarmıştır.

Anahtar Kelimeler: İngilizce Hazırlık Okulu Eğitim Programı, Program

değerlendirme, İngiliz dili öğretimi, Yabancı diller

viii

To my beloved father

ix

ACKNOWLEDGEMENTS

To begin with, I express my genuine appreciation to Prof. Dr. Ahmet OK for his

patience, motivation, and immense knowledge throughout this research study.

At any stages in the course of study, I benefited from his aspiring guidance,

constructive criticisms and precious support. His careful editing contributed

enormously to the production of this thesis.

I wish to express my sincere thanks to my committee members, Prof. Dr. M.

Levent İnce and Assist. Prof. Dr. Gülçin Tan Şişman for their valuable

feedback, guide and contributions to make very necessary improvements.

Words cannot express how grateful I am to my father Faruk Özdoruk and my

mother Ezel Özdoruk for all of the sacrifices that they have made on my behalf.

I, of course, owe my deepest thank to them, for everything they are, and have

been, and always will be for me. I am grateful to my family, and of course to my

brother, Berk Özdoruk.

My special thanks go to Fatih Höke, the man who taught me to never give up,

not only for his love and endless care but also for being my support in the

moments when there was no one to answer my queries. Thanks for everything

you have done for me, my beloved husband. A special word of thanks also goes

to his dear family for their sincere appreciation and encouragement.

I would also like to thank all of my friends who supported me in completing my

thesis and incented me to strive towards my goals. I feel really lucky to have

friends like them. Without their help, it would be more difficult to walk on this

way.

x

I take this opportunity to express my gratitude to all faculty members of

Yıldırım Beyazıt University School of Foreign Languages for their help and

contributions especially to the director Mümin Şen and the assistant director

Müge Akgedik Can. This evaluation study could have been successfully

conducted with their passionate participation and input. I also thank all my

colleagues from Yıldırım Beyazıt University School of Foreign Languages who

provided insight and expertise that greatly assisted the research. I am

exceptionally grateful to Aslı Elmacı Üstün, who encouraged me to be part of

this family and has guided me throughout the years. Thanks for always believing

in me.

I should not forget my students, who welcomed me and were keen to help with

the project. All of them have been there to support me in every part of this

study. Thank you one and all.

xi

TABLE OF CONTENTS

PLAGIARISM .......................................................................................................... iii

ABSTRACT .............................................................................................................. iv

ÖZ .............................................................................................................................. vi

DEDICATION ........................................................................................................ viii

ACKNOWLEDGEMENTS ...................................................................................... ix

TABLE OF CONTENTS .......................................................................................... xi

LIST OF TABLES .................................................................................................. xiv

LIST OF ABBREVIATIONS .................................................................................. xv

CHAPTER

1. INTRODUCTION .................................................................................................. 1

1.1. Background of the Study ......................................................................... 1

1.2. Context of the Study ................................................................................ 3

1.3. Purpose of the Study ............................................................................... 5

1.4. Research Questions ................................................................................. 6

1.5. Significance of the Study ........................................................................ 7

1.6. Definition of the Terms ........................................................................... 8

2. REVIEW OF THE LITERATURE ...................................................................... 11

2.1. Curriculum Evaluation .......................................................................... 11

2.2. Summative Evaluation and Formative Evaluation ................................ 12

2.3. Curriculum Evaluation Approaches ...................................................... 14

2.4. Stufflebeam’s Context, Input, Process, Product Model (CIPP) ............ 17

2.4.1. Context Evaluation .................................................................... 18

2.4.2. Input Evaluation ........................................................................ 19

2.4.3. Process Evaluation .................................................................... 20

2.4.4. Product Evaluation .................................................................... 21

2.5. Curriculum Evaluation Studies Conducted Abroad and in Turkey ....... 22

xii

2.6. Summary of the Literature Review ....................................................... 27

3. METHOD ............................................................................................................ .30

3.1. Overall Design of the Study .................................................................. 30

3.2. Research Questions ............................................................................... 33

3.3. Subjects of the Study ............................................................................. 33

3.4. Data Collection Instruments .................................................................. 38

3.4.1. Student Questionnaire ............................................................... 38

3.4.1.1. Piloting of the Questionnaire ......................................... 39

3.4.1.2. Validity and Reliability ................................................. 40

3.4.2. Instructor Interview Schedules .................................................. 41

3.5. Data Collection Procedure ..................................................................... 42

3.6. The Researcher’s Role ........................................................................... 43

3.7. Data Analysis Procedure ....................................................................... 44

3.8. Limitations of the Study ........................................................................ 45

4. RESULTS ............................................................................................................. 47

4.1. Students’ Demographic Characteristics................................................. 47

4.2. Students’ Perspectives on Curriculum ................................................... 48

4.2.1. Students’ Perspectives on Curriculum Context ......................... 48

4.2.2. Students’ Perspectives on Curriculum Input ............................. 51

4.2.3. Students’ Perspectives on Curriculum Process ......................... 56

4.2.4. Students’ Perspectives on Curriculum Product ......................... 61

4.3. English Instructors’’ Perspectives on Curriculum ................................. 63

4.3.1. Instructors’’ Perspectives on Curriculum Context .................... 63

4.3.2. Instructors’’ Perspectives on Curriculum Input ........................ 65

4.3.3. Instructors’’ Perspectives on Curriculum Process ..................... 65

4.3.4. Instructors’’ Perspectives on Curriculum Product .................... 68

4.4. Summary of Findings ............................................................................ 71

5. DISCUSSIONS AND IMPLICATIONS ............................................................. 75

5.1. Discussions ............................................................................................ 75

5.1.1. Context Evaluation .................................................................... 75

xiii

5.1.2. Input Evaluation ........................................................................ 77

5.1.3. Process Evaluation .................................................................... 80

5.1.4. Product Evaluation .................................................................... 83

5.2. Strengths and Weaknesses of the Program ........................................... 84

5.3. Implications for Practice ....................................................................... 85

5.4. Implications for Future Research ......................................................... 87

REFERENCES ......................................................................................................... 89

APPENDICES .......................................................................................................... 95

A. STUDENT QUESTIONNAIRE (TURKISH) ......................................... 95

B. STUDENT QUESTIONNAIRE (ENGLISH) ....................................... 100

C. INTERVIEW QUESTIONS (INSTRUCTORS) (TURKISH) .............. 105

D. INTERVIEW QUESTIONS (INSTRUCTORS) (ENGLISH) .............. 108

E. INTERVIEW QUESTIONS (PROG. TEST.) (TURKISH) .................. 111

F. INTERVIEW QUESTIONS (PROG. TEST.) (ENGLISH) ................... 114

G. STATISTICAL TABLES ...................................................................... 117

H. SAMPLES OF THE STUDENTS’ ANSWERS ................................... 120

I. METU ETHICAL COMMITTEE APPROVAL .................................... 122

J. TURKISH SUMMARY.......................................................................... 123

K. TEZ FOTOKOPİSİ İZİN FORMU ....................................................... 138

xiv

LIST OF TABLES

TABLES

Table 3.1 Demographic Distribution of the Students ................................................. 35

Table 3.2 Distribution of Interviewees’ by Teaching Experience .............................. 37

Table 4.1 Students’ Perspectives on Context ............................................................. 49

Table 4.2 Students’ Perspectives on Input ................................................................. 53

Table 4.3 Students’ Perspectives on Process .............................................................. 56

Table 4.4 Students’ Perspectives on Product ............................................................. 61

xv

LIST OF ABBREVIATIONS

YBU: Yıldırım Beyazıt University

SFL: School of Foreign Languages

EMI: English as the Medium of Instruction

AGE: Assessment in General Exam

ILC: Independent Learning Center

CEF: Common European Framework

SPSS: Statistical Package for Social Sciences

1

CHAPTER 1

INTRODUCTION

1.1 Background of the Study

As a requirement of the 21st century demanding high qualities on the side of

intellectual, social and professional development, individuals are expected to

become competent foreign language users who can communicate with others

easily and effectively through oral and written expression in both real-life and

academic contexts. In this respect, English is of vital importance due to its being

a global and widely used language without any boundaries. This situation brings

about a greater interest in English language teaching and learning in all

educational settings, especially at tertiary level which aims to raise individuals’

language in the light of their academic purposes for their prospective future.

“In course of time, English has established itself as the world language of

research and publication and it is being used by a multitude of universities and

institutes of learning all around the world as the language of instruction”

(Flowerdew & Peacock, 2001, as cited in Tunç, 2010, p. 1). Going beyond its

natural borders, it becomes a global language and there has been a growing

interest in learning English as a foreign language. Teaching and learning English

has been inevitably important in Turkey for a long time. “The highlights of the

initiatives from the early 1980’s include prioritizing English as the dominant

foreign language taught in schools as well as teaching English as the medium of

instruction (EMI) in many higher educational institutions. The initial drive to

implement EMI in higher education stemmed from the assumption that scientific

and technological competitiveness hinged on Turkey’s educated population’s

2

ability to access and publish academic information in English. This sentiment is

still true today and has spread to most disciplines of academia. In fact, the

number of English-medium universities has skyrocketed since 1995 with the

opening of over 22 new private foundation institutions and a nearly 20 percent

increase in the number of students receiving EMI” (Kırkgöz, 2009, as cited in

Yal, 2011, p. 3). Therefore, when English language teaching is taken into

consideration, English Language School preparatory programs in tertiary

education have a key value in preparing and teaching a foreign language to

university students.

As English has become the language of education in Turkey, Daloğlu (1996)

draws attention to one of the most principal requirements of teaching a language

effectively is having a clearly defined curriculum in terms of its teaching goals

and specific objectives. Thus, achieving high quality in language instruction

especially for academic purposes can be seen the outcome of having a good

curriculum. From this perspective, the evaluation of English Language School

curriculum becomes an integral aspect of curriculum. Evaluation is a course of

actions in which we investigate data in order to make modifications, additions,

alterations or/and eliminations on curriculum. (Ornstein & Hunkins, 1998).

Therefore, it can be said that a systematic and perpetual curriculum evaluation is

not only for improving the quality of teaching English for academic and

communicative purposes but also for providing information to both language

instructors and learners for future strategies.

Considering the significance of English language education, it is vital to

examine perceptions of the students and the instructors in order to provide a

comprehensive perspective of all aspects of the process of learning English

within a preparatory school. Henceforth, curriculum evaluation issue in Yıldırım

Beyazıt University, being a newly opened English-medium university, has

gained importance regarding the status of the English preparatory school

3

program implemented. Furthermore, since the program was newly established,

the effectiveness of the current practices of the program has not been evaluated

yet. Thus, the questions such as what the English instructors’ and students’

perspectives are about the curriculum implemented in Yıldırım Beyazıt

University Preparatory School of English Language in terms of its context,

objectives, presented course content, teaching methods, materials and

assessment, and what components of the curriculum should be maintained,

modified, and eliminated for better success are left unanswered. In other words,

the curriculum needs to be evaluated to gather information about the instructors’

and the students’ perceptions. Consequently, the researcher’s purpose is to find

the answers to these questions, “as all the programs need to be evaluated to find

out whether the developed and organized experiences are producing the

intended outcomes or results and to diagnose the strengths and weaknesses of

the plans and organizations” (Tyler, 1949, as cited in Tunç, 2010, p. 6).

1.2 Context of the Study

Yıldırım Beyazıt University (YBU) was founded in 2010 as the fifth public

university in the capital of Turkey, Ankara. Having 7 faculties, 1 college, 4

institutes, and 1 conservatory, it aims to meet the needs of new generations. The

key objective of the university is to support students in the days they would take

the first steps in life and to open new horizons for both our students and society.

Knowing the importance of language education and English medium of

instructions, the university’s most of the departments including faculty of

Medicine and Engineering are designed as a 100 % English medium instruction.

Therefore, the English Preparatory Program at YBU has a very crucial

prominence in terms of preparing the students for their future career as highly

competent English language users in academic and social contexts (ybu.edu.tr,

2015). The ultimate goal of the English Preparatory Program is expected to

equip the university level students with essential academic language skills,

4

mostly focusing on reading and writing for academic purposes, which are

required for pursuing their departmental courses.

The mission of the School of Foreign Languages (SFL) at Yıldırım Beyazıt

University, is to equip its students with English language skills in accord with

the quality standards of international level. It aims to enable the students at YBU

develop their study skills needed to follow departmental courses, become

independent language learners, make an effective use of this language in their

academic studies and professional lives. To achieve the mission and purposes of

SFL at YBU, the curriculum development process within the framework of

international standards is set in line with the Common European Framework

(CEF), the policies of the School of Foreign Languages of Yıldırım Beyazıt

University and the needs and interests of the students.

The preparatory school curriculum runs on a modular system. The seven (7)

modules in the program are named as Basic / (A) and Basic + / (A+) for the

elementary levels, Independent / (B) and Independent + / (B+) for intermediate

levels, Upper / (C), Upper + / (C+) and Advanced / (C++) for advanced levels.

In relation to the level specifications of CEF, the basis for the curriculum by

setting the international standards, each module has different goals and

objectives.

The program has been designed by experienced instructors in English Language

Teaching at academic level by the Curriculum Planning Unit. It is composed of

five skills integrated as Grammar, Reading, Writing, Listening, and Speaking.

The classrooms of Preparatory School at Yıldırım Beyazıt University are mostly

composed of 25 students in each. The students are assessed at the beginning,

during and at the end of the process. The students who enter the university first

take the Placement Test. The aim of which is to place the students at the

5

appropriate level. The students who get the required score at the Placement Test

have the right to take the Assessment in General Exam (AGE). Those who get

69,5 out of 100 start their departments. Those who get below 69,5 are placed at

the suitable level in accordance with their grades and enroll the preparatory

program.

1.3 Purpose of the Study

As one of the newly opened English-medium University, YBU has the

responsibility of teaching English with its preparatory English Language school.

Therefore, it is vital to develop an effective program as it plays an important role

in the language learning and teaching process. By means of providing a

thorough picture of curriculum, its evaluation helps administrators, instructors,

and curriculum planners to make wise decisions, changes, additions and

deletions to develop and improve students’ English Language skills.

The aim of the curriculum of School of Foreign Languages Preparatory

Program at Yıldırım Beyazıt University is to provide students at different

proficiency levels with a knowledge of basic English so that they can develop

and enrich their vocabulary, comprehend and respond to what they hear and

read; communicate and express themselves effectively in various academic,

professional and social contexts. For this purposes, it started to be implemented

in the academic year of 2013-2014. As newly implemented, it is open to be

changed and improved. Evaluation of a curriculum is necessary at this phase to

provide the basis for improvement and feedback on continuous curriculum

adjustments and processes of curriculum implementation. There is a great need

for school of foreign languages instructors to improve the students’ learning

process in order to meet their needs to teach effectively. Therefore, informing

stakeholders about the problems of the preparatory school curriculum plays a

crucial role in qualifying teaching English language. Students’ perspective is

equally important in curriculum evaluation because it helps to facilitate the

6

understanding of the impact and outcome of the education program. Knowing

the extent to which students have achieved the outcomes specified in the

curriculum is fundamental to both improving teaching and learning. Thus,

context, input, process and product components of the curriculum are to be

evaluated.

Curriculum evaluation is an important way of determining whether a program is

functioning in reality as it was planned and it accomplishes its function.

Therefore, it refers to a formal and systematic determination process conducted

to decide about future of a program. The evaluation of the curriculum responses

whether it achieves the intended goals of the program in relation to context,

input, process and product and suggest relevant adaptations by contributing to

the improvement by means of the collection and analysis of data.

To accomplish these aims, curriculum workers may follow different ways

because there is not one best way. The purpose of the evaluation, the nature of

the program, the individuals or stakeholders involved, and on the timescales and

resources available leads the way to conduct an evaluation. It is also vital to

perform them in a principled, systematic manner.

The aim of this study is to examine the curriculum of English Preparatory

Program of Yıldırım Beyazıt University School of Foreign Languages based on

the perspective of instructors’ and students by applying context, input, process

and product components of the CIPP evaluation model developed by

Stufflebeam (1971).

1.4 Research Questions

Depending of the purpose of the study, the following research questions were

developed by the researcher:

7

1. What are the intermediate level students’ perspectives about the English

curriculum implemented at Yıldırım Beyazıt University Preparatory School of

English Language in terms of its objectives, course content, materials and

teaching methods, and assessment procedures?

2. What are the English instructors’ perspectives about the English curriculum

implemented in Yıldırım Beyazıt University Preparatory School of English

Language in terms of its objectives, course content, materials and teaching

methods, and assessment procedures?

3. What are the strengths and weaknesses of the curriculum implemented in

Yıldırım Beyazıt University Preparatory School of English Language?

1.5 Significance of the Study

Due to the high demand of English language education in higher educational

institutions, preparatory schools have gained a tremendous amount of attention.

Therefore, there is a great need for Foreign Language Departments of the

universities to meet this necessity, giving an effective English Teaching

Programs along with identifying the students’ needs. Being one of the new

established universities, the curriculum of School of Foreign Languages

Preparatory Program at Yıldırım Beyazıt University has been designed for

teaching English proficiently and this brings about issues waited to be

developed. With the light of a broad literature review, CIPP model is chosen to

provide a systematic approach to evaluate many aspects of the program to

provide an analytic and rational basis for program decision-making, based on

the students’ and instructors’ perceptions. In this manner, the results of the

research study is hoped to give an in-depth understanding about how to improve

the English Preparatory School Program of YBU. The systematic evaluation will

focus on the context, input, process and product components of the program by

providing a holistic view through the students’ and the instructors’ perceptions.

Moreover, CIPP model is one of the most commonly used models especially in

8

curriculum evaluation although there are numerous models and approaches. In

this study, the most suitable model is CIPP model as it clearly centers on

decision making and the findings of this study will be significant for

improvement of the present curriculum.

This research study is aimed to be a guide for administrators, instructors, and

curriculum planners in English Preparatory School Programs of universities

offering English Medium Instruction (EMI). In this context, the study will

inform the curriculum workers about the perceptions of both students and

instructors and the insights along with the strength and weaknesses of the

program. The outcome of the study is aimed to provide a framework for

developing a better curriculum by defining how effective the practices are

regarding to the students’ and instructors’ ideas. With the help of this particular

study, administrators can make necessary changes, additions, deletions,

adaptations, and decisions to improve students’ competencies.

This comprehensive research study is expected to serve as a sample for

universities’ preparatory school programs to understand the deficiencies to be

improved since the results of the study will provide information in relation to the

materials, teaching and assessment practices.

1.6 Definition of the Terms

Instructors: This refers to lecturers who are working at YBU English language

school during the 2014-2015 academic years. These lecturers are recruited by

the university and implement the curriculum set by the YBU School of English

Language at least in this year.

Students: This refers to learners who are studying English Preparatory School

Program of YBU during 2014-2015 academic years. These learners are studying

9

English language program to be proficient in English as a requirement of their

undergraduate degrees.

Context Evaluation: Context evaluation refers to need assessment and provides

a rationale for determination of objectives of the curriculum. It questions what

program already exist helps in defining objectives for the program and helps

evaluating problems, properties, and opportunities within a defined community

and environmental context (Stufflebeam & Shinkfield, 2007). According to

Stufflebeam and Shinkfield, it aims to explain the relevant context, identify the

target population and assess the needs, diagnose problems underlying the needs,

and judge whether project goals are sufficiently responsive to the assessed

models. The context evaluation phase answers the extent to which, or in what

aspects, a change will affect the main dimensions of a project (Gibton, 2002,

s.193). For this study, context evaluation is used to identify the goals to

recognize the needs.

Input Evaluation: Input evaluation refers to assessment of strategies for

achieving the objectives set by Preparatory Program at Yıldırım Beyazıt

University School of Foreign Languages. In the input evaluation phase,

alternative strategies to realize the aims of a program are taken into account,

how the preferred strategy will be applied, how personnel will be evaluated, and

whether they are competent to apply the strategy (Stufflebeam, 1995, p. 167), so

it is essential for decision making on matters of design. In other words, input

evaluation is conducted to give information on how to make use of resources to

realize the aims of a program (Mohebbi, et al., 2011, p. 3287). The study, in this

phase, will point out what resources are available, what alternative strategies for

the program should be considered, and what plan seems to have the best

potential for meeting needs facilitates design of program procedures.

Process Evaluation: Process evaluation indicates monitoring the project’s

procedural barriers and unanticipated defects, identifying needed in-process

10

project adjustments, obtaining additional information for corrective

programmatic changes, documenting the project implementation process, and

regularly interaction with and observing the activities of project participants

(Stufflebeam & Shinkfield, 2007). In other words, process evaluation provides

information to understand and overcome procedural difficulties and decisions.

Product Evaluation: Product evaluation refers to identifying and interpreting

the program outcomes. It includes measuring, understanding, and judging the

program’s outcomes. In this study, the obtained findings, and program

attainments are pointed out for this phase, and the changes as the result of

attending the preparatory courses at YBU are discussed.

11

CHAPTER 2

REVIEW OF THE LITERATURE

This chapter provides an overview about curriculum evaluation and the CIPP

model developed by Stufflebeam shortly focusing on different conceptions of

curriculum and different curriculum evaluation models. Since the study aims to

evaluate an English language preparatory school curriculum through CIPP

model, a review of the Stufflebeam’s Context, Input, Process and Product Model

including basic principles and the components of the model are presented.

Afterwards, relevant research studies carried out on preparatory curriculum and

curriculum evaluation in Turkey and abroad are presented and finally a

summary of literature review is included.

2.1 Curriculum Evaluation

Curriculum evaluation is an important way of determining whether a program is

functioning in reality as it was planned. Evaluation, which is “the systematic

collection of information to assist in decision making” (Gredler, 1996, p. 3), is

important because it “makes possible the assessment of the quality of curriculum

once it is put in place as well as maintenance of that curriculum on an ongoing

basis” (Brown, 1995, p. 24). Therefore, curriculum evaluation mainly

emphasizes on gathering information about the curriculum so as to identify

whether it works successfully and responds to not only implementer but also

learner needs. It is an important way of assessing if a program is functioning in

practice as it was indented. And as a result it supports improving the quality of

program up to the identified weaknesses (Erozan, 2005). Therefore, it refers to

a formal and systematic determination process conducted to decide about future

12

of a program in terms of whether it is working well or not by means of the

collection and analysis of data. In this regard, it helps ensuring quality as it

allows a program to be improved when weaknesses are identified, and it keeps a

strong and successful program untouched by verifying its success (Rosenbusch,

1991). To accomplish this aim, curriculum workers may follow a variety of

models, depending on the purposes and the types of decisions, which have been

suggested in the educational literature for conducting program evaluation.

Additional significant reason for evaluating curriculum is to gain information

about a planned change. The process of evaluation can usually inform the nature

and implementation of an innovation. Concisely, evaluation is an indispensable

part of curriculum and it is necessary to make decisions about it. The purpose of

the evaluation, the nature of the program, the individuals or stakeholders

involved, and on the timescales and resources available leads the way to conduct

an evaluation.

2.2 Summative Evaluation and Formative Evaluation

Evaluations may be conducted at any of several stages of program development

and implementation. “An evaluation may be a pilot study of an early version of

a program or it may be a review of an established operation with the goal of

possible change or termination” (Cronbach, 1982, p. 2). Michael Scriven (1967)

differentiated these two roles of evaluation and introduced into the literature of

evaluation the concept of formative and summative evaluation.

Formative evaluation involves gathering and sharing data for development of a

program. During the planning of the program, the formative evaluator’s job is to

give information about the program to the program planners and staff in order to

help adjust it to the setting and improve it (Morris & Fitz-Gibbon, 1978).

Generally, formative evaluation is performed throughout the development of a

program, and it is frequently performed more than once (Scriven, 1991). The

13

aim thw type of evaluation is to confirm that “the goals of the instruction are

being achieved and to improve the instruction if necessary by means of

identification and subsequent remediation of problematic aspects” (Weston, Mc

Alpine & Bordonaro, 1995). For Scriven (1991), formative evaluation gathers

information to assess the effectiveness of a curriculum and guide school system

choices as to which curriculum to adopt and how to improve it. Hence, it

provides data to allow immediate changes to be made when it is needed. An

important aspect of formative evaluation is being conducted during the

operation of a program in order to supply information for policy makers and

program directors to be evaluated in improving the program. In brief, as an

ongoing assessment, formative evaluator focuses on providing immediate

feedback to the developers to check current status of the program and to make

necessary revisions in terms of modifying and planning the upcoming changes.

Summative evaluation is conceptualized by Scriven (1991) as “done for or by

any observers or decision makers who need evaluative conclusions for any

reasons other than development” (p. 20). Summative evaluation is conducted at

the end of the program to provide program developers or commissions with

decisions about that program’s achievements, unintended consequences and to-

be- improved practices. The summative evaluation is more outcome-focused

than process, and its function is not to work with the policy makers or program

developers proposing changes for improvement while the program is ongoing,

but rather to gather data and offer a summary report demonstrating in what

extend it has reached its goals, objectives, and outcomes. Subsequently,

summative evaluation allows policy makers to make result-based decisions on

future implementations and develop a better understanding of the process of

change, and finding out what works, what doesn’t and why.

While formative evaluation occurs during the implementing of a new or existing

curriculum, summative evaluation aims at getting the total picture of the quality

14

of a produced and then taught curriculum (Ornstein & Hunkins, 1998).

Formative evaluation make decisions about program development including

revisions and changes possible, on the other hand, summative evaluation brings

about results related to program continuance.

Furthermore, these two evaluations address different audiences. While formative

evaluation is suitable for program administrators and staff, who are responsible

for developing the curriculum, summative evaluation address potential

consumers; students, teachers and other professionals, who fund sources and

supervisors. For these reasons, summative evaluation practices more numerical

data when it is compared to formative evaluation. Despite the fact that

summative evaluation is a necessity to make decisions during the developmental

stages of the various components of unites in a particular program, summative

evaluation is vital to judge the summed effects of the stabilized program to

determine its future.

2.3 Curriculum Evaluation Approaches

In the light of a present review of literature on curriculum evaluation, it reveals

that evaluation has a very long history leading to a wide variety of curriculum

evaluation models which can be used to evaluate a program. According to

Gredler (1996), the purpose of the evolution is to determine the extent to which

curriculum had achieved its stated goals. Evaluation is the basis for the

identification of strengths and weaknesses in the program, followed by re-

planning, implementation and evaluation. Depending on the purposes and the

types of decisions, a number of approaches on program evaluation have been

suggested, so choosing an appropriate model is essential. Although researchers

may select different models for different implementations, the choice may

change in accordance with the nature of the program, participants, or the

evaluation purpose. The important point is to decide which model is the most

suitable for the curriculum to be evaluated if it is not suggested already. The

15

models vary relating to different curriculum evaluation approaches; underlying

reasons behind these differences of classifications are generally associated with

the evaluators’ diverse philosophies, methodologies, values and perspectives.

As Erden (1995) states, researchers can choose the most appropriate model in

terms of their purposes and conditions during their curriculum evaluation

processes or they can develop a new one making use of the existing ones. Over

the years, a number of approaches conducing program evaluation have been

proposed in the curriculum evaluation literature. Fitzpatrick, Sanders and

Worthen (2011) classify the evaluation approaches under the categories of

objectives-oriented evaluation approach, decision-management oriented

evaluation approach, consumer-oriented evaluation approach, expertise-oriented

evaluation approach, and participant-oriented evaluation approach.

Objectives-Oriented Evaluation Approaches can be differentiated by their

purposes that are specified, and then evaluation focuses on the extent to which

those purposes are achieved. Fitzpatrick, Sanders and Worthen (2011) elucidate

that in education, the activity could be as short as one-day classroom lesson or

as complex as the whole schooling enterprise. It is also clarified that the

information gained from an objectives-oriented evaluation approaches could be

used to reformulate the purposes of the activity, the activity itself, or the

assessment procedures and devices used to determine the achievement of

purposes. R.W. Tyler’s objective-driven model and Provus’s Discrepancy

Evaluation model are the most influential example of objective-oriented

evaluation approaches.

Decision-Management Oriented Evaluation Approaches is meant to serve

decision makers. According to Fitzpatrick, Sanders and Worthen (2011), its

rationale is that evaluative information is an essential part of good decision

making and that the evaluator can be most effective by serving administrators,

policy makers, boards, practitioners, and others who need good evaluative

16

information. Management-oriented evaluation approaches highlights different

levels of decisions and decision makers, this approach clarifies who will use the

evaluation results, how s/he will use them, and what aspect(s) of the system s/he

is making decisions about. The decision maker is the audience to whom a

management-oriented evaluation is directed, and the decision maker’s concerns,

informational needs, and criteria for effectiveness guide the direction of the

study (Fitzpatrick, Sanders & Worthen, 2011, p. 88). The Stufflebeam’s

‘decision-oriented evaluation approach’, which is designed to help

administrators make good decisions, is recognized as the CIPP model.

Stufflebeam views evaluation as the process of delineating, obtaining and

providing useful information for judging decision alternatives. These processes

are executed for four types of administrative divisions each of which represents

a type of evaluation; Context, Input, Process and Product. These evaluations

may be conducted independently or in an integrated sequence (Gredler, 1996).

Another very popular management model is Michael Patton’s Utiliation Focused

Evaluation model (1997) aiming to serve decision makers’ needs in managing

programs.

Consumer-Oriented Evaluation Approaches are supported by independent

agencies or individuals who take responsibility to gather information on

educational or other human services products, or assist others in doing so. They

can be considered as summative evaluation approaches. Fitzpatrick, Sanders and

Worthen (2011) explain that “these products generally include: curriculum

packages, workshops, instructional media, in-service training opportunities, staff

evaluation forms or procedures, new technology, software and equipment,

educational materials and supplies, and even services to agencies” (p. 100).

Michael Scriven is the most noted proponent of this type of evaluation (Payne,

1994). Eisner’s (1985) Connoisseurial model of evaluation is another well-

known example of Consumer-Oriented Evaluation Approaches. This approach

asks the evaluator to approach an assessment with the goal of revealing

17

something to the neither consumer that a connoisseur would know and see that

the consumer may nor (Pinar, Reynolds, Slattery & Taubman, 1995).

Expertise-Oriented Evaluation Approaches to evaluation, “probably the oldest

and most widely used, depend primarily upon professional expertise to judge an

institution, program, product or activity” (Fitzpatrick, Sanders & Worthen,

2011, p. 112). It is based on judgements on individual knowledge, experience

and its criteria for judging evaluations is use of recognized standards;

qualifications of experts.

Participant-Oriented Evaluation Approaches, responding to an audience’s

requirements for information, rely on understanding and revealing the

complexities of program. Its purpose is fulfilling the needs of stakeholders by

giving an accurate portrayal with an inductive reasoning and discovery, as

opposed to the end product of a plan. Robert Stake’s Participant-oriented

evaluation approaches is one of the most influential examples of this approach.

2.4 Stufflebeam’s Context, Input, Process, Product Model (CIPP)

Stufflebeam is the pioneer of management oriented evaluation approaches to

help managers be able to make correct decisions about the program (Fitzpatrick,

Sanders & Worthen, 2011). Stufflebeam’s model (1971) provides clear and

robust data for decision making with all of its four evaluation stages namely as

Context, Input, Process, and Product. One of the strengths of the CIPP model is

to enable evaluators to generate highly important questions in the evaluation

process as it is a very useful and practical tool. Evaluators can determine a

number of questions for each component in the model (Karataş & Fer, 2011, p.

593) and use these in the program evaluation process. Furthermore, the CIPP

model makes preparations for holistic evaluations and includes reconstructed

systemic elements to meet universal evaluation needs (Wei et al., 2012, p. 139)

Similarly, Robinson (2002) states that the CIPP framework was developed as a

18

means of linking evaluation with program decision-making. It aims to provide

an analytic and rational basis for program decision-making, based on a cycle of

planning, structuring, implementing and reviewing and revising decisions, each

examined through a different aspect of evaluation –context, input, process and

product evaluation. The CIPP model is an attempt to make evaluation directly

relevant to the needs of decision-makers during different phases and activities of

a program.

There are types of decision making with regard to planning, structuring,

application and reuse, within in the CIPP model, consisting of four phases

(Stufflebeam, 1971, p. 5). These processes are performed to have four types of

administrative divisions each of which represents a type of evaluation. While

context evaluation supports the detection of current needs and problems in the

program to determine goals and objectives of it, input evaluation includes the

means to achieve these goals and objectives throughout the program. Process

evaluation investigates the performed implementation of these means and

possible modifications for improvement whereas product evaluation tries to

compare the expected findings with the actual ones. Each of these evaluation

components are summarized in the following sub-titles.

2.4.1 Context Evaluation

The original focus of context evaluation is to provide rationale for setting

objectives. (Gredler, 1996). Context evaluation is often referred to as needs

assessment. Context evaluation asks “What needs to be done?” and helps assess

problems, properties, and opportunities within a defined community and

environmental context (Stufflebeam & Shrinkfiel, 2007). According to

Stufflebeam and Shrinkfield, the purpose of context evaluation is to describe the

relevant situation, identifying the target population and assess its needs. The aim

is also identifying opportunities for addressing the needs, diagnose problems

underlying the needs, and judge whether project goals are sufficiently

19

responsive to the assessed needs. Data for context evaluation can be gathered

through questionnaires, system analyses, document reviews, interviews,

observations, and diagnostic tests. “The results of a context evaluation are

intended to provide a sound basis for either adjusting or establishing goals and

priorities and identifiny needed changes” (Stufflebeam & Shinkfield, 1985, p.

172). According to Orntein and Hunkins (1998), context evaluation includes the

setting the program takes place. It aims to explain the appropriate environment,

describe the required conditions to that environment, focus on unmet needs and

missed opportunities and diagnose the reason for unmet needs. “Context

evaluation is actually an environmental analysis a reading of the reality in which

the individuals find themselves and an assessment of that reality in light of what

they want to do. This diagnosis stage of evaluation is not a one-time activity. It

continues to furnish baseline information regarding the operations and

accomplishments of the total system” (Ornstein & Hunkins, 1988, as cited in

Tunç, 2010, p. 26). Furtermore, Demirel also (2007) states that context

evaluation is the phase in which current situation and all the factors of the

program are evaluated. The aim of context evaluation is collecting information

for identification of the objectives and goals. It focuses on unmet needs, missed

opportunities, and the reasons why needs weren’t met (Demirel, 2007).

2.4.2 Input Evaluation

The current perspective of input evaluation is searching out and critically

examining potentially appropriate approaches intended to bring about change

(Stufflebeam & Shrinkfield, 1985, p. 173). From this point of view, it is

undeniable to say that input evaluation helps prescribe a project to address

identified needs. It asks “How should it be done?” and identifies practical

designs and educational strategies that will most likely achieve the desired

outcomes. According to the authors, the purpose of input evaluation is to help

the audience consider alternatives in terms of their particular needs and

circumstances and to help develop a better plan (Stufflebeam & Shrinkfiel,

20

1985). With the help of Input Evaluation, as Yüksel (2010) describes, “the

decision makers can decide on functionality of the plans, definition of the

solution strategies, and selection of resources and activities” (Yüksel, 2010, p.

38). This phase, as it stated, questions whether the targets are clearly stated, the

program’s objectives and instructional strategies are appropriate for the school’s

objectives and instructional strategies, and the other strategies will help the

program to accomplish its goals. As Ünal (2013) states, input evaluation also

serves structuring the decisions. It takes into consideration the options for the

selected strategy in order to meet the needs of program objectives.

2.4.3 Process Evaluation

Process evaluation includes monitoring the project’s procedural barriers and

unanticipated defects, identifying needed in-process project adjustments,

obtaining additional information for corrective programmatic changes,

documenting the project implementation process, and regularly interacting with

and observing the activities of project participants (Stufflebeam & Shrinkfiel,

2007). In a word, the main purpose of process evaluation is to provide feedback

on the implementation, serving two other functions. They are, first, to provide

information to external audiences who wish to learn about the program and,

secondly, to assist program stuff, evaluators, and administrators in interpreting

program outcomes (Gredler, 1996, p. 48).

Another important point explained by Gredler is that the success of a process

evaluation highly depends on the evaluator who is responsible for observing and

documenting program activities. Process evaluation is to answer the questions

such as to what extent students have succeeded curriculum objectives or if the

program is functioning adequately. Instructional staff use evaluation results to

judge program performance and consequently, to improve it (Finch &

Crunkilton, 1989). In the same way, as Demirel (2007) states, process evaluation

21

aims to give periodic feedback to the people who are responsible for conducting

the program.

This phase of the evaluation controls the decisions taken during the

implementation of the program and manages the program itself. Process

evaluation techniques include on-site observation, participant interviews, rating

scales, questionnaires, records analysis, photographic records, case studies of

participants, focus groups, self-reflection sessions with staff members, and

tracking of expenditures (Zhang, et.al. 2011).

2.4.4 Product Evaluation

Product evaluation of Stufflebeam’s CIPP method (1971) is meant to identify

and assess project outcomes. This phase of evaluation seek answer to the

question “Did the project succeed?” The primary function of product evaluation

is “to measure, interpret, and judge the attainments of a program” (Stufflebeam

& Shrinkfiel, 1985, p. 176). It also questions which needs of all the participants

were met and whether the program accomplished its general goals and skills.

According to the authors, the primary use of product evaluation is to determine

whether a program should be continued, repeated, and/or extended to other

settings. Product evaluation is an indispensable part of an evaluation as it serves

as a guide in order to moderate the program for a better serve the needs of the

participants (Stufflebeam & Shinkfield, 1985). Product evaluation aims to assess

program’s quality. Hence it focuses on program graduates, as it is the end

product of a curriculum. This evaluation occurs in a realistic work environment

in order to determine the worth of curriculum. Employers and supervisors can

also be included in the evaluation as the data sources (Finch & Crunkilton,

1989). Demirel (2007) also clarifies that evaluator gathers data about the

product of the program and focuses on the difference between the expected

output and the actual output. The data gathered through product evaluation gives

22

information to decision makers about whether or not the program continue, what

would be the changes and how these changes would be made.

2.5 Curriculum Evaluation Studies Conducted Abroad and In Turkey

The literature is analyzed in terms of curriculum studies conducted in both

Turkey and abroad in terms of their methodologies and results in order to set a

research background framework.

To begin with, Bayram (2011) conducted a study to evaluate the English

Preparatory Curriculum at TOBB University of Economics and Technology

Department of Foreign Languages from teachers’ and students’ point of view.

The evaluation is based upon CIPP method. The sample of this study was

formed from all of the students (366) studying at the English Preparatory

Program at TOBB ETU Department of Foreign Languages during 2009-2010

academic year and the teachers (39) implementing the same program. Since the

aim of this study was to determine what the teachers and students think about

the program, survey method was used when conducting the study. The

questionnaire was developed by the researcher for the purpose of data

collection. The results of the study show that teacher’ opinions regarding the

context, input, process and product of the program tend to be more positive than

that of the students. When the four dimensions of the program are concerned, it

has been concluded that the mean scores of the teachers are higher than the

mean scores of the students.

The effectiveness of an in-service teacher training program was evaluated by

Şahin (2006) with the use of CIPP model. In this study, the Department of Basic

English and the Department of Modern Languages of the School of Foreign

Languages at Middle East Technical University (METU) were by answering the

question whether it achieved its objectives. Suggestions regarding the future of

the programs were provided. The results of the study revealed that while the

23

curriculum was affective in terms of achieving its objectives, there could be

improvements in some part of the program. The main problem of the program is

the model was nonlinear, which made it difficult to concentrate on a particular

level of evaluation at a particular time. Therefore the suggestion for a more

linear and definite model for the evaluation of the program was proposed.

Gerede (2003) conducted a study to evaluate the effects of curriculum renewal

project at Anadolu University. To collect data, questionnaires and interviews

were used to evaluate the old and renewed curricula of Preparatory Program

depending on the students’ perceptions. The perceived language needs of the

students were the base for the evaluation at five English-medium departments at

Anadolu University in terms of meeting students’ language needs. The findings

of the study revealed that there were significant differences between the two

curricula, and relevant suggestions were made for the renewal projects in terms

of students’ language needs.

Tunç (2010) carried out a study which evaluates how effective Ankara

University Preparatory School program by gathering the perspectives of

instructors and students. The researcher applied CIPP evaluation model. The

sample of the study was 406 students and 12 instructors in the 2008-2009

academic years. A self-reported student questionnaire and an interview schedule

were utilized to gather data about their perceptions. In order to obtain more

detailed information about the preparatory school, written documents were

examined. Results of the study indicated that the program at Ankara University

Preparatory School partially served for its purpose, but some improvements in

the physical conditions, content, materials and assessment dimensions of the

program were required to make the program more effective.

24

Another evaluation study was carried out by Yanık (2007) who aimed to

evaluate the English language curriculum of the sixth, seventh and eighth grades

of public primary schools. The major areas of investigation were the teachers’

and students’ perceptions of the curriculum goals and content, instructional

strategies, evaluation and assessment procedures, learner attitudes and the

problems encountered during the curriculum implementation. Data were

collected from 368 teachers and 1235 students randomly selected from the 21

cities and 42 towns of the seven regions of Turkey through a questionnaire for

teachers and students. The results revealed that the implementation process of

the English language curriculum showed differences in relation to the facilities

of schools and classrooms, teacher and student characteristics and their

perceptions.

Özkanal (2009) conducted an evaluation study to investigate the English

Preparatory Program of Eskişehir Osmangazi University, Foreign Languages

Department. The aim of this study was to find out whether the program was

successful and suggest a new model for the Preparatory Program. To obtain data

for the study, two questionnaires and an interview were carried out with 354

students and 27 instructors of the program. The results revealed that there were

some problematic areas particularly in technical English. Consequently, the

study proposed the necessity for an English Preparatory model and an increase

of the qualities of the program.

Tekin (2015) carried out a study which evaluates the English Language

Teaching (ELT) and English Language and Literature (ELL) preparatory classes

of a state university in Turkey through CIPP aiming to gather data for an

illuminative evaluation of the prep classes program in order to highlight certain

dimensions of the current program to make the necessary changes. In this

respect, 106 preparatory class students and two lecturers participated in the

study. To get more reliable results, the data were triangulated with qualitative

25

and quantitative means. The results revealed that the majority of the participants

were satisfied with the present program and its components except for physical

conditions. The lecturers reported the need for a new program that should aim to

help students gain more communicative skills. Although the majority of the

participants are pleased with the existing program, there is still need for a

change, especially in terms of the physical conditions, according to the findings.

One more study was carried out by Akar (2009), whose study aimed to

understand the effectiveness of the foreign language teacher training colleges

(FLTTC) in Poland. Also, the difficulties the teachers experienced were

investigated via a two-way mixed method, a case study and survey. With an

understanding of in-depth information related to the purpose and process of this

program, the findings of the study revealed that FLTTCs were mainly used so as

to learn a foreign language and to get a better job. Additionally, the study

suggested that the participants generally had positive perceptions of their

teaching in the classroom.

Another study was carried out by Karataş (2007) by using Stufflebeam’s (1971)

CIPP model. The researcher evaluated the syllabus of the English II Language

program applied in the Modern Languages Department at Yıldız Teknik

University (YTU). Two questionnaires were administered to the teachers and the

students to collect data and the findings of the study revealed that there were

some significant differences between the teachers’ and the students’ opinions.

The study also revealed that the program did not provide necessary knowledge

of English in terms of business English and there should be variety in audio and

visual materials. Furthermore, the results showed that the program should give

importance to speaking, listening and group activities for gaining business

English skills after the program objectives were revised.

26

Biçer (2009) carried out a study to provide a general picture of newly adopted

IT program in vocational high schools. This study was designed as a formative

evaluation based on CIPP Model-Process Evaluation to see what extent the

objectives and content of Vocational high schools’ new IT curriculum satisfied

the needs, how it was implemented in schools, how it was practiced under

different circumstances and which factors influenced its implementation process

were examined. The research study included 683 Grade 11 students and 83 IT

teachers from 28 vocational high schools in the 7 urban district of Ankara.

Descriptive statistics and quantitative data analysis techniques were utilized to

analyze the data gathered via interviews and questionnaires developed by the

researcher. Results of the study indicated that the objectives of the new IT

program were responsive to the local, national and global IT sector and catching

the demanded skills in the world of work. Therefore, this study also attempted to

reveal the problems related to students, teachers, schools quality indicators and

program modules in order to supply the deficiencies in the early stages of

program implementation.

Steinart and Snell (2005) conducted a study to evaluate a faculty development

project to support the teaching and evaluation of professionalism of medical

students by the use of CIPP evaluation model. To collect data, 152 faculty

members with important educational responsibilities attended one or more

development activities. Then, all faculty chairs, undergraduate and postgraduate

program directors were invited to a half-day workshop. 35 participants attended

the workshop, as it was limited, to test out the working definitions of the

attributes of professionalism, examine the strengths and weaknesses of diverse

teaching methods, and receive immediate feedback. According to the results of

the participators, there were agreement on the cognitive base and attributes of

professionalism, consensus on the importance of teaching and evaluating

professionalism, and self-reported changes in teaching practices.

27

Moreover, Combs et al. (2008) also conducted an evaluation study on online

courses based on the CIPP evaluation model because of its flexibility in

providing formative and summative results. The result of the study revealed

that; with the online learning students in the faculty-wide, the grading of

assignments should be detailed as the face-to-face component is absent, and

students tend to be reluctant to email or post questions. The absence of the

friendly give-and-take presented in face-to-face conversation made written

criticism seem harsh, so it was particularly important for the instructor to stress

not only the positive points but also the specific details to be improved. A quick

turnaround time was also essential, so students were able to tackle future

assignments with increased success.

Nam (2005) carried out a study in South Korea, which focused on perceptions of

college students and their English teachers regarding the new communication-

based English curriculum and instruction in a specific university-level English

program. The findings of the study showed that while students generally seemed

to have somewhat negative opinions about the effectiveness of the new

curriculum in terms of improving speaking ability, vocabulary knowledge and

serving for future needs. Moreover, the findings showed that the new

communication-based EFL curriculum with its materials and instructional

activities have several weaknesses of; thus, it does not serve to the needs of its

learners.

2.6 Summary of the Literature Review

Based on the literature review, the importance of curriculum evaluation and its

functions are undeniable. In educational settings, it is an essential part of the

curriculum development. With the help of evaluation, strengths and weaknesses

of the program are identified, whether the curriculum is fulfilling its purpose is

discovered. Kelly (1999) defines the curriculum evaluation as the process by

which we attempt to measure the value and effectiveness of any particular piece

28

of educational activity. Obviously, this evaluation process is essential to

enhance students’ learning by providing content that is relevant and appropriate

to their needs.

Similar to different reasons of conducting evaluation, there are various ways to

conduct program evaluation. For this research study, Stufflebeam’s Model CIPP

(1971) is selected as the main evaluation model in the sense that it provides

clear and robust data for decision making with all of its four evaluation stages

namely as Context, Input, Process, and Product. Context evaluation supports the

detection of current needs and problems in the program to determine goals and

objectives of it. Input evaluation includes the means to achieve these goals and

objectives throughout the program whereas process evaluation investigates the

performed implementation of these means and possible modifications for

improvement. Finally, product evaluation tries to compare the expected results

with the actual ones. Owing to the formative nature of the model, some

recommendations and remedies are offered to eliminate shortcomings of the

target program. The problem statement and the research questions are analyzed

and this model is selected as an evaluation model for this research in order to

provide information to the decision makers on the objectives, goals, materials,

instructional activities, learning experiences and assessment.

The literature reviewed clearly indicated that among the evaluation models

aiming different aspects of program evaluation and target audience, CIPP model

has been the most common decision-oriented approach. In the light of this

literature, it can be seen that the common purpose of CIPP is to evaluate the

program from a systematical point of view and the main aim is to find

suggestions to reach a better program by taking insights of the ones that

experiencing it. Researches on curriculum evaluation also indicate that the

participants of curriculum evaluation studies are mainly students who are being

implemented the program and teachers who conduct the program. To collect

29

data for curriculum studies, almost all of them use questionnaires and/or

interviews for gathering information about the students’ and the teachers’

perceptions on the program which are intended to be evaluated. For the purpose

of analyzing quantitative data gathered from questionnaires, descriptive

statistics, mean and standard deviations are carried out by the use of a software

program, SPSS. All qualitative data gathered from the interviews are analyzed

via content analysis.

30

CHAPTER 3

METHOD

This chapter covers information related to the design followed in the study,

research questions, participants, instrumentation, data collection procedures,

data analysis, reliability and validity, assumptions, and limitations of the

research.

3.1 Overall Design of the Study

In this study, the aim was to evaluate Yıldırım Beyazıt University School of

Foreign Languages Preparatory English curriculum through the perspectives of

instructors and students using context, input, and process and product

components of the CIPP evaluation model developed by Stufflebeam (1971).

Within the scope of this evaluation study, information collected from eight

instructors who are currently working at preparatory program of Yıldırım

Beyazıt University School of Foreign Languages, two members of program and

testing office, and all students of intermediate level.

Both quantitative and qualitative data were collected from the participants in

order to describe their characteristics and a convergent mixed method was

selected as the most proper design for this study. A mixed method research

design is a procedure for collecting, analyzing, and “mixing” both quantitative

and qualitative research in a single study to understand a research problem

(Creswell, 2012). In this study, the quantitative data was gathered through a 40-

item-questionnaire from the students and analyzed by the use of descriptive

statistics. As for the qualitative data, semi-structured individual interviews were

31

conducted to the instructors and program and testing office members of the

preparatory school.

The researcher gathered quantitative and qualitative data concurrently. Firstly,

the instructors’ perceptions were examined in terms of their implementation of

the same curriculum, then the learner`s perceptions were surveyed. Both

qualitative and quantitative data were analyzed separately and mix the two data

by merging the findings during interpretation and sometimes during reporting

the results. The “Convergent Design” was the mixed method of this study

because the researcher aims to obtain a more complete understanding from two

data to corroborate findings from different methods. Another reason behind this

selection is, as Creswell (2012) states, convergent design is used when there is

need to collect both types of data in one visit to the field and/or both types of

data are equal value for answering the research questions.

The questionnaire was administered by the researcher and the quantitative data

gathered by the instrument were analyzed by a statistical program, SPSS for

Windows-Version 20 software. The interviews with the preparatory school

instructors and testing office members were also processed by the researcher.

While quantitative data were analyzed through means, standard deviations, and

percentages. The qualitative data were analyzed through descriptive content

analysis for each question and grouping under related components of the

evaluation model. Figure 3.1 summarizes the flow of research study.

32

Figure 3.1 Flow of the Study

33

3.2 Research Questions

The research questions of the study;

1. What are the intermediate level students’ perspectives about the English

curriculum implemented at Yıldırım Beyazıt University Preparatory School of

English Language in terms of its objectives, course content, materials and

teaching methods, and assessment procedures?

2. What are the English instructors’ perspectives about the English curriculum

implemented in Yıldırım Beyazıt University Preparatory School of English

Language in terms of its objectives, course content, materials and teaching

methods, and assessment procedures?

3. What are the strengths and weaknesses of the curriculum implemented in

Yıldırım Beyazıt University Preparatory School of English Language?

3.3 Subject of the Study

The researcher gathered data from all the students studying at the Independent +

/ (B+) level preparatory program at Yıldırım Beyazıt University (YBU) School

of Foreign Languages during 2014-2015 spring semester.

The students who enter the university first take the Placement Test prepared by

Program and Testing Office of Yıldırım Beyazıt University Preparatory School

of English Language. The aim of which is to place the students at the

appropriate level in accordance with their English language knowledge. The

students who get the required score (59,5 out of 100) at the Placement Test have

the right to take the Assessment in General Exam (AGE). Those who are

successful in AGE (69,5 out of 100) start their departments. Those who get

below 69,5 are placed in their level in preparatory schools. The reason for

choosing Independent + / (B+) level students was that they had been in the

institution for 3 periods and all the students who were being implemented the

34

curriculum were the students who successfully completed Basic / (A), Basic /

(A+), and Independent / (B) levels. Subsequently, they were considered to know

much about the preparatory English program and the institution’s physical

environment, facilities, and instructors. They had been exposed to the practices

of preparatory school of YBU long enough to develop a view about the

curriculum.

All Independent + / (B+) level preparatory program students (n = 264) were

aimed to be involved in the study, consisting of 11 classrooms. The data

collection instrument was administered to nearly all of the students (n = 243)

from Independent + / (B+) level. The researcher’s classroom was excluded from

the research study and totally, the subjects of the study were consisted of 243

students from 10 classrooms.

As it can be seen in Table 3.1, among the participants, 56 % of them were

female (n = 136), and 44 % were male (n = 107). The age range of the subjects

was between 17 and 27 (n = 196). 0.5 % (n = 1) of the students were at the age

of 17, 16.3 % (n = 32) of them were at the age of 18, 46.4 % (n = 91) of them

were at the age of 19, 26.5 % (n = 52) of them were at the age of 20, 4.1 % (n =

8) of them were at the age of 21, 1.5 % (n = 3) of them were at the age of 22, 1.5

% (n = 3) of them were at the age of 23, 1.5 % (n = 3) of them were at the age of

24, 0.8 % (n = 2) of them were at the age of 25, and 0.4 % (n = 1) of them were

at the age of 27.

The number of the participants according to their departments is also displayed

in Table 3.1. 12.4 % of the students were from Faculty of Law (n = 30), while

13.2 % of them were from Faculty of Humanities and Social Sciences (n = 32).

Majority of the participants with the highest ratio 27.3 % were from Faculty of

Political Sciences (n = 66), 24.8 % were from Faculty of Business

35

Administration (n = 60), and 20.7 % of them were from Faculty of Engineering

(n = 50).

Table 3.1

Demographic Distributions of Students

Variables f %

Sex Female 136 56

Male 107 44

Total 243 100

Age 17 1 0.4

18 32 13.2

19 91 37.4

20 52 21.4

21 8 3.3

22 3 1.2

23 3 1.2

24 3 1.2

25 2 0.8

27 1 0.4

Total 196 100

Faculty Law 30 12.4

Medicine 3 1.2

Humanities and Social

Sciences

32 13.2

Islamic Sciences 1 0.4

Business Administration 60 24.8

Engineering 50 20.7

Political Sciences 66 27.3

TOTAL 243 100

36

Departments of the Yıldırım Beyazıt University including Medicine,

Engineering, Political Sciences and Business Administration offer 100 %

English instruction while the Faculty of Law and Humanities and Social

Sciences offer 30 % English instruction. Islamic Sciences offers instruction in

Turkish and after Preparatory School in Arabic Language. According to

demographic information of the participants, 74 % of the participants (n = 179)

were in departments offering 100 % English instruction. 25.6 % of the

participants (n = 62) were in departments offering 30 % English instruction. 0.4

% of the participants were in department of Islamic Sciences which did not offer

English instruction at all.

Ten instructors participated in the study through interviews; 2 elementary level

(Basic / A+), 2 Pre-Intermediate level (Independent / B), 2 Intermediate level

(Indeendent + / B+), and 4 Upper (Advanced / C) level instructors stated their

ideas in relation to content, input, process, and product dimensions of the

evaluation study. Two of the ten instructors interviewed were members of

Program and Testing Office and they were teaching Pre-Intermediate

(Independent / B) and Upper (Advanced / C) Level. Out of 8 Program and

Testing Office members, 2 were interviewed as they were responsible for

monitoring the program and preparing the materials which were used for

students’ assessment. Gathering data about their perceptions about the program

was an inevitable part in this evaluation process as their duties include not only

development and coordination of the testing procedures but also supervision of

the esta