Assessment of the Needs on Evaluation of Different Schools in the Philippines

194

Essentials on Counseling and Education

Assessment of the Needs on Evaluation of Different Schools in the Philippines

Carlo Magno

Asian Psychological Services and Assessment

Marife Mamauag Help University, Malaysia

Neil Parias

Center for Learning and Performance Assessment De La Salle-College of Saint Benilde

The study assessed the evaluation needs of 81 schools nationwide. A survey questionnaire was constructed and validated by measurement experts. Descriptive statistics were obtained to describe the evaluation activities, practices and areas on evaluation needing technical assistance. Findings show that all schools conduct evaluations mostly on teacher performance (N=29.36%). There was an expressed need for technical assistance practically from planning to utilization of evaluation results, especially in areas of instrumentation and data analysis which were rated highly. The t-test results further indicate no significant difference in the needs between public and private schools. Implications and recommendations related to improve effective training programs that were drawn are discussed in the study.

Keywords: Program Evaluation, Assessment, Evaluation Needs

Schools offer different programs to enhance the teaching and the learning process. One way to determine whether certain educational programs needs to be improved or is effective is through proper evaluation. The information generated through proper evaluation is important by aiding practitioners in the education sector for undertaking various decisions and plans for programs offered. Although, very few schools engage in the real concept of evaluation which brings the point of assessing the practices and needs of schools in line with evaluation. There is a need to study how schools

195


undertake their evaluation and their needs in order to identify the problems faced by schools and to target specific ways to improve them. There is also a need to investigate how the practice of evaluation is being conducted in different areas to determine how evaluation is interpreted in different contexts. This report surveyed selected schools in Luzon, Visayas, and Mindanao to determine their needs in evaluation and how evaluation is being practiced currently.

The quality of the countrys basic and higher education continue to improve as it continues to face the challenges of the 21st century. The National Achievement Test has been consistently low supported by low ranking in international benchmarking studies such as the TIMSS. According to the Department of Education, the grim reality has been brought about primarily by two reasons: (1) the country is simply not investing enough in the education sector; and (2) the education establishment has been poorly managed. Different contexts adopt different viewpoints of evaluation. According to Fitzpatrick, Worthen, and Sanders (2004), some authors opt for a systems approach, while others view evaluation as a process of identifying and collecting information to assist decision makers. Others view evaluation as synonymous with professional judgment, where judgment of a programs quality is based on opinion of experts. In one school of thought, evaluation is viewed as the process of comparing performance data with clearly specified objectives, while in another evaluation is seen as synonymous with carefully controlled experimental research on programs. Others urge the importance of naturalistic inquiry or urge that value pluralism be recognized where the individuals evaluated play a prime role in determining what direction the evaluation study takes. These various points of view on evaluation bring about differences in opinion on how evaluation is supposed to be done but the worst are misconceptions on handling evaluations. Even if there are various ways to adapt evaluation, an important area that needs to be met is the end goal of evaluation which is to come up with judgment and overall value of a program and its relative value. The American Evaluation Association made effort to make the Guiding principles for Evaluation to properly guide evaluators. These guide entails that (a) inquiries should be data-based whatever is evaluated, (b) evaluators need to be competent, (c) honesty and integrity are needed in the process, (d) respect of the security, dignity, and self-worth of the respondent is important, and (e)

196


responsibilities for the general and public welfare must be articulated. Schools conducting the process of evaluation must be aware of these guiding principles to put into proper perspectives and justifications their practices.

Most often before a program is started, institutions would conduct needs assessment. An evaluation of a need seeks to identify and measure the level of unmet needs within an organization (Posavac & Carey, 2003). The planning would then be guided by answering the needs that arises. Astin (1993) expresses the need for schools to evaluate regularly since the quality of education provided to students as well as services are based on it. The quality of education likewise is measured through evaluation. Conducting needs assessment is concerned whether a problem or a need exists and to describe the problem of a program and then make recommendations for ways to reduce the problem. Basically, needs assessment determines whether there is a sufficient need existing to initiate a program, and if there is, then there needs to be assistance in program planning by identifying potential program models and activities that might be conducted to achieve goals. McKillip (1998) identified five processes on needs analysis which includes: (1) identification of users and uses, (2) description of the target population and service environment, (3) need identification, (4) needs assessment, and (5) communication. McKillip (1998) explains that needs assessment is to produce recommendations for action. Different studies may focus on the process and monitoring component of a program. Such studies focus on whether the program is being delivered according to some delineated plan or model or may be more openended, simply describing the nature of delivery of the successes and problems encountered (Fitzpatrick, Worthen, & Sanders, 2004). What is Evaluation? The term evaluation has been used rather loosely without definition beyond what was implicit in context. Yet among professional evaluators, there is no uniformly agreed-upon definition of precisely what the term evaluation means. In fact, Michael Scriven, one of the founders of evaluation, recently noted that there are nearly sixty different terms for evaluation that apply from one context to another. These include adjudge, appraise, analyze, assess, critique, examine, grade, inspect, judge, rate, rank, review, score, study, test, and so on (cited in Patton, 2000). Scriven further concluded that

197


the various definitions of evaluation are reflective not only of the immense importance of the process of evaluation in practical life, but the explosion of a new area of study as well. Indeed, evaluation can be described as an elastic word that stretches to cover judgments of many kinds (Weiss, 1997). Lincoln and Guba (1986) defined evaluation as a form of disciplined inquiry that applies scientific procedures to the collection and analysis of information about the content, structure and outcomes of programs, projects and planned interventions. As cited in Clarke and Dawson (1999), few more definitions are as follows:

Program evaluation is the systematic collection of information about activities, characteristics, and outcomes of programs for use by specific people to reduce uncertainties, improve effectiveness, and make decisions with regard to what those programs are doing and affecting (Patton, 1986). Evaluation is a type of policy research, designed to help people make wise choices about future programming. Evaluation does not aim to replace decision makers experience and judgment, but rather offers systematic evidence that informs experience and judgment. Evaluation strives for impartiality and fairness. At its best, strives to represent the range of perspectives of those who have a stake in the program. (Weiss as quoted in Alkin, 1990) Evaluation is usually defined as the determination of the worth or value of something in this case, of educational and social programs, policies, and personnel judged according to appropriate criteria, with those criteria explicated and justified (House, 1993; Scriven, 1991) Evaluation research is the systematic application of social research procedures for assessing the conceptualization, design, implementation, and utility of social intervention programs (Rossi & Freeman, 1993).

198


Early in the development of the field, Scriven (1967) defined evaluation as judging the worth or merit of something. Many recent definitions encompass this original definition (Mark, Henry, & Julness, 1999; Stake, 2000; Stufflebeam, 2001). The agreement was that evaluation is determining the worth or merit of an evaluation object (whatever is evaluated). More broadly, evaluation is identification, clarification, and application of defensible criteria to determine an evaluation objects value (worth or merit) in relation to those criteria (Flitzpatrick, Sanders, & Worthern, 2004). The Conduct of Needs Assessment

A need refers to a discrepancy between what is and what should be.

There are at least five discrepancies that people could have in mind when they speak of needs. There might be a discrepancy between an actual state and (a) an ideal, (b) a norm, (c) a minimum, (d) a desired state, or (e) an expected state. Needs also refers to something (X) that people must to be in a satisfactory state; without X they would be in an unsatisfactory state; with X they achieve but do not exceed a satisfactory state (Posavac & Carey, 2003).

As one of the prerequisites of planning school-wide program change, the North Central Regional Educational Laboratory (1998) suggests that a comprehensive needs assessment should be undertaken first. This is because this process will yield the database from which the planning team develops its vision of the future. Through the needs assessment, a school identifies its strengths and weaknesses and specifies priorities for improving student achievement and meeting challenging academic standards. Conducting a needs assessment helps planners focus better on school-wide issues and link goals with hard data. Every aspect of the school is a candidate for assessment. However, experienced planners advise concentrating on how the school addresses the comprehensive academic needs of all the students in the school, especially students who are educationally disadvantaged, neglected or delinquent, migrant, indigenous people, or vulnerable to the dangers of drug or alcohol addiction (NCREL, 1998). The NCREL (1998) suggests the following steps in conducting needs assessment: (a) clarify the school vision; (b) create a school profile focus areas to consider include student achievement, curriculum and instruction, professional development, family and community involvement, and school

199


context and organization; (c) determine data collection methods and plans; (d) collecting data and summarizing evidence; and (e) analyzing program needs and setting goals. Needs assessment questions are concerned with establishing (a) whether a problem or need exists and describing that problem, and (b) making recommendations for ways to reduce the problem, i. e., the potential effectiveness of various interventions. There are a number of approaches to needs assessment used in the social sciences. The discrepancy model (e. g., Kauffman & English, 1979) is perhaps the most frequently used model in education. This model focuses decision-making on discrepancies identified between what is expected and what occurs. The marketing model (e. g., Nickerns, Puga, & Noriega, 1980) defines needs assessment as a feedback process used by organizations to learn about and adapt to the needs of their client populations. This model focuses on determining the needs of a target population and meeting them. Another approach to needs assessment is that of Keeney and Riaffa (1976). They described a decision-making model that uses decision theory (specifically multi-attribute utility analysis). This model assumes that decision-makers show biases in judgment when confronted with complex, multidimensional information. Purpose of the Study

To determine the needs of school in line with their evaluation process, the following research questions are posed:

1. Do schools conduct evaluation of their different programs? 2. What are the activities that schools commonly evaluate? 3. What evaluation tasks need specific improvement? 4. Who are the personnel that commonly conduct evaluation in schools? 5. What are the different ways employed by schools in gathering data for

evaluation? 6. What are the technical needs of schools in the process of evaluation? 7. What are the different sources of information does schools get in

knowing how to conduct evaluation?

200


8. What are the suggestions for improvement of different schools in the process of their evaluation?

Method

Research Design The design used in the study is descriptive. Through the descriptive design the frequencies and means on the categories of school needs on evaluation are identified. The study employed the quantitative analysis where the frequencies of responses are counted and the percentage is reported. The breakdown of the participants according to region and type of school is shown in Table 1. Table 1 Breakdown of Participants by Region and Type of School Luzon Visayas Public Private Total Public Private Total Frequency 16 28 44 0 15 15 Percentage 36.36 63.64 54.32 0 100 18.52 Mindanao Total Public Private Total Public Private Total Frequency 3 19 22 20 61 81 Percentage 13.64 86.36 27.16 24.69 75.31 100 Participants There are 81 respondents who participated in the study coming from different institutions. The participants were selected through convenience sampling. The position of the respondents varied that includes principals, directors, administrators, assistant principal, coordinators and guidance counselors. The participants basically belong to the upper and middle level management positions.

201


Instrument The researcher together with other experts in measurement and evaluation constructed a needs assessment inventory focusing on the needs of schools in evaluation of their various curricular programs. The survey is composed of 8 items that reflects how evaluation is practiced and identifies the need for various aspects of evaluation. The items identifies (1) if evaluation is being conducted in the school, (2) activities evaluated, (3) activities the needs improvement, (4) who handles the evaluation, (5) how data is gathered during evaluation, (6) areas in evaluation that needs technical assistance, (7) information where evaluation is taken and (8) suggestions how to improve the evaluation. Procedure The instrument was constructed for the purpose of surveying out the needs of schools on evaluation. The survey was reviewed and revised. It was administered to principals, assistant principals, coordinators and other school heads from different schools in Luzon, Visayas and Mindanao. The participants were given a letter and the purpose of the needs assessment was explained to them. The administrators were asked to accomplish the survey form. Some surveys forms were answered immediately and others needs time to answer it and requested to be retrieved in another time. After accomplishing the forms the school heads were again debriefed about the purpose of the assessment. Each school surveyed were furnished with a copy of the needs assessment report together with a transmittal letter.

Results

The data were coded and the frequencies were then converted in to percentages to report the majority of selection of every category on the needs of schools in evaluation.

202


Table 2 Percentage of Conducting Evaluation in Schools Luzon Visayas Mindanao Total

Public Private Public Private Public Private

f % f % F % f % f % f % f %

Evaluation is Conducted

12 75 28 100 15 100 3 100 19 100 77 95.06

Evaluation is not conducted

4 25 0 0 0 0 0 0 0 0 4 4.94

N Total 16 100 28 100 15 100 3 100 19 100 81 100

Table 2 shows that most schools in the Philippines are conducting

evaluation. A large percentage of the participants report that they are conducting evaluation in their schools. There are four schools public school participants who reported that they are not conducting evaluation in their school.

Table 3 Activities that Schools Evaluate

Luzon Visayas Mindanao Total


Activities F % f % F % F % f % f % f %

Teacher Performance

12 75 27 96.43 14 93.33 3 100 19 100 75 92.59

Administrative Performance (e.g., coordinator, principal, director, etc.)

7 43.75 20 71.43 14 93.33 2 66.67 16 84.21 59 72.84

Support Staff Performance

7 43.75 22 78.57 13 86.67 2 66.67 16 84.21 60 74.07

Implementation of New Academic Programs

10 62.5 17 60.75 11 73.33 2 66.67 11 57.89 51 62.96

Teacher Training Programs (e.g.,

12 75 25 89.29 14 93.33 2 66.67 16 84.21 69 85.19

203





seminars, symposia, etc.) Selecting Students for Academic and Special Awards

9 56.25 25 89.29 11 73.33 3 100 15 78.95 63 77.78

Guidance and Counseling Programs

9 56.25 20 71.43 9 60 1 33.33 14 73.68 53 65.43

Homeroom Guidance Program

9 56.25 17 60.71 10 66.67 1 33.33 13 68.42 50 61.73

Administrative Services (e.g., maintenance, engineering, accounting, etc.)

7 43.75 17 60.71 10 66.67 2 66.67 12 63.16 48 59.26

Student Organizations

9 36.25 22 78.57 11 73.83 2 66.67 13 68.42 57 70.37

Student Publications

5 31.25 18 64.29 8 53.33 1 33.33 9 47.37 41 50.62

Sports Development Program

8 50 19 67.86 6 40 1 33.33 12 63.16 46 56.79

Cultural Activities

9 56.25 18 69.23 10 66.67 2 66.67 11 57.89 50 61.73

Community Service

7 43.75 17 60.75 10 66.67 2 66.67 14 73.68 50 61.73

Retreat, Recollection and other Formation Programs

5 31.25 22 78.57 13 86.67 0 0 16 84.31 56 69.14

Service feedback

4 2.5 16 57.71 6 40 1 33.33 9 47.37 36 44.44

Canteen/Cafeteria Evaluation

11 68.75 22 78.57 12 80 3 100 13 68.42 61 75.31

Personalized Student Program

1 6.25 0 0 1 6.66 0 0 0 0 2 2.47

School Building 0 0 1 3.57 0 0 0 0 0 0 1 1.23

System 0 0 0 0 0 0 0 0 1 5.26 1 1.23

204





Performance

Classroom Instruction

0 0 0 0 1 6.66 0 0 0 0 1 1.23

Integrated Admin Performance

0 0 0 0 1 6.66 0 0 0 0 1 1.23

Student Assistance

0 0 0 0 1 6.66 0 0 0 0 1 1.23

Health Services 0 0 0 0 1 6.66 0 0 0 0 1 1.23

Organizational Climate

0 0 0 0 0 0 0 0 1 5.26 1 1.23

Security / Janitorial

0 0 0 0 0 0 1 33.33 0 0 1 1.23

Students 0 0 0 0 0 0 0 0 1 5.26 1 1.23

Alumni Services 0 0 0 0 0 0 0 0 1 5.26 1 1.23

Student Services 0 0 0 0 0 0 0 0 1 5.26 1 1.23

Colloquim 0 0 0 0 0 0 0 0 1 5.26 1 1.23

Table 3 shows that most of the school in the Philippines conduct evaluation on teacher performance. In Luzon, public school commonly evaluate teacher performance and teacher training programs. In Visayas, private schools commonly evaluate teacher performance, administrative performance, and teacher training programs. All public and private schools in Mindanao evaluate teacher performance, selecting students for academic awards and canteen/cafeteria evaluation. The top five activities that are mostly evaluated across the schools and region are (1) teacher performance, (2) teacher training, (3) selecting students for academic awards, (4) support staff, (5) canteen and cafeteria evaluation. The schools does not evaluate much of their school building, system performance, classroom instruction, integrated administrative performance, student assistance, health services, organizational climate, security/Janitorial, students, alumni services, student services, colloquim.

205


Table 4 Activities that Needs to be Improved



Activities F % f % F % f % f % f % f %

Teacher Performance

3 100.0 12 42.86 7 46.67 2 66.67 5 26.32 29 35.80

Administrative Performance (e.g., coordinator, principal, director, etc.)

0 0.0 12 42.86 9 60.00 1 33.33 7 36.84 29 35.80

Support Staff Performance

2 66.7 8 28.57 7 46.67 2 66.67 6 31.58 25 30.86

Implementation of New Academic Programs

5 166.7 9 32.14 4 26.67 1 33.33 3 15.79 22 27.16

Teacher Training Programs (e.g., seminars, symposia, etc.)

4 133.3 14 50.00 8 53.33 1 33.33 6 31.58 33 40.74

Selecting Students for Academic and Special Awards

2 66.7 5 17.86 5 33.33 2 66.67 5 26.32 19 23.46

Guidance and Counseling Programs

2 66.7 11 39.29 3 20.00 0 0.00 5 26.32 21 25.93

Homeroom Guidance Program

1 33.3 7 25.00 6 40.00 0 0.00 5 26.32 19 23.46

Administrative Services (e.g., maintenance, engineering, accounting, etc.)

2 66.7 10 35.71 4 26.67 1 33.33 4 21.05 21 25.93

Student Organizations

1 33.3 7 25.00 9 60.00 1 33.33 7 36.84 25 30.86

Student Publications

1 33.3 6 21.43 8 53.33 0 0.00 5 26.32 20 24.69

Sports Development

1 33.3 11 39.29 4 26.67 0 0.00 9 47.37 25 30.86

206


Program

Cultural Activities

1 33.3 7 25.00 3 20.00 1 33.33 7 36.84 19 23.46

Community Service

1 33.3 11 39.29 2 13.33 1 33.33 6 31.58 21 25.93

Retreat, Recollection and other Formation Programs

2 66.7 8 28.57 4 26.67 0 0.00 6 31.58 20 24.69

Service feedback 1 33.3 6 21.43 4 26.67 1 33.33 5 26.32 17 20.99

Canteen/Cafeteria Evaluation

3 100.0 11 39.29 6 40.00 2 66.67 5 26.32 27 33.33

School Building 0 0.0 1 3.57 0 0.00 0 0.00 0 0.00 1 1.23

Security 0 0.0 0 0.00 0 0.00 1 33.33 0 0.00 1 1.23

Students 0 0.0 0 0.00 0 0.00 0 0.00 1 5.26 1 1.23

Alumni Services 0 0.0 0 0.00 0 0.00 0 0.00 1 5.26 1 1.23

Student Services 0 0.0 0 0.00 0 0.00 0 0.00 1 5.26 1 1.23

Colloquim 0 0.0 0 0.00 0 0.00 0 0.00 1 5.26 1 1.23

Table 4 shows that there is variation in the different activities that needs improvement in evaluation. For both the public and private schools in Luzon, the evaluation on teacher training programs needs to be improved. In Visayas, the evaluation on administrative performance and student organizations needs to be improved. In Mindanao public school, the evaluation on teacher performance, support staff, selecting students for awards, and canteen/cafeteria needs to be improved. For the private schools in Mindanao, the evaluation on sports development needs top be improved. The top three activities where evaluation needs improvement include (1) teacher training program, (2) teacher performance, and (3) administrative performance. The others have tied percentage on their concerns for evaluation improvement.

207


Table 5 Persons who Conduct Evaluations

Luzon Visayas Mindanao


Persons f % f % f % f % f % f %

Guidance Counselor 5 6.17 12 14.81 0 0 8 9.88 2 2.47 15 18.52

Teachers 5 6.17 10 12.35 0 0 4 4.94 3 3.70 11 13.58

Coordinators 4 4.94 5 6.17 0 0 8 9.88 3 3.70 14 17.28

Assistant Principal 0 0 6 7.41 0 0 5 6.17 1 1.23 7 8.64

Principal 11 13.58 19 23.46 0 0 10 12.35 3 3.70 15 18.52

Division Supervisor 4 4.94 0 0 0 0 0 0 0 0 0 0

School President 0 0 1 1.23 0 0 0 0 0 0 0 0

Director 0 0 6 7.41 0 0 0 0 0 0 1 1.23

Human Resources 0 0 1 1.23 0 0 0 0 0 0 0 0

Evaluation Committee 0 0 1 1.23 0 0 0 0 0 0 1 1.23

Deans / VPA / AVP 0 0 0 0 0 0 3 3.70 0 0 0 0

ITEO 0 0 1 1.23 0 0 0 0 0 0 1 1.23

Superintendent 0 0 0 0 0 0 1 1.23 0 0 1 1.23

Students and Parents 0 0 1 1.23 0 0 1 1.23 0 0 0 0

Research / Evaluation Office

0 0 0 0 0 0 4 4.94 0 0 1 1.23

Administrative Council 0 0 0 0 0 0 1 1.23 0 0 1 1.23

Club Moderator 0 0 0 0 0 0 1 1.23 0 0 0 0

Student Affairs Coordinator

0 0 0 0 0 0 1 1.23 0 0 0 0

Program Coordinator 0 0 0 0 0 0 1 1.23 0 0 1 1.23

ARM / TQM Expert 0 0 1 1.23 0 0 1 1.23 0 0 2 2.47

Researcher 0 0 1 1.23 0 0 0 0 0 0 1 1.23

Psychometrician 0 0 0 0 0 0 0 0 0 0 2 2.47

Department Head 0 0 1 1.23 0 0 0 0 1 1.23 2 2.47

Academic Supervisor 0 0 0 0 0 0 0 0 0 0 1 1.23

Evaluation activities in schools are most often conducted by guidance counselors and school administrator, who may be are not properly trained in evaluation and measurement. The public and private schools in Luzon and Visayas, the principal is mostly conducting the evaluation. In Mindanao, the public schools have different evaluators such as the teachers, principals, and

208


coordinators. In the private schools, the guidance counselor and the principal conducts most of the evaluation. Table 6 Data Gathering Procedures for Evaluations

Luzon Visayas Mindanao

Public Private

Public

Private Public Private

Procedures f % F % f % f % f % f %

Inventory/Questionnaires

9 11.1

1 26

32.10

0 0 15

18.52

3 3.70

18

22.22

Focus Group Discussion 3 3.70

13

16.04

0 0 8 9.87 2 2.46

11

13.58

Surveys 5 6.17 8 9.87 0 0

10

12.34

2 2.46

14

17.28

Personal Interview 12

14.81

13

16.05

0 0 12

14.81

3 3.70

10

12.35

Observation 14

17.28

23

28.40

0 0 12

14.81

2 2.46

16

19.75

Tests 10

12.34

14

17.28

0 0 3 3.70 2 2.46

7 8.64

Experiments 3 3.70 4 4.93 0 0 1 1.23 0 0 1 1.23

Workshop 0 0 0 0 0 0 1 1.23 0 0 0 0

Extent Document 0 0 0 0 0 0 1 1.23 0 0 0 0

Visitors 0 0 0 0 0 0 1 1.23 0 0 0 0

School Community Feedback

0 0 1 1.23 0 0 1 1.23 0 0 1 1.23

Meetings 0 0 0 0 0 0 1 1.23 0 0 1 1.23

The top 5 data collection methods in evaluation commonly used by

schools are inventory/questionnaires (n=71.87%), observation (n=67, 82.72%), personal interview (n=50, 61.73%), surveys (n=39, 48.15%), and tests (n=36, 44.44%). These data collection tools seem appropriate to use considering that most often evaluation being practiced is focused on teacher performance. FGDs and community feed back are rarely used in evaluation of schools. In the public schools in Luzon, observation is the most common technique of gathering data while the private schools uses mostly inventories and questionnaires. In Visayas, the private schools also commonly use inventories and questionnaires. In Mindanao, the public

209


schools commonly use inventories and personal interview while in the private schools, majority uses inventories and questionnaires in gathering information for evaluation. Table 7 Technical Needs on Evaluation Luzon Visayas Mindanao


M SD M SD M S

D M SD M SD M SD

Planning the evaluation

2.94

1.124

2.82

1.188

3.25

.662

3.00

.00 3.13

.743

Conceptualization

3.00

1.095

2.82

1.056

3.43

.756

2.67

.577 3.14

.663

Instrumentation (constructing assessment forms, etc.)

3.44

.629 2.75

1.076

3.27

.799

3.00

1.414

3.27

.704

Data Analysis 3.25

.931 2.89

.994 3.46

.660

3.50

.707 3.00

.866

Report Writing 2.75

1.238

2.82

.905 3.09

.701

2.50

.707 2.53

.516

Utilization of results

2.69

1.250

2.86

1.145

3.46

.776

2.50

.707 3.12

.697

Dissemination of results

2.56

1.094

2.71

1.117

2.85

.689

2.0 .000 2.47

.743

Schools highly need technical assistance mostly in instruction and data analysis, implying a need for more expertise in the field of evaluation. Most of the public school in Luzon has a high need on instrumentation in the evaluation process, for the private schools they have a moderate need on data analysis of evaluation results. In Visayas, high need came out on almost all areas but data analysis and conceptualization has the most need. In Mindanao, public schools has a high need also in data analysis while for the private schools, they have a high need on instrumentation.

210


Table 8 Comparison of Public and Private Schools in their Technical Needs in Evaluation t df p value Mean Difference SE

Planning -0.20 71 0.83 -0.05 0.27

Conceptualization -0.42 73 0.67 -0.10 0.25

Instrumentation 1.53 74 0.12 0.37 0.24

Data analysis 0.92 74 0.35 0.22 0.24

Report writing -0.30 70 0.76 -0.07 0.24

Utilization -1.45 74 0.15 -0.40 0.27

Dissemination -0.68 72 0.49 -0.17 0.26

Total 0.38 79 0.69 0.69 1.77

Numerically, it appears that public and private schools rating of needs differ in instrumentation and utilization. The public and private school were compared on the factors on the technical needs of schools on evaluation using the t-test for two independent samples. The comparisons on all factors did not reach significance. This means that the technical needs of public and private schools on evaluation are in the same level.

For the public schools in Luzon, the most of the schools have the discussion group as a source of information in conducting evaluation while the private schools have the seminar and workshops as the venue in learning about evaluation. In Visayas, private schools commonly use books and seminars and workshops in learning about evaluation. In Mindanao, public schools use journals and discussion groups and school based experts as a source of learning about evaluation while private schools have seminars and workshops as the source of information.

211


Table 9 Source of Information when conducting Evaluation

Luzon Visayas Mindanao TOTAL

Public Private Public Private Public Private Public Private Total

Source of Information f % f % f % f % F % f % f % f % f %

Books 12 75 16 57 - - 13 87 2 67 15 79 14 74 44 71 58 72

Journals 10 63 11 39 - - 11 73 3 100 14 74 13 68 36 58 49 60

Internet 10 63 16 57 - - 9 60 1 33 14 74 11 58 39 62 50 62

Discussion Groups 11 69 18 64 - - 10 67 3 100 11 58 14 74 39 62 53 65

School-Based Experts 9 56 18 64 - - 12 80 3 100 16 84 12 63 46 74 58 72

External Experts 4 25 14 50 - - 10 67 2 67 7 37 6 32 31 50 37 46

Seminars/Workshops 14 88 21 75 - - 13 87 2 67 19 100 16 84 53 85 69 85

Colloquia 1 6 5 18 - - 4 27 2 67 3 16 3 16 12 19 15 19

Inter-school Collaboration/Consortium

5 31 17 61 - - 7 47 1 33 9 47 6 32 33 53 39 48

Accreditation/Certification Documents/Manuals

5 31 14 50 - - 12 80 2 67 15 79 7 37 41 66 48 59

Formal Education 0 0 0 0 - - 1 7 0 0 0 0 0 0 1 2 1 1

Number of Cases 10 28 0 15 3 19 19 62 81

Note. N=91

Institutionalization of evaluation is important in school improvement. Theres a high need to improve the technical aspects of evaluation, particularly the methods. The responses from the comments and suggestions on improving the evaluation process in schools were listed then clustered into emerging themes on where each strand would fall under. The strands were grouped into four clusters: personnel, practice, relational, and study. The cluster on personnel refers to the characteristics of people who will perform the evaluation which includes qualification, training, expertise and open mindedness. The practice refers to strands on consistency of evaluation, areas that needs to be improved, and specific courses of actions. The relational cluster emerged where respondents indicated having a discussion and involvement will improve the evaluation practice. The study cluster is based on suggestions such as training, workshops and observations on how evaluation is done. This basically refers to learning the system of evaluation.

212


Table 10 Suggestions to Improve the Practices of Evaluation

Personnel (f = 18)

Practice (f = 41) Relational (f = 4)

Study (f = 8)

Evaluation must be done by a qualified personnel

Evaluators in schools must be trained how to do evaluation

Leaders should know if their performance is effective

People who evaluate must be experts

The administrators needs to be open minded

Skills, Training, Study

Attitude Number, Staffing

Always do assessment

Conduct needs assessment

do it on a regular basis

evaluation must be consistent

evaluation results should be well utilized

evaluation should be done with sincerity

evaluation should be implemented properly

evaluation should be planned highly

hire extra personnel for such function

lessen the load of guidance counselors to do evaluations

minimize school contests to concentrate on evaluation

needs conceptualization

Provide materials for improvement

should be objective rather than subjective

use data as basis for improvement

Group discussion involvement

Have seminars to make more teaching effective

observation seminar about

question technique seminar on Table

of Specifications seminars and

workshops

Note. f=frequency of responses

213


The respondents recommendations in order to improve the practice of evaluations in schools are categorized into three main themes, namely, personnel, practice and relational. The first category, i.e., personnel, is further sub-categorized into skills/training/study, attitude and number/staffing. On the other hand, the second category, i.e., practice, is further sub-categorized into (a) institutionalization, (b) instrumentation, (c) focus/purpose, (d) social marketing, (e) utilization, (f) dissemination, (g) methodology, (h) planning, and (i) meta-evaluation. Among the three main categories, it appears that the practice of evaluation in schools is the most problematic. It also appears that if program evaluation is institutionalized, technical areas such instrumentation will be improved, as well as the school communitys overall acceptance of this practice.

Discussion

Areas Evaluated

Previous studies point to five dimensions (or attributes or correlates),

namely, teacher quality, learning climate, school leadership, school identity, and curriculum and student outcomes. These attributes of effectiveness are the very expectations from schools. When schools conduct evaluation, do they give priority to these areas? The results indicate that they do it partially. With 75 out of 81 of these schools conducting evaluation of teacher performance and teacher training programs, it appears that they are mindful of their accountability in terms teacher quality. The same could be said with regard to school leadership, with 59 out of 75 schools evaluating the performance of directors, principals, coordinators and other personnel in leadership position. However, there is inadequate data to conclude that these schools actually assess their accountability in terms of the learning climate, school identity, and curriculum and student outcomes. Areas for Improvement

The schools sense of responsibility is still apparent as far as teacher

quality and school leadership are concerned when they prioritized teacher training programs and teacher/administrative performance among the areas for improvement. They even indicated the need to improve the evaluation of

214


support staff performance. It implies the respondents awareness of the complementary roles that teachers, administrators and staff play in having effective schools. It is also interesting to note the possible implication of the need to improve student organizations and their evaluation. It is an indication that schools would like to explore this area as part of their accountability in terms of curriculum and student outcomes. Evaluators

It is alarming to note that principals, coordinators and guidance

counselors primarily conduct evaluations in schools. School personnel are reported to have more than a handful of tasks to do. They prioritize people-oriented tasks such as listening, encouraging, inspiring, mentoring, coaching, enabling, and empowering the other members of the school community. The issue is having enough time to conduct real program evaluation, assuming that they are already engaged with people-oriented tasks. The respondents have shown a different notion of the term evaluation compared to the researchers intended meaning.

There is a need to expose the basic skills and expertise of school personnel who conduct program evaluation, as well as the consumers or users of the results for making data-grounded decisions. There is also a need for a more positive attitude toward evaluation, especially on the part of school administrators. The following recommendations imply the need to institutionalize program evaluation in schools like Evaluation task should be part of the program and of the schedule,Evaluation should be done at a regular basis to identify early problems about the program.., and The administrators must be evaluation-oriented such that a Research and Development/Evaluation Office must be established with qualified persons or experts. Institutionalization of program evaluation in schools therefore may be accomplished through the review/revision of the organizational structure, mainstreaming in the schedule and having a more positive orientation or mindset about this practice. It is recommended that existential issues such as the what and the why of program evaluation should be addressed too. After this has been done, the technical aspect may follow. Both quantitative and qualitative data support the need to enhance the way program evaluation is being conducted in schools, particularly evaluation planning,

215


instrumentation, methodology, report writing, utilization, dissemination and meta-evaluation. Data Gathering Methods Used

There is a need to intensify the use of qualitative data gathering

methods like observation, interview and focus group discussion in order to enrich the data being collected through the prevalent use of questionnaires. This may be particularly true for the schools in Visayas and Mindanao. There may be a need to review how schools in general use observation as a way to gather data, particularly all Luzon schools. Sources of Information Regarding Evaluation

A good project for school networks and consortia is the listing of

locally-published and foreign authored reading materials on educational program evaluation. Better yet, these organizations may develop and publish a practical handbook on the conduct of evaluation in the Philippine setting. Other prevalent modes of transferring evaluation know-how may be maximized such as seminars/workshops. Another area that may be explored is the improvement of the Educational Measurement and Evaluation subject in teacher training institutions in terms of breadth and depth of coverage this could have a greater impact and multiplier effect. There could also be a need to enhance the conceptual knowledge and technical skills of the professors handling this subject. When the conduct of program evaluation is institutionalized, maybe through the creation of a separate office manned by equipped personnel (or the schools internal expert in evaluation), it will be a clear manifestation of the schools seriousness in assuring their accountabilities for both its internal and external clients. Implications

As adult learners, the respondents prefer primarily tacit sources of

knowledge such as seminars/workshops, maybe because not everything can be captured in written sources of information about evaluation. There is a need to review of the measurement and evaluation subject in the

216


undergraduate course in education since they only emphasize on test construction and neglect the part on the concept and nature of evaluation. There is a growing need to teach the concept of evaluation in the undergrad

Evaluation centers may be strategically located maximize school organizations; tap graduates of the on educational evaluation.

Is evaluation growing in the Philippines? How well are we doing it? Measurement focuses on test construction. It does not go beyond what teaching and learning is all about. Universities need to invite more enrollees on their measurement and evaluation programs. There is a great call on institutionalization of program evaluation in schools.

Recommendations

1. Explore the institutionalization of accreditation among public schools

to make them more conscious of program evaluation. 2. Intensify information dissemination and training on evaluation

through school consortia and in-house trainings facilitated by external experts.

3. School organizations should consider seminars/workshops as a highly potential source of information, particularly on program evaluation. Well-planned and high-level interaction among the participants may be really helpful in developing internal experts.

4. External experts may conduct in-house training for school administrators, teachers and counselors to maximize the transfer of knowledge.

5. Consortia of schools may be a good venue for the sharing of knowledge and development of practical tools on evaluation in the Philippine context.

6. Provide funding for faculty development in school to provide school personnel engaged in evaluation to gain technical knowledge in the field.

7. Upgrade technical skills on evaluation of those currently handling evaluation activities, especially the guidance counselors and school administrators.

8. For school administrators, to tap school based experts to train potential faculty and staff evaluation and measurement. Resource sharing through consortium and inter-school collaboration.

217


References Astin, A. W. (1993). Assessment of excellence: The philosophy and practice of

assessment and evaluation in higher education. Arizona: The Oryx press.

Clarke, A., & Dawson, R. (1999). Evaluation Research: An Introduction to Principles, Methods and Practice. London: Sage.

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Evaluations basic purpose, uses, and conceptual distinctions. In Program evaluation alternative approaches and practical guidelines (3rd ed.) (p. 21). New York: Pearson Education, Inc.

Kaufman, R., & English, F. W. (1976). Needs assessment: A guide for educational managers. Arlington, VA: American Association of School Administrators.

Keeney, R., & Raiffa, H. (1976). Decisions with multiple objectives. Wiley, New York.

Lincoln, Y. S., & Guba, E. G. (1988). Do inquiry paradigms imply inquiry methodologies? In D.M. Fetterman (Ed.), Qualitative approaches to evaluation in education (pp. 89-115). NY: Praeger.

Mark, M. M., Henry, G. T. & Julnes, G. (1999). Toward an integrative framework for evaluation practice. American Journal of Evaluation, 20 (2), 177-198.

McKillip, J. (1998). Need analysis: process and techniques. In L. Bickman & D. J. Rog (eds.), Handbook of Applied Social Research Methods. Thousand Oakes, CA: Sage.

Nickens, J.M., Purga, A.J., & Noriega, P.P. (1980). Research methods for needs assessment. Washington, DC: University Press of America.

North Central Regional Educational Laboratory. (1998). Conducting a comprehensive needs assessment. [on-line] Available: http://www.ncrel.org/pd/needs.htm

Patton, M. (2000). Utilization-focused evaluation. In K. Lellaghan, D. Stufflebeam, & L. Wingate (Eds.), International Handbook of Educational Evaluation (pp 223-242). NY: Kluwer International Handbooks of Education.

218


Posavac, E. J., & Carey, R. G. (2003). The assessment of need. In Program evaluation methods and case studies (6th ed.). New Jersey: Prentice Hall.

Scriven, M. (1967). The methodology of evaluation. In R. W. Tyler, R. M. Gagne, & M. Scriven (Eds.), Perspectives of curriculum evaluation (pp. 39-83). Chicago, IL: Rand McNally.

Stake, R. E. (2000). Case Studies. In N. K. Denzin, & Y. Lincoln (Eds.), Handbook of Qualitative Research (pp. 134-164). Thousand Oaks, CA: Sage Publications, Inc.

Stufflebeam, D. L. (2001). The metaevaluation imperative. American Journal of Evaluation, 22(2), 183-209

Weiss, C. H. (1997). How can theory-based evaluation make greater headway?. Evaluation Review, 21(4), 501-524.

219


Dr. Carlo Magno is presently the Vice President for Research and Testing in the Asian Psychological Services and Assessment. He finished his PhD in Educational Psychology major in Measurement and Evaluation. He has over 80 published scholarly works in international, refereed and abstracted journals. His research interest would include self-regulation, metacognition, learner-centeredness, parental involvement, and language learning. He was recognized in 2011 by the National Academy of Science and Technology as the Outstanding Young Scientist and in the same year by the Global Science Academy as one of the renowned 200 Global Scientists in the world.

Ms. Marife Mamauag is presently a faculty and director of the Testing Center in Help University, Malaysia. She finished her academic requirements for PhD in Educational Psychology major in measurement and evaluation at De La Salle University, Manila. She also earned her MS in Educational Measurement and Evaluation from the same university. Her research interest includes test development, particularly on scaling non-cognitive measures and program evaluation.

Mr. Neil Parias presently the director of the Center for Learning and Performance Assessment of the De La Salle-College of Saint Benilde. He has finished his academic requirement for PhD in Educational Psychology major in Measurement and Evaluation at De La Salle University. As a test specialist, he was involved in the development and standardization of a number of aptitude tests. He used to be the Assistant Executive Director of the Catholic Educational Association of the Philippines.