Research Journal of Social Sciences - aensiweb.net 2015/1-12.pdf · Research Journal of Social Sciences ... Development and Validation of Observation Protocol for Gender Equity in

Copyright © 2014, American-Eurasian Network for Scientific Information publisher

Research Journal of Social Sciences

JOURNAL home page: http://www.aensiweb.com/RJSS 2015 March; 8(2): pages 1-12.

Published Online 24 November 2014. Research Article

Corresponding Author: Marie Paz E. Morales, Educational Policy Research and Development Center, Philippine Normal

University, 1000 Manila, Philippines.

E-mail: [email protected]

Development and Validation of Observation Protocol for Gender Equity in Classroom: Towards Achieving Gender Equality in Education 1Marie Paz E. Morales and 2Allen A. Espinosa 1Educational Policy Research and Development Center, Philippine Normal University, 1000 Manila, Philippines. 2Faculty of Science, Technology and Mathematics, Philippine Normal University, 1000 Manila, Philippines.

Received: 25 September 2014; Revised: 20 October 2014; Accepted: 1 November 2014; Available online: 24 November 2014

© 2014 AENSI PUBLISHER All rights reserved

ABSTRACT

The present study develops an observation protocol intended to evaluate gender equity inside the classroom. Literature about gender

and education and research instrument development were reviewed prior to the development of the observation protocol. The developed

instrument was then content and face validated by panel of experts and was pilot tested to in-service teachers. Kappa statistics and intra-

class coefficient was used to determine agreement among and within raters which yielded a value of 0.83 for Kappa coefficient and 0.70

and 0.93 for single and average intra-class coefficient respectively. Reliability coefficient Cronbach‟s alpha was computed from the instrument which yielded a value of 0.935. Finally, factor analysis was also utilized to determine the statistical constructs of the instrument.

Revision was done based from the result of the validation process. The final instrument yielded eight constructs anchored on the 21st

Century Learning Framework: (1) Instruction and assessment (learning actuators, expanding literacies, climate of assessment and transparency); (2) Classroom management and environment (changing habits and roles); (3) Teacher and student interaction (mentoring and

community); (4) Medium of instruction (mentoring community); (5) Teaching strategy (self-initiated transfer and thought and abstraction);

(6) Loco parentis (changing habits and roles); (7) Instructional material (climate and assessment; and (8) Verbal teacher response (changing habits and roles). Further validation is hereby recommended to standardize the developed observation protocol.

Key words: gender equity, gender equality, classroom observation protocol, research instrument development

INTRODUCTION

Education has always been valued and recognized

as a crucial instrument in improving the welfare of the

country and in alleviating poverty. Meinardus [10]

claimed that the human capital theory highlights the

significant function of the quality of the country‟s

education to its economic development. Accordingly, if

the citizens are better educated, the greater are the

chances of economic development. It has been singled

out that the modern world is casually termed as

“knowledge society” because education and

information production have become the factors

influential to international competitiveness.

With the aim to achieve an educated community,

Education for All(EFA) was conceived and launched at

the World Conference on Education for All in 1990 by

UNESCO, UNDP, UNFPA, UNICEF and the World

Bank. The EFA movement is a global commitment of

providing quality basic education for all children,

youth and adults. Mesa [11] confirmed that equal

access to education makes it possible for social and

economic mobility of the less fortunate.

One of the six Education for All goals is to

eliminate gender disparities in the primary and in the

secondary levels by 2005, and to achieve gender

equality in education by 2015. EFA focuses on

ensuring girls‟ full and equal access to and

2 Marie Paz E. Morales and Allen A. Espinosa, 2015 / Research Journal of Social Sciences 8(2), March, Pages: 1-12

achievement in quality basic education. This has been

pursued as the Millennium Development Goal of East

Asia and Pacific Regional UN Girls‟ Education

Initiative [20]. The major goal is to ensure the

availability of quality education for all girls and boys

across East Asia and the Pacific region. The first

attempt to achieve this goal is to reach gender parity or

equal proportions of girls and boys, in both the primary

and the secondary levels. The Philippines was

identified as one of those countries that indicated near

parity in 2005. In fact, in 2008, Manahan [9] quoted

Pres. Gloria Macapagal-Arroyo of saying “We are

proud of the gains in gender equality in the Philippines.

We are number one in the world in terms of gender

parity in literacy and in health.”In addition, the

Philippines has been identified to score relatively high

on global gender equality indices indicating an overall

improvement in recent decades in the situation of

women. Several researches [3-5,9] found the same

theme on gender parity in the Philippines.However, as

pointed out by Mesa [11], most studies on gender

disparities were focused on Philippines as a country

compared to other countries. The need to assess gender

disparity within the country is hereby deemed

necessary.

Even with a good stance in gender parity and

gender equality which is indicated by an overall

improvement, thePhilippines is still far from achieving

gender equality in education, a goal that goes beyond

gender parity and access to education (gender equality

toeducation)[20]. This also includes equality in the

quality of education received and in the teaching and

learning processes (gender equality in education) as

well as in the opportunities available through education

(gender equality through education). Apparently, in

classrooms, gender biases can be observed in textbooks

and learning materials, with women and men often

portrayed in stereotypical roles (e.g., girls looking after

younger siblings, women doing errands, boys playing

football, men as policemen, doctors or leaders, etc.). It

is also observed in the practices of teachers and how

they interact with students - girls/women and boys/men

still often given tasks which are considered typically

„feminine‟ or „masculine‟ academic and training fields

[20].

As recommended by UNESCO [20], while gender

parity indices are calculated and used to monitor

progress, the more qualitative aspects of achieving

gender equality should also be looked into. Indicators,

data and information are needed to assess the extent to

which gender equality is being achieved in terms of

processes and outcomes in education. This will then

lead to understanding of gender gaps that will lead to

actions addressingsuch gaps. Thus, the objective of the

present study is to developand validate agender equity

in classroom observation protocol. This instrument is

intended to assess the gender equality in education

quality specifically in the aspects of instructional

language, classroom materials and teaching aids,

textbooks, curriculum, activities and interactions, and

classroom management. Specifically, the study aims to:

identify pre-deterministic constructs, criteria and

indicators of gender equity in classroom observation

protocol based from literature reviews, and

expert‟s/teacher‟s views;

develop and validate gender equity in classroom

observation protocol based on the pre-deterministic

construct, criteria and indicators of gender sensitive

education; and

test the inter-rater and intra-rater reliability of the

gender equity in classroom – observation protocol.

Literature and Background:

Terminologies on Gender Studies:

In gender studies, gender equality refers to the

principle of assessing quality of men and women and

their right to enjoy equal conditions realizing their full

human potentials to contribute to and benefit from the

results of development, and with the state recognizing

that all human beings are free and equal in dignity and

rights Gender equity is the fairness and justice in the

distribution of benefits and responsibilities between

men and women. It recognizes that men and women

have different needs and power that these should be

identified and addressed in a manner that rectifies the

imbalance between them. Gender sensitivity connotes

the ability to understand and consider the socio-cultural

factors underlying gender-based discrimination,

socialization of men and women into certain behaviors

or opportunities, power relations between men and

women, as well as the different needs, problems and

levels of access to resources that they have. Finally,

gender discrimination refers to any distribution,

exclusion or restrictions made on the basis of socially

constructed gender roles and norms that prevent a

person, female or male, from enjoying full human

rights.

Gender and Education:

UNESCO‟s [20] education goals include among

others ensuring that by 2015 all children, particularly

girls, children in difficult circumstances and those

belonging to ethnic minorities, have access to, and

complete, free and compulsory primary education of

good quality and eliminating gender disparities in

primary and secondary education by 2005, especially

achieving gender equality in education by 2015, with a

focus on ensuring girls‟ full and equal access to and

achievement in basic education of good quality.

One of the Millennium Development Goal (MDG)

to promote gender equality and empower women was

launched in May 2002 by UNGEI aimed to eliminate


gender disparity in primary and secondary education

preferably by 2005 and at all levels by 2015 [20]. This

MDG program sought to establish networks and

partnerships among experts and organization

promoting equality in education. East Asia and Pacific

Regional UNGEI‟s major goal was to ensure

availability of quality education for all girls and boys

across region. As a monitoring scheme, UNESCO [20]

used Gender Parity indices to indicate equal or near-

equal proportions for girls and boys in the different

levels of education. Accordingly, of the 19 countries

with available data for 2005, only 16 had reached the

target of gender parity in primary education while at

the secondary level, only 9 countries achieved parity by

2005. Philippines was included in the list of 16

countries that reached gender parity in primary

education but not in secondary and tertiary levels.

EFA‟s Global Monitoring Report showed that

based on the past trends, 1991-2005, Philippines

included among the list of countries which are at risk

of not being able to achieve gender parity in all levels

by 2015 or 2025. This was supported by Social Watch

that although the Philippines remains among the

countries with a relatively high gender equity index,

scoring alongside Australia and Latvia, its pace of

development in gender equity over the past years is

dismal prodding Social Watch International monitors

to rate Philippines‟ GEI progress as stagnant.

Accordingly, its pace of development is almost at the

same rate as countries with the lowest scores in the

GEI of 2007

Dator-Bercilla [6] claimed that the Philippines

may be relentlessly attempting to maintain gender

equality and equity, yet efforts may not be enough to

facilitate transformative process that will produce

equality and equity between women and men in the

Philippines. In support, magna carta of Women (RA

9710) institutes that all government agencies shall

develop and make available information, education and

communication materials on their specific programs,

services and funding outlays on women empowerment

and gender equality. To this effect, education sectors

specifically the Department of Education (DepEd),

Commission on Higher Education (CHED), and

Technical Education and Skills Development (TESDA)

shall among others develop and promote gender-

sensitive curriculum and develop gender-fair

instructional materials. In the same line, Gender

Equality Guide provided a checklist for gender equality

in the education system which includes:

review of the formulation of courses for possible

gender-bias;

analysis of gender biases in the theories and

perspectives discussed in the course content;

analysis of gender biases in assessments;

review of activities for possible gender biases; and

review of reading materials, instructional and

curriculum materials and textbooks; and review of

class policies for possible gender biases

Supporting the checklist provided are the ideas of

Herz and Sperking [8] who provided a simple formula

for improving gender equality in education: making

girls‟ education affordable, making education a

practical reality, making schools more girl-friendly and

improving education quality. Further, UNESCO

developed a Gender Equality Framework that describes

four dimensions of gender quality in education –

equality of access, equality in the learning process,

equality of educational outcomes and equality of

external results – reinforcing the need to bring gender

equality to, within, and through education [7]. UNGEI

presented a three-section program to overcome gender

disparities. Area one is focused on enhancing access

and participation of marginalized groups which

included recommendations such as:

building schools at a safe walking distance;

placing community at the center if action;

expanding outreach through alternate modes of

schooling; and expanding opportunities for secondary

schooling

Area two highlights improving school equality

with a gender perspective. This includes inequitable

practices within schools such as teacher attention,

expectations and valuing, differential treatment within

the classroom; and often unequal access to school

spaces, resource and facilities. This also includes

creating child-friendly/girl friendly school

environments and overcoming systematics biases in the

teaching-learning process. Finally, area three focuses

on improving learning outcomes which highlight

consideration of boys‟ learning needs, confronting

child labor and ensuring the school-to-work transition

work for girls.

Instrument Development:

In reviewing gender situation across the world,

EFA‟s Global Monitoring Report, concluded that

“gender parity goal has been missed and gender

equality remains elusive”. Admittedly, there is a

considerable concern that the Gender Parity Index

(GPI) can be misleading for assessing true gender

equality in education. Because of this, several new

measurements have been included such as Gender

Equality Index (GEI) developed by UNESCO [20],

Education Development Index (EDI), Gender

Development Index (GDI) and Gender Empowerment

Measure (GEM). As claimed by UNGEI [20], these

measures specifically GPIs in Asia – Pacific do not

adequately reflect local realities and that they easily

mask disparities at subnational levels. Despite debates

and some observable progress in quantitative terms

neither the goal of gender parity nor that of gender


equality has made sufficient progress. Thus, comes in

the qualitative and other nature of measures to extract

better observations in terms of gender equality specific

to education quality. Tracy and Lane [19] in their study

on gender-equity teaching behaviors, used the Gender

Equity Observation Form(GEO) to assess a 15-minute

segment of a teacher‟s use of his or her gender-

equitable teaching behaviors (GETBs) by recording

tally marks each time when he/she observes the

following techniques: acceptance, praise, use of higher-

level questions, positive and negative criticism, wait

time and proximity. Although, GEO form can describe

gender biases in the classroom and contribute in

achieving gender parity and gender equality in the

aspect of quality education, the instrument is a self-

rating instrument which highlights frequency. Thus,

there is a perceived need for another instrument for

classroom observation to better extract more and

detailed descriptions of gender-equitable teacher

behaviour and gender-biases in the classroom.

Developing a new instrument entails a lot of

quantitative analysis. But qualitative approaches are

also needed to probe further, clarify and establish

verified quantitative results. These processes

(quantitative design and qualitative approach) are

highlighted in developing Views on Nature of Science

and Education (VOSE) and Colorado Learning

Attitude Science Survey (CLASS). Adams et al., [1]

used literature reviews, existing survey instruments and

pilot study which included student interview and face

& content validation by experts focused on qualitative

approaches to extract probable contents and pre-

deterministic constructs of the intended survey

instrument. Comparably, Chen used quantitative design

with qualitative approaches in the development of

Views on Nature of Science and Education (VOSE).

Chen claimed that the results were dependable because

the items originated from the respondents‟ point of

view instead of from the researcher‟s presumption of

reasonable answers, thus has a high reliability.

Methodology:

Mixed method was utilized in the study.

Quantitative research design combined with qualitative

approaches was used in developing a classroom

observation protocol intended to gender and culture

disparity and equality in education quality. Survey

research was used to determine the feasibility of the

instrument in the area of evaluation gender biases,

disparities and gaps in education quality. The study

consisted of three major stages: Preparation and pilot

study; item design and construction; and validation and

reliability determination.

A. Participants of the Study:

In all the three stages, convenient sampling was

done to identify the appropriate participants for each of

the stages identified. In the preparation and pilot study,

the chosen participants were 45 graduate students of

the Philippine Normal University - College of Graduate

Studies and Teacher Education Research taking Master

of Arts in the different disciplines. These participants

are currently teaching in basic education. Five experts

were identified to validate and re-validate the

developed instruments in the design and construction

stage. About 145 basic education teachers were

identified for the third stage of the study.

Table 1: Participants of the Study.

Stages of the Study Participants/Sample Sampling Process

Preparation and Pilot Study 45 graduate students Convenient sampling

Item Design and Construction 5 Experts Convenient sampling

Pilot Testing and Data Analysis 145 basic education teachers Convenient sampling

B. Summary: Table 2: Summary of the Methodology.

Stages of the Study Data Collection Data Analysis

Preparation and Pilot

Study Interviews, literature search and review, administration

of open response survey

Transcriptions

Item Design and

Construction Focus group discussions and interviews

Checklist for content validity of the instrument

Average ratings and Aiken‟s content

validity coefficient

Inter-rater Kappa coefficient

Intra-class coefficient

Pilot Testing and Data

Analysis Evaluation checklist and survey Cronbach alpha coefficient

Factor Analysis

C. Stage 1: Preparation and Pilot Study:

Using literature reviews focused on gender equity

and gender equality in education quality, pre-

deterministic constructs of the intended tool were

identified. The instrument was highly influenced by the

different strategies provided by to the classroom a

more equitable place. Major construct he identified

included: teacher/student interactions, instructional

styles and analysis of instructional materials. The pilot

study open-response survey to selected graduate


students of the Philippine Normal University. They

were asked several questions on the constructs

presented by Frawley (2205). Transcriptions and

summaries were used data deduced from the pilot study

and the information derived from literature and was

used as bases of the content and format of the

evaluation tool.

D. Stage 2: Item Design and Construction:

Initial design and format of items resulted to

version 1 or the draft version of the evaluation tool.

This instrument was subjected to content and face

validation by five experts. Faculty of the Philippine

Normal University formed the pool of experts. Panel

interview with the experts who validated the instrument

was conducted to clarify the content and design of the

items of draft version. Their suggestions were

incorporated in the revision of the draft version. A

second validation was done by the same experts. Three

of these experts were the same experts who validated

draft version. The validation was conducted to

determine if all the comments and suggestions were

properly implemented.

E. Stage 3: Pilot Testing and Data Analysis:

Analysis of data gathered from the validation

procedure included computation of evaluation averages

of the experts and content validity coefficient.

Agreement among raters and within raters was also

determined using Kappa statistics and intra-class

coefficient. Factor analysis was also used to determine

the statistical constructs. Reliability of the instrument

was based on the empirical process of the survey

design as well as statistical tests of all the items and per

category. This scheme of qualitatively establishing the

reliability was adopted from the study of Chen. It was

part of the procedure of the research that the reliability

was qualitatively established by conducting interviews

with the would-be end-users of the instrument. In this

case, the process ensured that there would be internal

consistencies of the items in the instrument which were

based from the possible arguments of the end users.

The instrument was used by 145 basic education

teachers teaching in government schools in the

National Capital Region to quantitatively measure the

reliability. From this, Cronbach‟s alpha which

represents the reliability measure of the instrument was

determined.

Results and Discussion

Version 1- Gender Equity Classroom Observation

Protocol:

The draft version (version 1) of the observation

protocol was developed using literature reviews and

empirical data provided by the pilot study. Forty

graduate students surveyed expressed their idea and

concepts of a classroom setting free of gender and

culture gaps and biases. They specifically identified

steps which they would undertake to ensure that there

are no gender stereotyping in their classroom in the

following areas as presented in the summarized

accounts below:

Classroom Environment:

Proper gender roles are explained to students

Teasing about gender issues is not encouraged

I give equal opportunity to them. I created

activities that will help them to communicate to each

other.

Everybody can be a volunteer in any task.

Even during seating arrangement, it should be in

random not in terms of gender.

We try to eliminate gender stereotyping by treating

all individuals as if they belong to one group.

Students are arranged in a Male-Female-Male-

Female manner. Genders are arranged alternately

Everybody is given a chance to be assigned in

work or position. If there are stereotyping , I clarify it

to students.

Make sure to recognize both genders in terms of

achievements etc.

Seating arrangements are done alphabetical,

alternating boys and girls.

Equal treatment to the male and female students in

terms of giving rewards, punishments etc.

I ensure that boys and girls have the same

privileges inside the classroom.

I assure that students are not bullying, maintain

peace and order in classroom.

The seating arrangement is not arranged based on

gender but either alphabetical or based on student

preference.

Choosing the right words

The activities provided for the students is suitable

for girls and boys or female and male students. There

are equal opportunities provided for them.

I make sure that the seating arrangement is mostly

alternate boys and girls.

They are seated randomly making sure that boys

can are evenly distributed since they are lesser

Balance the context by citing other examples other

than those found in the book.

I make sure that the environment has no gender

stereotyping by means of designing the classroom in

such a way that it doesn’t look girlish or boyish,

positing some quotations about equality among

genders (and even races and ethnicity).

Student-Teacher Interaction:


Giving my students an equal attention and respect

their opinions

Discussion using people as examples does not

focus in one gender.

Equal chances to both genders (as to who) during

discussions, or group activity.

All inquiries are entertained by the teacher.

By not being biased with the student of opposite

gender. By trying to get along with them and

understanding their nature.

Explain that it is not the intention. Then cite other

examples that includes/apply to both gender.

Providing activities that are best suited for both

girls and boys.

I try to make sure that they both understand. Just

like the topics on reproductive system, although it

concerns either male or female parts, care and others,

I try to explain that they must know especially care

concerns.

Everyone is given equal treatment.

Being fair to each of the students/gender.

Both gender is given equal opportunity to

participate in the class.

Students are open and they are welcome to ask

and interact with the teacher.

I ensure that boys and girls can express their ideas

and feelings.

No favouritism

Positive reinforcements are given equally for both

genders.

No biases/favouritism

Avoid talking about topics that would induce

stereotyping.

Assessment:

Fair assessment is given to every student

regardless of their gender.

In terms of performance assessment, the task to be

performed is generic in nature.

Uniform assessments are given to all members of

the class.

Editing of test questions especially about

situational analysis.

Assessment strategies are applicable for both

genders.

Same level of questions in the assessment, same

grading system

I give evaluation to both male and female students

with the same evaluative materials.

They are graded fairly and objectively.

Activities and exams are given to both genders.

Equality of the type of tests. Being sensitive with

the questions/items/sentences.

Assessment tools are designed for both male and

female students

Questions are for both genders

Equal chance in oral and written outputs. Well

distributed genders in groupings

By constructing test questions/performance-based

test that suits the nature of male and female.

These graduate students-participants also

unambiguously defined how they would ensure that the

instructional materials that they are using such as

journals, textbooks, multi-media materials are gender

sensitive. Below were their major views and ideas:

By reviewing it and modifying it to become gender

sensitive.

Peer review and books are also

reviewed/evaluated before bring recommended as

textbook.

Most of the science lessons I teach are not gender

sensitive (Physics/Chemistry).

Instructional materials should be inspected first

before using them in the classroom.

Both parties are involved and there are no

stereotyping.

Instructional materials are reviewed first not just

by the faculty but also the management before they are

adopted.

We examine the instructional materials that we use

as to gender sensitivity.

Standard-based books, journals, textbooks and

multimedia.

I study and read first to ensure gender sensitivity

Pre-evaluation of books and pre-reading of

journals.

It is evaluated by AC, SAC and Teachers

We conduct peer evaluation in using the

instructional materials.

I make sure that it does not contain words/sample

situations that prejudice either male or female learner

The language used is evaluated.

I choose carefully all the instructional materials

used in the discussion

I review them ahead of time. In instances there are

gender biases, I opt to look for alternative.

They further mention about the advantages of a

gender sensitive classroom and instruction as: gender

sensitive classroom and instruction will give an equal

opportunity to all students regardless of their gender, it

provides a healthy psychological environment for the

learners, no gender stereotyping, issues and concerns

that will post changes against a particular gender are

eliminated, everyone is treated equally, students won‟t

be able to feel discriminated, everyone is given

confidence and peaceful, friendly learning

environment, encourages peace/equity, encourages


student to study and excel (healthy competition), and

students can develop self-confidence and self-respect.

In addition to these, they described that gender

sensitive classroom will lead to students gaining more

confidence in the class, promotes a more peaceful

environment, would address bias in terms of

assessment and evaluation, would create a harmless

relationship, students would have equal opportunities

to show their capabilities and to strengthen their

potentials, in Biology it is better to consider issues like

reproductive concepts for boys and girls, more

encouraging and conducive settings for learners,

students would be free in expressing themselves since

there are no biases in the classroom, and students

realize their potential based in ability not gender.

Furthermore, the survey study provided the

participants‟ insights on how the government should

help in achieving gender sensitivity and gender equity

in the classroom as presented below:

By conducting seminars to schools and other

institutions.

By sharing ads/infomercial about gender

sensitivity.

Review the instructional materials

Educate the science teachers about the issue.

Include the discussion in the curriculum

Parallel textbooks, activities and separate lessons

for relevant topics in Biology.

Make sure that the instructional materials

provided are suitable for all genders.

Equal opportunities should be given to both

gender although of course there are some provisions

limited to specific gender that the government should

address.

Enforce greatly gender sensitivity and gender

equity in the classroom.

Provide activities which are applicable to both

genders.

There should be seminars for the teachers on how

to develop a gender sensitive classroom and on how to

maintain gender equity.

Empower women in the society.

Recognize that women can have decent work too

and can be influential in the community.

More information.

Conduct gender sensitivity seminars to teachers to

be implemented by the participating teachers at this

school.

Include this in the curriculum.

Books and other learning materials must cater

equal interest of male and female. There should be no

books that give examples with male only as character

or female only which give the stigma that a particular

book is or a specific gender only

Educating people about the issue

Give seminars to students and teachers about the

effect of gender sensitivity in every individual.

Strengthening the value of united family.

From the above desrciptions provided by the

study,draft version was designed and developed and

was subjected to descriptive and quantitative content

validation by identified experts. Only descriptive

validation was done for face validation. Descriptive

validation highlighted the use of phrases or words to

describe the assessment of the items. These were

presented as comments, remarks or suggestions of the

experts. Quantitative content validation made use of

the 15-item validation checklist. It is a 5-point Likert

scale checklist consisting of items that characterize a

good survey checklist related to the use of culture in

teaching. A summary of the averages per expert ratings

in validating the version 1 is presented in Table 1.

Table 1: Content and Face Validity of Gender Equity Classroom Observation Protocol.

Experts Average Comments/Suggestions

Expert 1 5.00

Expert 2 4.73

Expert 3 4.00

Expert 4 4.70 Consider revising some items for consistency and coherence. Change some headings also to make them consistent with the others.

How do we measure this po? Whether the teacher uses more male or female examples?

How will teacher role modelling measure gender equity? Seems to be very general. What do you mean with gender-role content?

The last statement in section F is a repetition of #2.

Expert 5 3.90 Change boys to males and girls to females

Typo errors and punctuations

Too many subheadings and categories Change font

Average 4.45

On the average, the evaluators had rated the

developed instrument 4.45 out of 5.0 (Table) indicative

that the ratersassessed the instrument within the higher

range of the Likert scale. This suggested a fair quality


tool in construction and valid content wise. The third

column provided the suggestion and comments of the

experts. Experts 3, 4, and 5 provided comments and

suggestions for the improvement of the instrument. To

clarify some of the experts‟ comments, panel interview

with themwas done. The panel interview was part of

the non-conventional process to ensure the empirical

reliability of the ratings and validation processes of the

experts.

Revised version of Gender Equity in Classroom:

Observation Protocol:

Ensuring that proper revisions were done as per

experts‟ comments and suggestions, the revised version

(v.2) was subjected to a second round of content and

face validation. A better rating with an over-all mean

of 4.98 out of 5.00 by the five ratersimprovingthe

initial rating of the instrument from the first validation

cycle. Each of the raters evaluated the instrument as

very close to 5.0 (Table 2).

In addition to mean values of experts‟ ratings,

content validity coefficient was calculated per checklist

item to ensure that the instrument was actually rated as

a content valid instrument. These coefficients were

shown in Table 3.

Table 2: Content and Face Validity of the Revised Version.

Experts Average Comments/Suggestions

Expert 1 5.0 All revisions suggested were integrated. Expert 2 4.7

Expert 3 5.0

Expert 4 4.9

Expert 5 5.0

Average 4.98

Table 3: Content Validity Coefficient (VIK) the Draft and Revised Versions.

Items Aiken’sVIK (Content Validity Coefficient) of the Draft Version

Aiken’sVIK (Content Validity Coefficient) of the Revised Version

Average 0.90 0.98

The closer the Aiken‟s validity coefficient to one,

the higher content validity an item had [2]. The experts

who rated the items found them valid in terms of

content as shown in the values of content validity

coefficients (VIK≈1.0). The items in the evaluation tool

are content valid in terms of in the aspect of gender

equity in education quality. For the second round, the

computed content validity coefficient was much closer

to 1.0 (VIK = 0.98) suggesting a better content validity

than the draft version. Experts‟ high rating on the

content validity of the instrument positively suggests

that they view the revised version of the instrument as

conforming to the concepts and principles of gender

equity in education quality. They believe that the

instrument as a whole will be able to perform its

intended task of determining gender gaps and equity in

all the aspects of teaching and learning processes done

inside the classroom.

Reliability:

An indicator of an instrument consistency is

known as reliability. It can indicate the stability of the

data sets across applications or across time. Different

types of reliability are used in different situations such

as test-retest and split half; however, internal

consistency reliability is most appropriate for a test

administered once. It indicates the extent to which the

examinees‟ observed scores are similar to their true

scores.

Table 4: Reliability Measure of Gender Equity Classroom Observation Protocol.

Instrument No. Of Cases (n) Cronbach‟s Alpha (α)

Gender Equity Classroom Observation Protocol 145 0.935

After the pilot testing, the scores were then

subjected to a statistical treatment to determine the

reliability coefficient Cronbach Alpha of the

instrument. The reliability coefficient was calculated to

be .935. This reliability coefficient is acceptable

because according to Fraenkel and Wallen and Hinkle,

the acceptable values of alpha ranges from 0.70 to

1.00. Further analysis of the Cronbach alpha was made

if the item is deleted to ensure precision and agreement

among items. Table 4 shows that items in the

developed instrument were indeed valid and reliable

even if a certain item is removed.

Inter-rater agreement was established to be able to

make sure that experts‟ evaluation or validation are

consistent. The inter-rater coefficient for the final run

of validation was provided in Table 5.


Table 5: Inter-Rater Coefficient of the Revised Version.

Inter –Rater Coefficient

E1-E2 E1-E3 E1-E4 E1-E5 E2-E3 E2-E4 E2-E5 E3-E4 E3-E5 E4-E5

Kappa 0.83 0.83 0.83 0.83 0.83 0.83 0.83 0.83 0.83 0.83

As shown in Table 5, all experts agree that the

instrument they were validating and evaluating was

within the standard excellent category as also presented

in the mean values of their ratings (Tables 1 and 2)

and in the Aiken‟s validity coefficients for the two

versions of the instrument (Table 3). Interpretations of

the Kappa coefficients were based on the standards set

by Landis. Meanwhile, the Intra-class coefficient

which is a descriptive statistics that provides the

composite of intra-observer also referred to as intra-

observer variability which is the deviation of a

particular rater‟s score as presented in Table 6.

Table 6: Intra-Class Coefficient Revised Version.

Kind of Measure Intra-Class Correlation p-value

Single 0.70 0.001*

Average 0.98 0.001*

* significant at 0.05

From Table 6, the index of variability for one

single rating is 0.70 classified as very strong

agreement. While the index for the reliability or

agreement of different raters averaged together is 0.98,

close to +1 (perfect) agreement. In both cases (single

and average), difference of measures of scores is

significant (p < 0.05) which means that there were

variable scores but these scores are still in agreement

with each other both within the same rater or among

raters. It can be deduced that intra-rater agreement is

high that supports the validity and reliability of the

instrument.

Further analysis of the reliability of the instrument

was done using factor analysis. Table 7 indicates eight

identified factors with the factors‟ respective items.

Cumulatively, the 10 factors have already explained

86.325% of the total variance and only about 13.675%

is attributed to other variables. After rotation, the

variance leveled off to the 10 factors, however, only

eight factors were attributed to high variance of the

corresponding items. The highest variance is attributed

to factor 1.

Table 7: Factors with the corresponding Items per Construct.

Factors Items

1 B5, B6, B8, B9, B14, C8, C11, C14, C16, C21, C22, C23, C24, C25, C26, C27, C28, D1, D2, D3, D4

2 B7, B10, B11, B12, C13, C15, C16, C17, C18, C19, C20

3 C2, C3, C4, C5, C6, C7

4 B3, C10

5 B1,

6 B4, B5, B14

7 B13

8 C1

Analysis of the items included in each of the

factors resulted to common theme for the items in each

of the factors. Factor 1 with the highest variance

thematically leads to student belief on the use of the

vernacular and culture in the teaching and learning

process. In Factor 2, the items pertain to student views

and beliefs on the integration of culture and language

in the teaching and learning process. Factor 3

suggested a theme related to learning science and

learning literacy; factor 4 on teacher‟s role; and factor

5 on constructivism. Table 8 provided the summary of

the factors inclusive of the sample items identified in

these factors.

Table 8: Sample items in the constructs of Gender Equity Classroom Observation Protocol.

Factor 1: Instruction and Assessment(Learning Actuators, Expanding Literacies, Climate of Assessment and Transparency)

B5: Expects equal academic achievement between males and females

B6: Invites both male and female visitors with non-traditional occupations into the classroom.

Factor 2: Classroom Management/Environment(Changing Habits and Changing Roles)

C13: Practices gender-neutral reading of, and writing on, students' work

C15: Doesn‟t ignore sexist remarks made by the students, but challenges them to be gender sensitive instead

Factor 3: Teacher-Student Interaction(Mentoring and Community)

C2: Provides more consideration, acclamation, and constructive feedback to males than with females

C3: Calls male students by name and asks them more often with complex and abstract questions than female students


Factor 4: Medium of Instruction(Mentoring and Community)

B3:Balances questions between males and females during class discussions and observes wait-time

C10: Stereotypes in the language being used.

Factor 5: Teaching Strategy(Self-Initiated Transfer and Thought & Abstraction)

B1: Encourages cooperative learning in cross-gender groupings by mixing the seating arrangement among males and females and by

avoiding dividing students into a single-gender activity groups.

Factor 6: In Loco Parentis(Changing Habits and Changing Roles)

B4:Gives equal help and in-depth guidance to females as well as with males


Factor 7: Instructional Material(Climate of Assessment)

B13:Ensures that books, computer programs, and other curriculum materials are free from stereotyped gender-role behavior

Factor 8: Verbal Teacher Response(Changing Habits and Changing Roles)

C1: Uses "effort-appreciation" statements more often with male than female students.

The identified factors corresponded to the different

constructs initially identified as elements of teaching in

the survey study prior to the design of the instrument.

Moreover, these factors are found to be counterparts of

the nine major construct of the “Inside-Out Leaning

Model” designed for the 21st century framework by

TechThought which are learning actuators, changing

habits, transparency, self-initiated transfer, mentoring

& community, changing roles, climate of assessment,

thought and abstraction and expanding literacies. The

items in each of the constructs matched with the

characteristics and descriptions of the nine major

components of Inside-Out Learning Model for the 21st

Century Framework. From Table 8, factor 1 intially

identified as Instructiona and assessment construct

matched with the characteristic features of learning

actuators, expanding literacies, climate of assessment

and transparency; factor 2 intially themed as classroom

management and environment matched with changing

habits and changing roles; factor 3 grouped as teacher-

student interaction and factor 4 themed as medium of

instruction corresponded with Mentoring and

Community; factor 5 termed as teaching strategy

coordinated with the characteristics of Self-initiated

Transfer and Though & Abstraction; factor 6 as In

Loco Parentis and factor 8 as verbal teacher response

matched with Changing Habits and Changing Roles;

and factor 7 as Instructional Material as climate of

Assessment.

With the matched constructs and items under each

construct, use of this material as classroom observation

protocol to detect gender biases and promote gender

parity and equality may have a high chance of success.

The tool may also serve as guide for teachers who

would rather promote gender equality in their classes

for better education quality.

Table 9: Summary of statistical characteristics of the Gender Equity Classroom Observation Protocol.

Version 1 Version 2

*n = 47

Content Validity

o Over All Mean = 4.45 out of

5.00

o Aiken‟s Content Validity

Coefficient ( VIk)= 0.90

*n = 47

Content Validity o Over All Mean = 4.98 out of 5.00

o Aiken‟s Content Validity Coefficient ( VIk) = 0.98

Cronbach‟s alpha (α) = 0.935

Inter-rater reliability: 0.83

Average Intra-Class Correlation: 0.98

8 Factors extracted from factor analysis: o Factor 1: Instruction and Assessment (Learning Actuators, Expanding Literacies, Climate

of Assessment and Transparency)

o Factor 2:Classroom Management/Environment (Changing Habits and Changing Roles) o Factor 3: Teacher-Student Interaction (Mentoring and Community)

o Factor 4: Medium of Instruction (Mentoring and Community)

o Factor 5: Teaching Strategy (Self-Initiated Transfer and Thought & Abstraction)

o Factor 6: In Loco Parentis (Changing Habits and Changing Roles)

o Factor 7: Instructional Material (Climate of Assessment)

o Factor 8: Verbal Teacher Response (Changing Habits and Changing Roles)

The designed instrument included two versions.

The second version was an improved edition of the first

one with better average ratings of the expert and

Aiken‟s coefficient of validity, acceptable internal

consistency reliability, and extracted factors.Inter-rater

and intra-class correlation are high which notes that

evaluators of the instrument do have common

perception of how the instrument would fair in

determining the gender status in a classroom. It

included features such as Likert-scale format for easier

analysis, items are grouped according to constructs

which were statistically identified, the statistically

identified constructs match the pre-determined

constructs, qualities of a good and acceptable survey

instrument are exemplified by the developed


instrument as evaluated by experts, and items in the

constructs were empirically determined.

Conclusions and Recommendations:

The study developed an observation protocol for

gender equity in a classroom. The research instrument

was developed based on predeterministic constructs,

criteria and indicators of gender sensitive education.

Content and face validation by panel of experts was

conducted to polish the research instrument.

Afterwards, pilot testingof the instrument to in-service

teachers was done to gather quantitative and qualitative

data. The data collected was then subjected to Kappa

statistics and intra-class coefficient to determine

agreement among and within raters which. This yielded

a value of 0.83 for Kappa and 0.70 and 0.93 for single

and average intra-class coefficient respectively .

Reliability coefficient Cronbach‟s alpha was also

computed from the instrument. This yielded a value of

0.935. The computed reliability coefficients prove that

the instrument is valid and reliable. Finally, to further

analyze the instrument, factor analysis was also utilized

to determine the statistical constructs of the instrument.

Results of the validation process helps in the

finalization of the research intrument. The final

instrument yielded eight constructs anchored on the

21st Century Learning Framework: (1) Instruction and

assessment (learning actuators, expanding literacies,

climate of assessment and transparency); (2)

Classroom management and environment (changing

habits and roles); (3) Teacher and student interaction

(mentoring and community); (4) Medium of instruction

(mentoring community); (5) Teaching strategy (self-

initiated transfer and thought and abstraction); (6) Loco

parentis (changing habits and roles); (7) Instructional

material (climate and assessment; and (8) Verbal

teacher response (changing habits and roles).

Further validation is hereby recommended to

standardize the developed observation protocol.It is

also suggested that the instrument be localized to and

test the validity of the developed instrument other

nationalities.

References

1. Adams, W.K., 2006. A new instrument for

measuring student beliefs about physics and

learning physics: the Colorado Learning Attitudes

about Science Survey. Retrieved June 6, 2013

fromwww.phet.colorado.edu/.../New%20Instrume

nt%20For%20Measuring%20Student%2Beliefs.pd

f.

2. Aiken, L.R., 1985. Three coefficients for

analyzing the reliability and validity of

ratings.Educational and Psychological, 45: 131-

142.

3. Balisacan, A. and S. Piza, 2003. 'Nature and

causes of income inequality in the Philippines',

Background Paper for the EADN Income

Distribution Project.

4. Balisacan, A. and N. Fuwa, 2004. “Changes in

spatial income inequality in the Philippines.”

Research Paper No. 2004/34.World Institute for

Development Economics Research.

5. Cruz, L., 2007. Philippine gender gap index status

of Philippine women in Science. Retrieved July 5,

2012 from http://www.asiainnova.es/pdf/cruz.pdf.

6. Dator-Bercilla, J., 2009. Facing the continuing

challenge of gender inequality and inequity.

Retrieved last July 20, 2013 from

http://www.socialwatch.org/sites/default/files/pdf/

en/09_missingtargets.pdf.

7. Herz, B., 2006. Educating girls in South Asia:

Promising approaches, UNGEI series UNICEF

ROSA. Retrieved July 15, 2013 from

http://www.unicef.org/rosa/education_2483.htm.

8. Herz, B. and G. Sperking, 2004. What works in

girls‟ education: Evidence and policies from the

developing world, Council on Foreign Relation.

Retrieved July 15, 2013 from http://www.cfr.org.

9. Manahan, M., 2010. Focus on women and men in

the Philippines ---Closing the gender gap.

Retrieved July 5, 2014 from

http://www.focusweb.org/philippines/fob-

articles/socio-economic-monitor/391-focus-on-

women-and-men-in-the-Philillines-closing-the-

gender-gap.

10. Meinardus, R., 2003. The crisis of public

education in the Philippines Retrieved July 17,

2013 from

http://www.fnf.org.ph/liberalopinion/crisis-public-

education-philippines.htm.

11. Mesa, E., 2007. Measuring education inequality in

the Philippines.Philippine Review of Economics,

44(2): 33-70. Retrieved July 12, 2013 from

http://www.pre.econ.upd.ph/index.php/pre/article/

view/227/0.

12. _______, 2007. Social Watch Report. Retrieved

July 15, 2013 from

http://www.socialwatch.org/node/9292.

13. _______, 2007. UNESCO. GENIA ToolKit for

Promoting Gender Equality in Education.


http://www.unescobkk.org/education/gender/resou

rces/genia-toolkit/.

14. _______, 2010. United Nations Millennium Goal

3: Promote Gender Equity and Empower Women.


http://www.un.org/millenniumgoals/gender.shtml.

15. _______, 2009. East Asia Pacific Regional

UNGEI Retrieved July 12, 2013 from

http://www.ungei.org.


16. _______, 2010. East Asia Pacific Regional

UNGEI Retrieved July 12, 2013 from

http://www.ungei.org.

17. _______, 2013. Gender Equality Guide. Retrieved

July 15, 2013 from

http://pcoo.gov.ph/genderfairmedia/buk3_gender_

equality_guide.pdf

18. TeachTought, 2014. The Inside-Out Learning

Model: A 21st Century Framework. Retrieved

October 1, 2014 from

http://www.teachthought.com/learning/clarifying-

the-difference-between-students-and-learners/

19. Tracy, D.M., M.B. Lane, 1999. Gender-equitable

teaching behaviors: Preservice teachers‟ awareness

and implementation. Equity & Excellence in

Education, 32(3): 93-104.

20. UNESCO, EFA Global Monitoring Report 2008.


http://www.unesco.org/new/en/education/themes/l

eading-the-international-agenda/efareport/

Appendix A:

Gender Equity Classroom Observation Protocol Factor 1: Instruction and Assessment (Learning Actuators, Expanding Literacies, Climate of Assessment and Transparency)

Factor 1: Instruction and Assessment


B6: Invites both male and female visitors with non-traditional occupations into the classroom.

Factor 2: Classroom Management/Environment (Changing Habits and Changing Roles)

C13: Practices gender-neutral reading of, and writing on, students' work

C15: Doesn‟t ignore sexist remarks made by the students, but challenges them to be gender sensitive instead

Factor 3: Teacher-Student Interaction(Mentoring and Community)

C2: Provides more consideration, acclamation, and constructive feedback to males than with females

C3: Calls male students by name and asks them more often with complex and abstract questions than female students

Factor 4: Medium of Instruction (Mentoring and Community)

B3:Balances questions between males and females during class discussions and observes wait-time

C10: Stereotypes in the language being used.

Factor 5: Teaching Strategy (Self-Initiated Transfer and Thought & Abstraction)

B1: Encourages cooperative learning in cross-gender groupings by mixing the seating arrangement among males and females and by avoiding dividing students into a single-gender activity groups.

Factor 6: In Loco Parentis (Changing Habits and Changing Roles)

B4:Gives equal help and in-depth guidance to females as well as with males


Factor 7: Instructional Material (Climate of Assessment)

B13:Ensures that books, computer programs, and other curriculum materials are free from stereotyped gender-role behavior

Factor 8: Verbal Teacher Response (Changing Habits and Changing Roles)

C1: Uses "effort-appreciation" statements more often with male than female students.

Documents

Research Journal of Social Sciences - aensiweb.net 2015/1-12.pdf · Research Journal of Social Sciences ... Development and Validation of Observation Protocol for Gender Equity in