View
2
Download
0
Category
Preview:
Citation preview
ECNU Educational Research Methodology
Class 6 ( Chapter 8+9)Qualitative research designHistorical research
2016/5/19 1 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
CHAPTER 8Qualitative Research Design
2016/5/19 Fang Chen 陈芳 华东师大英语系2
ECNU Educational Research Methodology
1. definition2. common assumptions3. common components4. Perspectives for designing the qualitative
study5. reliability and Validity
Qualitative Research: Processes
2016/5/19 Fang Chen 陈芳 华东师大英语系3
ECNU Educational Research Methodology
McMillan and Schumacher (1993) “Primarily an inductive process of organizing data
into categories and identifying patterns (relationships) among categories.”
1. Definition
2016/5/19 Fang Chen 陈芳 华东师大英语系4
ECNU Educational Research Methodology
2016/5/19 Fang Chen 陈芳 华东师大英语系5
ECNU Educational Research Methodology2. Common Assumptions /
Epistemology of Qualitative Research
By epistemology of qualitative research we mean the underlying assumptions and basic ideas of how research is done.
5 basic descriptors as the underlying epistemology of qualitative research.
① Holistic② Natural Settings③ Reality exists as the subjects see it.④ Favor Post Hoc conclusions⑤ Phenomena are loosely modeled2016/5/19 6 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
⑤ Phenomena are loosely modeled
Qualitative research does not reduce phenomena to a basic set of components and therefore predictions (i.e., models) are not mechanically defined.
Prediction is flexible (more a set of concepts)
2016/5/19 7 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
In summary
2016/5/19 8 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
3. Common Components of Research
These are general components that are mostly common to qualitative research but they are quite flexible and depend heavily on natural settings.a. Working Design (or Emergent Design)b. Working Hypothesisc. Data Collectiond. Analysis and Interpretation
2016/5/19 9 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
2016/5/19 Fang Chen 陈芳 华东师大英语系10
ECNU Educational Research Methodology
a. Working Design
A preliminary design for getting a study under way…
Example: An ethnographic study of the school environment
of regular and learning disabled students to determine factors related to potential dropout.1. Which school to use? --- purposeful sampling2. Whom to interview or observe? --- teacher, students,
counselors3. Length of time?
2016/5/19 11 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
b. Working Hypothesis
In general, qualitative research is to use inductive inquiry. Data collection is usually done without any
preconceived theories. However, there are foreshadowing statements and sometimes even hypotheses that will direct the research.
Example: What is the role of the faculty in attempting to
reduce the dropout? (research question) Interaction of school administration and students.
(foreshadowed problem)
2016/5/19 12 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
c. Data Collection
Data are collected in natural settings; access to the setting is important:
Participant-observer Vs Observer
Interactive Vs Non-interactive
2016/5/19 13 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Interactive or non-interactive?
Typical methods of data collection include:
Interview Observation Collection and review of related documents
2016/5/19 14 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
How to collect data for our example on counseling and drop-out rate? Interview students and faculty. Observe the interaction taking place between
students and faculty. Review school records relative to factors such as
grading patterns. If available, interview recent dropouts.
Data Collection Example
2016/5/19 Fang Chen 陈芳 华东师大英语系15
ECNU Educational Research Methodology
skip
Two activities mentioned in the textbook: specimen records: a special case of observation
• A narrative description of one person in a natural noncontrived situation as seen by skilled observers over a substantial time period.
• Case study
oral history http://srmo.sagepub.com/view/the-sage-encyclopedia-of-social-science-research-methods/n670.xml
• Interviews taken through the use of a tape recorder???• Emphasize open questions; more efficient???
2016/5/19 16 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Skip: Specimen records: some details
2016/5/19 Fang Chen 陈芳 华东师大英语系17
ECNU Educational Research Methodology
Typically, the data can be quite large in qualitative research. A data record of 1000 pages or more is not unusual.
So it is also important to record all conditions and thoughts that are occurring during the collection as well.
2016/5/19 18 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
d. Analysis and Interpretation
Analysis is done during collection of the data so that working hypotheses can be checked and possibly modified.
Because of the large amounts of data it must be organized. This process is called Coding. Coding is a process of organizing data and
obtaining data reduction.
2016/5/19 19 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Analysis and Interpretation
In coding, the data is organized using categories. The various types of data are evaluated and
placed in the appropriate categories These categories are usually decided after the
data is collected (in some cases it may be before) They are selected by the researcher and are
specific to the study. Purpose for the research largely influences coding.
2016/5/19 20 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Coding Example
The dropout study. Teacher’s perceptions of potential dropout
characteristics Potential dropouts’ perceptions of school Patterns of potential dropouts’ academic
performance (behavior) patterns
2016/5/19 21 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Categories
Need not be mutually exclusive. Categories can be whatever as long as they
a) Accurately capture information.b) Are useful in describing phenomena.
2016/5/19 22 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
General codes
Depending on the data collected we might use: Setting or context codes.
• E.g. Vocational school, comprehensive school Subjects’ perceptions of people or things.
• E.g. Dropouts have different views about their teachers. Process codes
• E.g. different ways in which students go about dropping out of school
Based on experiences
2016/5/19 23 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Screenshot① Perspectives of children about interpersonal
conflicts.② Innovative capacity of secondary schools: team
involvement which has four categories.
More examples
2016/5/19 Fang Chen 陈芳 华东师大英语系24
ECNU Educational Research Methodology
4. Perspectives for Qualitative Research
In qualitative research the method of data collection and hypotheses are much more flexible than in quantitative research. In fact, there are two different approaches where the data collection is used to improve future data collection.a) The Funnel Approach
b) Modified Analytic Induction
2016/5/19 25 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
2016/5/19 Fang Chen 陈芳 华东师大英语系26
ECNU Educational Research Methodology
Questions
How do I choose between a funnel or modified analytic approach? Your case.
When to stop? The point of diminishing returns for data collection,
and additional data collection is not worth the time and effort.
2016/5/19 27 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
5. Reliability and Validity
Remember the traditional quantitative concepts of reliability and validity? They can cause problems in qualitative research: Reliability:
• External-Can replicate the study• Internal-Based on the internal constancy of experiment.
Validity:• External – Generalizability of conclusion• Internal – Can make a reasonable conclusion
2016/5/19 28 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Qualitative Views of Reliability
Reliability: External: So specific to natural settings, but can
make a persuasive argument so that the reader can evaluate if it can be replicated.
Internal: Typically, there are observations from multiple observers so we should compare the agreement rates. Or at least look at the agreement of conclusions.
• Interrator reliability• Concordance : the ratio of agreements to agreements and
disagreements
2016/5/19 29 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Qualitative Views of Validity
Validity External: Not usually concerned with
generalizability (tend to be case studies). Can be left up to the reader.
Internal : still apply; Can the data provide a conclusion?
2016/5/19 30 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
1. Interpretative validity Usefulness: the extent the report informs and
stimulates further research Contextual completeness: fullness and richness
of the description Research positioning: researchers’ direct and
indirect effects on the research sites, participants, etc.
Reporting style: description perceived as authentic or not
2. Trustworthiness
Arguing for internal validity
2016/5/19 Fang Chen 陈芳 华东师大英语系31
ECNU Educational Research Methodology
1. Interpretative validity2. Trustworthiness
1. Triangulation2. Member checking: subjects’ perceptions are
verified in a systematic manner3. Chain of evidence
• outlier• pattern matching• representative check of an interview or artifact• long-term involvement• coding check
Arguing for internal validity continued
2016/5/19 Fang Chen 陈芳 华东师大英语系32
ECNU Educational Research Methodology
1. Comparability the ability of other researchers to extend
knowledge based on the “richness and depth” of the description
2. Translatability the extent to which other researchers understand
the results given theory and procedures underlying the study
External validity:NOT the job of the qualitative researcher, but …
2016/5/19 Fang Chen 陈芳 华东师大英语系33
ECNU Educational Research Methodology
Break : 10 minutes
2016/5/19 Fang Chen 陈芳 华东师大英语系34
ECNU Educational Research Methodology
Use of Technology
Because of the large amounts of data, technology has made things a lot better
The Ethnograph NUD*IST (Non-numeric Unstructured Data-
Indexing, Searching, and Theorizing NVIVO
2016/5/19 35 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Types of Designs
While there are many designs specific to the study some general types are Case Study Ethnographic Phenomenological Historical Focus group
2016/5/19 36 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
CHAPTER 9:Historical Research
2016/5/19 37 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Historical Research
DEFINITION: A systematic process of searching for the facts and then using the information to describe, analyze, and interpret the past.
Goal: take a holistic approach to some setting occurring in the past, and understand and interpret it.
2016/5/19 38 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Values of Historical Research
1. Shows what is right and what is not working2. Helps provide insight to current events3. Provide perspective for decision making4. Can help predict future trends
• “Those who are unfamiliar with the mistakes of history are doomed to repeat them”
….
2016/5/19 39 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
The career path of a single school executive from 1914-1922.
P225 See conclusions.
Example
2016/5/19 Fang Chen 陈芳 华东师大英语系40
ECNU Educational Research Methodology
1. The past: within the immediate preceding year or go back
several centuries.2. May have a variety of foci: The history of the development of the teachers’
college in the US would focus on a movement or a concept.
Or we could focus on a specific institution.3. Deals with events occurring in natural rather
than contrived settings for the most part.
Basics
2016/5/19 Fang Chen 陈芳 华东师大英语系41
ECNU Educational Research Methodology
Interpretation is Key
Interpretation is important because the data collection is based on getting information from the past
Mainly, we are interpreting sources. Our interpretation reflects our values and interests. E.g. an interview, but from the historical data.
2016/5/19 42 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Knowledge of Sources is Key
Also, because all data is from previous sources it is important that we be very sensitive to the relevance of these sources (data) as not to bias their results.
Need to know where to look for information.
2016/5/19 43 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Sources of Information
Because the research involves the past they must use various sources that describe the period of study.
Written documents (books, papers, diaries, letters, etc…)
Physical remains or objects Folk songs, stories, personal interviews
2016/5/19 44 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Sources of Information
In collecting the data, it is important to differentiate between a primary and secondary source Primary – First hand account of event being studied
1) E.g. to stud desegregation problem– A court transcript of a desegregation hearing
2) John Dewey’s writing is the primary source to study his views
Secondary – Accounts that are at least one step away from event
1) A news paper editorial concerning the desegregation problem
2) An interpretation by Dewey’s student is secondary source evidence
2016/5/19 45 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Methodology
Edson (1986): “There is no single, definable method of historical inquiry.” (p.29)
However, there are 4 steps that have to be done in one way or the other. Figure 9.1.
① Identify the research problem② Collect and Evaluate Materials③ Synthesis of Materials④ Analysis, Interpret, and Formulate conclusions
2016/5/19 46 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
① Identify the research problem
This is the same as in all other designs. However, they are more flexible so the research problem and hypotheses (if they exist) can be as implicit or explicit as needed.
Also, in identifying the research problem and hypotheses you should be aware of the key terms (beware of vague hypotheses) Are there words that had or have different
meanings across time.
2016/5/19 47 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Characteristics of hypotheses in historical research
Hypotheses are not formulated in a statistical sense.
Instead, they are conjectures about characteristics, causes, effects, issues, etc…
Typically, these hypotheses rest on the assumption of fact.
https://en.wikipedia.org/wiki/The_Protestant_Ethic_and_the_Spirit_of_Capitalism
2016/5/19 Fang Chen 陈芳 华东师大英语系48
ECNU Educational Research Methodology
Example P228
a) Lewin and Hui 1989: Rethinking revolution: Reflections on China’s 1985 educational reforms.
b) Culter III, 1989: Cathedral of culture: The school house in American educational thought and practice since 1820.
c) Nelson-Rowe 1991: Corporation schooling and the labor market at General Electric.
d) Galenson 1998: Ethnic differences in neighborhood effects on the school attendance of boys in early Chicago.
2016/5/19 49 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
History of Education Quarterly by the History of Education Society.
Educational Studies by the American Educational Studies Association.
Comparative Educational Review by the University of Chicago Press
Harvard Education Review by the Harvard University Press
2016/5/19 Fang Chen 陈芳 华东师大英语系50
ECNU Educational Research Methodology
② Data Collection and Evaluation
Interpretation is key but because we are collecting sources that describe a time in the past we need to interpret these carefully.
We must evaluate each source to determine how useful it will be External Criticism Internal Criticism
These two are not mutually exclusive
2016/5/19 51 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Collection and Evaluation
External criticism: Establishes validity of DOCUMENT Here we are concerned with the reputation of the
article, mistakes, who was the author
Internal criticism: Establishes validity of CONTENT Did the author or document have a reason to lean
in a specific direction (i.e., leave out or add stuff)?
2016/5/19 52 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Collection and Evaluation
In evaluating external and internal criticism typically, more that one document is used. Evaluate in chronological order Use multiple sources
• Are reports consistent?
Without these, a source can’t be used
2016/5/19 53 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
③ Synthesis of Materials
In combining the material, we need to determine the correct set of information.
If sources contradict one another, we need to decide which one is correct.
Pull together information for continuity Maybe chronologically
Reformulate hypotheses
2016/5/19 54 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
④ Analysis, Interpret, Formulate
Final step is the decision making of the research problem
Hypotheses are supported or rejected
Recognize the possibility of alternative interpretations
2016/5/19 55 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Analysis, Interpret, Formulate
In making conclusions be sure not to:
Confuse associations with causality Fail to separate how people SHOULD act with how
people DO act Fail to distinguish between intent and
consequences
2016/5/19 56 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Quantitative methods in Historical Research
As was mentioned earlier, Historical Research is typically associated with qualitative research
However, there is the possibility that it will also use some quantitative analysis Usually these are summary statistics Use data from papers, files, census…
2016/5/19 57 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
Reporting Historical Research
Intro – Introduce the topic, ideas and research questions
Methods – Provides the method and form of the data
Results – Summarize the data that has been collected
Conclusion – Make general conclusions related to research problem
2016/5/19 58 Fang Chen 陈芳 华东师大英语系
ECNU Educational Research Methodology
The general structure of a plan or proposal
Reading
2016/5/19 Fang Chen 陈芳 华东师大英语系59
ECNU Educational Research Methodology
Reading:
Next week: Ethnographic Research
Assignment
2016/5/19 Fang Chen 陈芳 华东师大英语系60
Recommended