EDPSY 505

Conducting Quantitative Educational Research

Syllabus

• Key points– Introductory course

• Research design

– Office hours • Mon 3:30 to 5:00 and Tuesday 2:00 to 3:30• Or by appointment

Syllabus• Key points (cont.).

– Assessment plan.• Two exams (100 points each).

– Before exams we will generate a study guide.– Tend to select items from the textbook website.

» Multiple choice» Short Answer» Essay

– First exam items can be corrected for half credit.» We’ll talk about this more after the first exam.

• Study guide exam items (2 @ 20 points each).– Pass/fail.

• Final project (100 points).– We will discuss in greater detail today.

• Intermediate research project assignments (6 @ 10 points each).– Pass/fail.

Syllabus

• Assessment (cont.).– No extra credit.– No incompletes.

• Except for extreme circumstances (e.G., Illness, death of family member).

• Study Habits– Rule of thumb: 3 hours of study for every

credit hour.

Syllabus

• Professionalism– Pathfinder

• Honesty• Integrity

– Cheating, plagiarism, etc. will not be tolerated and will result in a referral to whoever is in charge of this place.

• Behavior– Class starts at promptly at 4:00– Turn off cell phones– If you must leave early let me know– Be respectful of others

Syllabus

• Work habits.– Read.– Due dates are non-negotiable.– All work should conform to APA guidelines.

• If in a field that does not use APA let me know in advance what guidelines your field uses.

• Other course policies.– Religious accommodations.– Disabilities.

• Inform and provide documentation.

Who Is Here?

• In groups of three or four.– Identify yourself and get know one another.

• Ask the following questions.– What are your professional experiences?– What is your program of study?– What topics within your program interest you?– What is your favorite recreational activity?– What is your claim to fame?

– Be prepared to introduce one member of your group.– Exchange email addresses and/or phone numbers.

– Contact if you miss class.– Bounce ideas.– New friends.

Discussion

• As a group discuss the following questions:– What is research?– Is reality knowable?– What does it mean to know something?– How does one go about knowing reality?

What Is Research?

How does one go about knowing reality?

It depends on how you answered the three other questions:

Ontology

Epistemology

Methodology

What is reality?

What is knowledge and what does it mean to know something?

A Paradigm Is

• "A set of basic beliefs (or metaphysics) that deals with ultimates or first principles. It represents a worldview that defines, for its holder, the nature of the 'world', the individual's place in it, and the range of possible relationships to that world and its parts, as, for example, cosmologies and theologies do." Guba & Lincoln.

A Paradigm Is

• Kuhn defines paradigms as having two characteristics:– "Their achievement was sufficiently

unprecedented to attract an enduring group of adherents away from competing modes of scientific activity."

– "It was sufficiently open-ended to leave all sorts of problems for the redefined group of practitioners to solve."

So what?

"Differences in paradigm assumptions cannot be dismissed as mere 'philosophical' differences; implicitly or explicitly, these positions have important consequences for the practical conduct of inquiry, as well as for the interpretation of findings and policy choices."

Guba & Lincoln

Paradigms As Human Constructions

• Any given paradigm represents simply the most informed and sophisticated view of its proponents.– Must rely on utility and persuasiveness rather

than proof.

Paradigms

• Positivism– Deterministic– Reductionism– Empirical observation and measurement

• Methods– Experimental, manipulative, verification

Paradigms (Cont.)

• Postpositivism– Theory testing– Probabilistic– Know reality imperfectly– Replication

• Methods– Experimental, surveys, causal-comparative,

observational, interviews

Paradigms (Cont.)

• Critical theory– Political– Empowerment– Collaborative– Change-oriented– Social justice

• Methods– Participatory action research

Paradigms (cont.)

• Constructivism– Understanding– Multiple participant meanings– Social construction– Theory generation

• Methods– Grounded theory, case studies, narrative

research

Critiques of Positivism

• Context stripping• Exclusion of meaning and purpose• Etic (outsider) vs. emic (insider)• Inapplicability of general data to individual cases• Exclusion of discovery dimension in inquiry • Theory-ladenness of facts• Value-ladenness of facts• Interactive nature of inquirer-inquired into dyad

Misunderstood as a central element “Passive”

Researcher“Active”

Researcher

Falsification

Accretion

“T”ruth

Verification

What “is” can only be

“what’s known”

“creatings”

“Is this the right conclusion?”

CP194

???

What Makes Research

Scientific?

According to the Dictionary Science:

• Is the systematic observation, identification, description, experimental investigation, and theoretical explanation of phenomena.

How does the No Child Left Behind Act of 2001 (NCLB)

impact educational research?

Who Cares? You should.

• The No Child Left Behind Act of 2001 uses the phrase “scientifically-based research” (SBR) 111 times.

• This has spawned an industry of consultants.

• It has created a very volatile atmosphere.

What Is ScientificScientific Research?(According to NCLB)

• The application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge.

• Systematic, empirical methods that draw on observation or experiment.

• Involves rigorous data analyses that are adequate to test the hypotheses.

• Is evaluated using experimental or quasi-experimental designs.

• Is reported in sufficient detail to allow replication.• Has been accepted by a peer-reviewed journal or

approved by an independent panel of experts through rigorous, objective, and scientific review.

What Paradigm Appears to Be Influencing NCLB?

Postivism?

Post-positivism?

Critical Theory?

Constructivism?

Between a Rock and a Hard Place

The rock.The rock.• Calling for scientifically based

research is good and needed.• “The recent enactment of no child

left behind, and its central principle that federal funds should support educational activities backed by “scientifically-based research,” offers an opportunity to bring rapid, evidence-driven progress – for the first time – to U.S. Elementary and secondary education.” – Coalition for evidence-based policy.

The hard place.The hard place.• Defining SBR as randomized

experimental designs is over-restrictive.

• “The requirement that research methods be restricted to group design with a preference for randomized clinical trials will significantly inhibit the development and validation of new scientific knowledge in education.” – American association on mental retardation (AAMR) board of directors.

“Council recognizes randomized trials among the sound methodologies to be used in the conduct of educational research and commends increased attention to their use as is particularly appropriate to intervention and evaluation studies. However, the council of the association expresses dismay that the department of education through its public statements and programs of funding is devoting singular attention to this one tool of science, jeopardizing a broader range of problems best addressed through other scientific methods. The council urges the department of education to expand its current conception of scientifically-based research.” – AERA council

What Is ScientificScientific Research?(According to the NRC)

• Science poses significant questions that can be investigated empirically.

• Science links research to relevant theory.

• Science uses methods that permit direct investigation of the question.

• Science provides a coherent and explicit chain of reasoning.

What Is ScientificScientific Research?(According to the NRC)

• Scientific findings replicate and generalize across studies.

• Scientists disclose research and encourage professional scrutiny and critique.

Mayer (2000)

• Let’s take a few minutes and read Mayer.

• What makes research “scientific”?

• How important is it that educational research be respected in “academia and in society in general”?

• Should “science” and “research” mean the same things in different disciplines?

• What questions/ comments do you have?

Mayer (2000)

The Big Picture

• There are many different research processes

• Each has its own:– Philosophy of inquiry– Methods of inquiry– Purposes for doing research– Processes and “rules”

• Quantitative research has its own• Here is one process:

A Scientific Process

Research Question

Defining the Problem

ReviewLiterature

ArticulateTheory

DefineHypothesis

Testing the Hypothesis

Subject sampling Instrumentation Research design Piloting

Collecting Data Choosing analyses Conducting analyses

Conclusions

Results or Findings

Scientific Thinking Vs. Everyday Thinking

• Everyday thinking– Biased questions

• Do you really support the war?

– Limited sampling• Your friends and family are different from my

friends and family

– Selective attention• Confirmation bias

– Inaccurate generalization• Stereotypes

Scientific Thinking Vs. Everyday Thinking (Cont.)

• Scientific thinking.– Empirical observations.

• Empirical: capable of being confirmed, verified, or disproved by observation or experiment.

– Systematic.– Objective.

• Less dependent on emotion or personal prejudices.

– Replicable.

Purposes of Scientific Research

• Exploratory– What is out there?

• Descriptive– What does this group look like?

• Explanatory– Why and how are these constructs related?

• Evaluation– Does this program work?

• Prediction– Who will become depressed?