NAFSA PRE-CONFERENCE NAFSA PRE-CONFERENCE WORKSHOP #11WORKSHOP #11

RESEARCH FOR INTERNATIONAL RESEARCH FOR INTERNATIONAL EDUCATION AND EXCHANGE: EDUCATION AND EXCHANGE: METHODS, POLICY, AND METHODS, POLICY, AND

PRACTICEPRACTICE

Washington D.C.Washington D.C.

Sunday, May 25, 2008Sunday, May 25, 2008

Research OverviewResearch Overview From Principles From Principles

to to PracticePractice

Bruce La Brack, Ph.D.Bruce La Brack, Ph.D.blabrack@pacific.edublabrack@pacific.edu

Presenting forPresenting forMichael Paige, Ph.D. Michael Paige, Ph.D.

r-paig@umn.edu

with thanks to Rick Suttonwith thanks to Rick Sutton

Identify your audience!Identify your audience!

Research findings—no matter how Research findings—no matter how powerful—are powerful—are uselessuseless if they if they don’t find the right audience.don’t find the right audience.

QED, your QED, your first research stepfirst research step is to identify the audience is to identify the audience that can implement/change the that can implement/change the policy you’ve targeted. policy you’ve targeted. They They are your readersare your readers. And, yes, . And, yes, politics are always involved!politics are always involved!

Why are you doing this Why are you doing this research?research?

To document/benchmark existing To document/benchmark existing operations?operations?

To demonstrate change/trends in To demonstrate change/trends in IE?IE?

To promote good ideas & To promote good ideas & initiatives?initiatives?

To focus attention on key To focus attention on key agenda items?agenda items?

To influence policy?To influence policy?

Constructing a Constructing a Testable HypothesisTestable Hypothesis

Frame your questionFrame your question Put your question into a positive Put your question into a positive statementstatement• If yes…If yes…• If no…If no…

Identify Methodology to determine Identify Methodology to determine yes/noyes/no

Diagram each step of research, data Diagram each step of research, data collection, and analysis (critically collection, and analysis (critically important if working with other officesimportant if working with other offices—e.g., IR, registrar, admissions, etc.)—e.g., IR, registrar, admissions, etc.)

““Research Design” typically Research Design” typically considered as methodologyconsidered as methodology

Sample size (random vs. Sample size (random vs. proportional)proportional)

Control groupsControl groups Pre-tests & post-testsPre-tests & post-tests Instrumentation and validationInstrumentation and validation Internal threats to validityInternal threats to validity External threats to validityExternal threats to validity ReplicationReplication

Sutton, Miller, & Rubin: “Research Design in Assessing Sutton, Miller, & Rubin: “Research Design in Assessing Learning Outcomes of Study Abroad Programs” in Learning Outcomes of Study Abroad Programs” in Guide Guide to Assessment in Education Abroadto Assessment in Education Abroad (2007) (2007)

Research Design Issues: Research Design Issues: The BasicsThe Basics

Choice of MethodologyChoice of Methodology Choice of MethodsChoice of Methods Cost RequirementsCost Requirements Time Requirements Time Requirements Sampling Requirements– who, how, Sampling Requirements– who, how, when, how many?when, how many?

Research Logistics – scheduling, Research Logistics – scheduling, travel, recordkeeping, budgets, etc.travel, recordkeeping, budgets, etc.

Research Team – who, how many, what Research Team – who, how many, what qualifications? qualifications?

Use internal/external consultants?Use internal/external consultants? Protection of Human Subjects Protection of Human Subjects (Institutional Review (Institutional Review Boards/Research Permission)Boards/Research Permission)

Data Collection MethodsData Collection Methods Interviews (scheduled or open?)Interviews (scheduled or open?) Focus GroupsFocus Groups SurveysSurveys Inventories (new or existing?)Inventories (new or existing?) Participant ObservationParticipant Observation Journals - (inter-rater Journals - (inter-rater reliability?)reliability?)

EthnographyEthnography Case StudyCase Study Authentic Texts/Written Authentic Texts/Written MaterialsMaterials

Purist vs. pragmatistPurist vs. pragmatist

If constituents are favorably If constituents are favorably inclined, quick-&-dirty inclined, quick-&-dirty research has immediate utility research has immediate utility & benefit.& benefit.

If constituents are If constituents are notnot favorably inclined, favorably inclined, imperfections of methodology imperfections of methodology will be focus of attack.will be focus of attack.

Methodology TaxonomyMethodology Taxonomy

Positivist Positivist • Theory DrivenTheory Driven• Theory TestingTheory Testing• Hypothesis TestingHypothesis Testing• DeductiveDeductive• ObjectiveObjective• Statistical Analyses of DataStatistical Analyses of Data• Common Methods: Surveys, Common Methods: Surveys, InventoriesInventories

Methodology, contd.Methodology, contd. InterpretiveInterpretive

• Theory GeneratingTheory Generating• Exploring "Lived Experience"Exploring "Lived Experience"• Deep/’Thick” Description (C. Deep/’Thick” Description (C. Geertz) Geertz)

• ExploratoryExploratory• InductiveInductive• SubjectiveSubjective• Co-Construction of Meaning Co-Construction of Meaning (Researcher and Subjects)(Researcher and Subjects)

• Common Methods: Interviews, Focus Common Methods: Interviews, Focus Groups, Participant Observation, Groups, Participant Observation, Ethnography, Case Study, Personal Ethnography, Case Study, Personal Stories, Content Analysis, Stories, Content Analysis, Journals, Photos, MediaJournals, Photos, Media

Methodology, contd.Methodology, contd.

Critical ResearchCritical Research• Co-Construction of Research GoalsCo-Construction of Research Goals• Co-Construction of MeaningCo-Construction of Meaning• Researches Social Issues (e.g., Researches Social Issues (e.g., Inequity and Inequality)Inequity and Inequality)

• Purpose-Driven Research (Redress Purpose-Driven Research (Redress Social Issues)Social Issues)

• SubjectiveSubjective• Common Methods: Case Study, Common Methods: Case Study, Community-Based Research, Action Community-Based Research, Action Research, Action PlanningResearch, Action Planning

Research Challenges/ControversiesResearch Challenges/Controversies Objectivity/SubjectivityObjectivity/Subjectivity

““Hard” Facts = Quantitative data Hard” Facts = Quantitative data vs. vs. ““Soft” Anecdotal = Qualitative dataSoft” Anecdotal = Qualitative data

Postmodernism/DeconstructionPostmodernism/Deconstruction (no ultimate truth/objectivity, all(no ultimate truth/objectivity, all relative to viewpoint of participant(s)relative to viewpoint of participant(s) and who controls narrative)and who controls narrative)

Research EthicsResearch Ethics

Protection of HumanProtection of Human Subjects & ProtocolsSubjects & Protocols Obligations to RespondentsObligations to Respondents Publication ConventionsPublication Conventions Intellectual PropertyIntellectual Property AnonymityAnonymity RewardsRewards Researcher BiasResearcher Bias Post-Project DisseminationPost-Project Dissemination

A spectrum A spectrum ********** of research of research dissemination vehiclesdissemination vehicles

Published articlesPublished articles Published research notesPublished research notes Occasional papers seriesOccasional papers series Conference presentations/proceedingsConference presentations/proceedings Inter-agency white papersInter-agency white papers Institutional white papersInstitutional white papers Internal reportsInternal reports Participating parties and institutions -Participating parties and institutions - aa critical and often neglected componentcritical and often neglected component

Applied Research Applied Research Practical Practical Research AgendaResearch Agenda

IE research movement is growing, but IE research movement is growing, but still at somewhat rudimentary stagesstill at somewhat rudimentary stages

Many fundamental questions about IE Many fundamental questions about IE not yet addressednot yet addressed Independent research projects (often Independent research projects (often small) frequently proceed without small) frequently proceed without intentional collaborationintentional collaboration

As current wave of research builds, As current wave of research builds, opportunities to connect points of opportunities to connect points of convergence & dissonance will emergeconvergence & dissonance will emerge

Researchers need to be aware of each Researchers need to be aware of each other and of broader contexts, but are other and of broader contexts, but are largely focused on serving local needslargely focused on serving local needs

Alas, Research is currently a luxury Alas, Research is currently a luxury for most international educatorsfor most international educators