Mixed Methods Approach in Research: Using an Example Focused on Gender and Race

MICHELLE V. PORCHE, ED.D.

BOSTON UNIVERSITY

RENÉE SPENCER, ED.D.

BOSTON UNIVERSITY

AdaptedfromaWebinarproducedbyMichellePorcheandMyraRosenReynoso,PhDforUMASSBoston,ICIwithsupportfromtheNaBonalInsBtuteonDisability,IndependentLiving,andRehabilitaBonResearch(NIDILRRgrantnumber90RT5024-01-00)

Outline • Intro• Whatismixedmethods

• Whatareadvantages/disadvantages

• Paperexample

• Resources

• QandA

John Creswell  Let’susequan>ta>veandqualita>vedata(orquan>ta>veresearchandquan>ta>veresearch)togethertogainamorecompleteunderstandingofourresearchques>ons.

 Mixedmethodsisaresearchapproach,popularinthesocial,behavioral,andhealthsciences,inwhichresearcherscollect,analyze,andintegratebothquan>ta>veandqualita>vedatainasinglestudyorinasustainedlong-termprogramofinquirytoaddresstheirresearchques>ons.

What is this Method Called?  Mul>-method Triangula>on Integrated Combined Quan>ta>veandqualita>vemethods Mul>-methodology Mixedmethodology Mixed-method Mixedresearch Mixedmethods

Basic reasons for using mixed methods: • Needdifferent,mul>pleperspec>ves,formorecompleteunderstandingofassocia>ons,processes,andmechanisms

• Hypothesisbuilding• Confirma>onandexplana>onofquan>ta>veresultswithdatadescribingqualita>veexperiences• Qualita>veinquiryasafirststepininstrumentdevelopment• NeedbePercontextualizedinstruments,measures,orinterven>onstoreachcertainpopula>ons• Needtoenhanceourexperiments• Needtogathertrenddataandindividualperspec>vesfromcommunitymembers• NeedtoevaluatethesuccessofaprogrambyusinganeedsassessmentANDatestofthesuccessoftheprogram

Why use mixed methods? § Validity–tocorroboratequantandqualdata§ Offset–offsetweaknessesofquantandqualanddrawonstrengths§ Completeness–morecomprehensiveaccountthanqualorquantalone§ Process–quantprovidesoutcomes;qualtheprocesses§ Explana>on–qualcanexplainquantresultsorvice-versa§ Unexpectedresults–surprisingresultsfromone,otherhelpstoexplain§ Instrumentdevelopment–qualemployedtodesigninstrument,e.g.,focusgroupstogenerateitempool§ Credibility–bothapproachesenhanceintegrityoffindings§ Context–qualprovidescontext;quantprovidesgeneral§ U>lity–moreusefultoprac>>oners

Quant and Qual Examples QUANTITATIVEMETHODS

 Descrip>vesta>s>cs,chi-squaretests Correla>ons,t-tests,ANOVAs,regressions

 Pathmodeling

 Propensityscorematching

QUALITATIVEMETHODS

 Interviews Open-endedsurveyques>ons

 Observa>ons Ethnography

 Contentanalysis

 Narra>veanalysis

NIH Grant Best PracSce Guidelines (Creswell, Klassen, Clark, & Smith, 2011)  Ra>onalewhymixedmethodsisbestsuited

• Useofrigorousestablishedquan>ta>veandqualita>vecriteriaformixedmethods

• Bothquantandqualdatacollectedandanalyzedinsometypeofintegratedway

• Jus>fica>onforuseofmixedmethods

• Cleardescrip>onofmethodsforeachcomponent,includinglimita>ons

• Statementofspecificbenefitofusingboth

Mixed Methods Proposals/Papers • Mul>pleconceptualframeworksofstudy

• Concurrentorsequen>aldesign

• Onemethodmayhavepriorityorbothmethodsmayhaveequalstanding

• Mixcanhappenthroughoutstudystar>ngatdatacollec>on,orisintegratedatanalysis

• Needforadequateresourcesfordatacollec>onandanalysis,includingqualita>veso\ware

• Sampling

• Wordlimitsinproposalwri>ngandinmanuscriptprepara>onforpeer-reviewedjournals–useoftablesandfiguresencouraged

Basic design mixed methods quesSons: ConvergentDesign

• Towhatextentdothequan>ta>veandqualita>veresultsconverge?

Explanatory

• Inwhatwaysdothequalita>vedatahelptoexplainthequan>ta>veresults?

Exploratory

• Inwhatwaysdothequan>ta>veresultsgeneralizethequalita>vefindings?

Preliminary Design ConsideraSons (Morse, 1991) Approach Type Purpose Limitations Resolutions

QUAL + quant Simultaneous Enrich description of sample

Qualitative sample Utilize normative data for comparison of results

QUAL Sequential Test emerging H, determine distribution of phenomenon in population

Qualitative sample Draw adequate random sample from same population

QUAN + qual Simultaneous To describe part of phenomena that cannot be quantified

Quantitative sample Select appropriate theoretical sample from random sample

QUAN Sequential To examine unexpected results

Quantitative sample Select appropriate theoretical sample from random sample

Creswell & Plano Clark, 2007

EmbeddedDesign

QUANTPre-testData&Results

QUANTPost-testData&Results

Intervention

QualProcess

Interpreta8on

QUANTData&Results

Interpreta8on

QUALData&Results

ConcurrentMixedMethodsDesigns

Why is Mixed Methods Research Valuable?  Answersques>onsthatothermodali>escannot

 ProvidesadeeperunderstandingoftheexaminedbehaviororabePerideaofthemeaningbehindwhatisoccurring

 Theinferencesmadewithmixedmethodsresearchcanbestronger

 Mixedmethodsresearchallowsformoredivergentfindings

 Mixedmethodresearchcanincludecultureinthedesignbygivingavoicetoeveryoneinvolvedinthebehaviorbeingexamined

Perceived Gender and Racial/Ethnic Barriers to STEM Success Grossman & Porche, 2014

Abstract

 Thismixed-methodsstudyexaminedurbanadolescents’percep>onsofgenderandracial/ethnicbarrierstoSTEM(science,technology,engineering,andmathema>cs)success,andtheirmeaning-makingandcopingregardingtheseexperiences.Thesampleincludessurveysfrom1024highschool-agedstudentsandinterviewsfrom53students.Logis>canalysisshowedthathigherscienceaspira>onssignificantlypredictedperceivedsupportforgirlsandwomeninscience.Analysisofinterviewsshowedthemesofmicroaggressions,responsestomicroaggressions,andgender-andracebasedsupport.Findingssuggestpar>cipantsvaryinpercep>onsofbarriers,yetaregenerallyop>mis>caboutovercomingsuchobstacles.

Design § Sequen>al:quan>ta>vesurveyfollowedbyin-depthqualita>veinterviewswithtargetedsubsample

§ Surveyresultsinformed“bucketcoding”ofinterviewnarra>ves

§ DeeperlevelmeaningcodingusinggroundedtheorytobePerunderstandassocia>ons

§ Narrowfocusonmanageableresearchques>on

Purposeful Interview Sampling (n=53)

HighInterestinSTEMLowAbilityinSTEM

HighInterestinSTEMHighAbilityinSTEM

LowInterestinSTEMLowAbilityinSTEM

LowInterestinSTEMHighAbilityinSTEM

InterestmeasuredbyplansforcollegeenrollmentinSTEMcoursesAbilitymeasuredbymathandscienceself-conceptDiversitybyrace/ethnicityandgenderStudentsfromeachofthefiveschools

Running head: PERCEIVED GENDER AND RACIAL BARRIERS TO STEM 1

Table 1 Sample Demographics (Student Report) and School Characteristics (Summarized in Publicly Available District Reports)

Liberal Arts Science High

Health Sciences

High

Technology

Design Finance Academy Full Sample

(Interview subsample) (n = 15) (n = 20) (n = 6) (n = 7) (n = 5) (n = 53)

Survey sample n = 605 n = 214 n = 56 n = 93 n = 56 n = 1024

% % % % % %

Gender Male 41.32 46.73 40.00 78.49 48.21 46.14

Female 58.68 53.27 60.00 21.51 51.79 53.86

Race/Ethnicity Black 5.81 29.91 45.45 38.04 50.91 18.26

Asian 31.56 25.23 0 1.09 1.82 24.02

Latino 3.82 15.89 25.45 41.30 12.73 11.33

White 43.52 11.68 7.27 1.09 0 28.52

Other 3.49 5.14 7.27 3.26 3.64 4.00

Bi/Multi-racial 11.79 10.28 14.55 15.12 25.45 13.28

School Characteristics Full District

Limited English Proficient 1.2 2.0 5.5 3.5 8.1 18.3

Low Income 27.9 61.6 66.9 64.6 59.2 72.7

Special Education 0.7 1.2 19.3 29.6 28.3 19.7

Dropout Rate 0.0 0.5 8.0 15.5 7.7 15.9

State Proficiency Tests (% pass)

Mathematics 100.0 98.0 49.0 34.0 22.0 55.0

English Language Arts 99.0 92.0 34.0 38.0 22.0 50.0

Table 2

Logistic Regression of Perceived support for underrepresented groups to STEM Success (Odds Ratios and 95% CI shown)

!Support for Girls and Women in Science

Support for African Americans and Latinos

in Science Gender (1=girls) 0.70 [0.50,0.99] 0.91 [0.69,1.22] Race African American/Afro-Caribbean 0.65 [0.39,1.09] 0.36 [0.22,0.57] Latino 1.33 [0.72,2.47] 0.47 [0.28,0.79] Asian 1.24 [0.74,2.08] 0.69 [0.46,1.03] Other 0.61 [0.28,1.31] 0.50 [0.23,1.08] Bi/Multi-Racial 1.13 [0.66,1.93] 0.60 [0.38,0.94] White (reference) 1 1 Nativity (1=U.S. Born) 0.59 [0.37,0.95] 0.77 [0.53,1.13] Maternal Education 0.97 [0.83,1.14] 0.97 [0.85,1.09] School Site Science High 0.77 [0.48,1.24] 1.17 [0.80,1.73] Health Sciences High 0.38 [0.20,0.74] 1.24 [0.65,2.37] Technology Design Academy 0.38 [0.20,0.70] 0.69 [0.40,1.19] Finance Academy 0.51 [0.26,1.02] 0.67 [0.35,1.27] Liberal Arts High (reference) 1 1 Science Self-Concept 0.96 [0.83,1.12] 0.93 [0.83,1.06] Math Self-Concept 0.97 [0.82,1.14] 1.07 [0.94,1.21] College Science Aspirations 1.46 [1.03,2.06] 1.09 [0.82,1.47] College Engineering Aspirations 1.05 [0.70,1.58] 1.18 [0.85,1.65] College Math Aspirations 0.95 [0.65,1.38] 1.26 [0.92,1.75] Note: Significant associations are in bold.

TheoreScally Informed Analysis  Iden>fica>onofrelevanttext–bucketcodes Readthroughconsideringhowtoconceptualize

 AppliedSue’sframeworkSueetal.(2007).Racialmicroaggressionsineverydaylife:Implica>onsforclinicalprac>ce.AmericanPsychologist,62(4),May-Jun2007,271-286.◦  Microassaults◦  Microinsults◦  Microaggressions

Microassaults: The dynamics of power  Jasmine,abi-racialAfricanAmerican/La>nastudent,stated:“Iguessscien>sts,youcansay,havepower.Idon’tknow.Andalotofpeopledon’tliketheideaofwomenhavingpower.Likewomenaresupposedtobelikeathomeorsomething.Orwiththekids.”

 24describedhowhismother,workinginapharmacy,lostapromo>ontoamanwhowaslessqualifiedandputinlesseffort:“Herbossgaveadudeajobsheearned.Shedidmoreworkthanhimandhegave[theman]thejob.”

Microinsults: Societal beliefs and assump9ons  AWhitestudent,JuliePe,madeadirectlinktosocietalstereotypes:Idothinkit’struethatalotof>mesgirlsarelessinterested.Andthatmightbebecauseoflikesociety,howthey’relikeitmightbelikeacycle,youknow?Likegirlsaretold,‘Ohgirlsarelessinterestedinscience.’Sothey’relike,‘Well,I’mlessinterestedinscience.’

 Bio10,anAfro-Caribbeanfemalepar>cipant,shared:AlotofpeoplelookataBlackpersonandseethatthey’renotgonnasucceed.Likeyouknow,they’regoingtobealwaysdependingonsomebodyelse,andtheymightlookatChineseandseethatthey’realwaysgoingtobegoodatscience,andlikethat’snotalwaysthecase.IknowalotofChinesepeoplethatgetD’sinscience.

Progress on microaggressions  Vicki,aWhitepar>cipant,talkedaboutprogressinreducinggenderbarriers:Ihaven’tencounteredanykindofdiscrimina>onagainstmeinscienceormathbecauseI’mgirl.Imean,Iknowthere’sthewholesortofblowupaboutthepresidentofHarvardsayinglikethatstuffaboutwomennotbeingabletodostuff,butImean,Ithinkthatasthings—as>meprogresses,it’sbecominglessandlessofanissue.

 Louis,aWhitestudent,reported,“There’salwaysgonnabesomeonewho’sgonnadiscriminateagainstpeoplelikethat,orseethemasdifferentbecauseoftheirbackgroundandstuff,butIthinkit’sgeqngmuchbePer.”

No microaggressions  Melonhead,aLa>nastudent,describedanopportunityforgirlsatschool:There’sthisthingforcomputers,ifyouwanttofixit,butonlyfemalescandoit.Soifyouactuallywanttosignupforit,youcangetascholarshipforit.SoIthinkthey’rereallysuppor>veofthat.

 ALa>nostudent,David,stated:It’sequal.It’sequal.‘Causewegotthesameamountofworkandstuffinschoolandwegotthesameguidelines.Ourstandardsarethesame.Ifyouknowwhatyou’redoing,thenthewayyoulook,orhowyoudostuff,doesn’tmaPer.

Table 3. Perceptions and experiences related to gender and race/ethnicity (n=53) Theme Total

Gender Race/ethnicity

Microaggressions Microassaults 45% 19% 36% Microinsults Progress No barriers Responses to microagressions Support

66% 36% 34% 40%

53%

57% 25% 28% 25%

30%

36% 17% 15% 36%

32%

Table 4. Perceptions and experiences related to gender and race/ethnicity (n=53) Theme Gender Race/ethnicity

total girls boys total

under-rep minorities

majority

(n=24) (n=29) (n=33) (n=20) Microaggressions Microassaults 19% 29% 10% 36% 39% 30% Microinsults Progress No barriers Responses to microaggressions Support

57% 25% 28% 25%

30%

67% 21% 33% 29%

58%

48% 28% 24% 21%

7%

36% 17% 15% 32%

36%

42% 24% 15% 39%

52%

25% 5% 15% 20%

10%

IntegraSon of Results  InterviewfindingsshedlightonsurveyresultsregardingperceivedSTEMsupport.

 Surveyfindingsforlowerlikelihoodthatgirlsperceivesupportforgirlsandwomeninscience,andthatunderrepresentedminori>esperceivesupportforAfricanAmericansandLa>nosinsciencemaybepar>allyexplainedbymoreprevalentqualita>vereportsofperceivedmicroaggressionsbythesegroupscomparedtotheirmaleorWhitecounterparts.

 Adolescentswhoexperienceorperceivemicroaggressionsrelatedtosciencemayperceivelesssupportforscienceaspira>onsfortheirgroup.

 Alackofassocia>onbetweenperceivedsciencesupportandhighscienceaspira>onsamongunderrepresentedminori>esissupportedbyqualita>vefindingsthatminoritystudentsexperiencediscrimina>on“notjustinscience,”whichmaybroadlyinhibitacademicaspira>ons,includingSTEM.

 Nocontradictoryfindingsemergedfromquan>ta>veandqualita>vedata.Consistentwiththegoalsofmixedmethodsresearch,qualita>vefindingsprovidedconfirma>onandnuanceforunderstandingquan>ta>veresults.

QuesSons

27

BooksonMixedMethodsResearch

 MICCRisathree-yearproject—co-ledbyBostonUniversity’sSchoolofEducation,MassINC’sGatewayCitiesInnovationInstitute(GCII),andtheRennieCenterforEducationResearch&Policy—thatwasdevelopedinclosepartnershipwiththeMassachusettsDepartmentofElementaryand

SecondaryEducation(MAESE)

 FormoreinformationonMICCR,pleasecontactLaVoniaMontouté,MICCRProgramDirector,atlmontoute@renniecenter.org.

 Madepossiblebyagrantfrom:

28