Social Data Science Data and Big data · (paid surveys) – Cheap talk – Diverse interpretations (e.g. 1-10 scales, Maasai example) – Very different quality: interview vs. internet

SocialDataScience

DataandBigdataDavidDreyerLassen

UCPHECONAugust12,2016

InGodwe trust,allothers mustbringdata

W.EdwardsDewing

Differenttypesofdata 2

Today:1.Empirical design2.datagenerating process3.modesofcollection

standardvsbig data;examples4.strategic dataprovision

roadmap

• Different datafordifferent questions• Theory andempirics,forecasting andhypothesis testing

• Effects ofcauses vs.Causes ofeffects• Datagenerating process• Modesofdatacollection – pros andcons• Strategicdatamanagementanddataproduction


Different datafordifferent questionsor

Different questions fordifferent data

Sometimes possible toseparatedatacollection processfromunderlying datagenerating process – andsometimes not

Fundamentaldifferencebetween what people doandwhat they say they do‘cheap talk’/‘putyour money where your mouth is’/honest/costly signaling


roadmap




What isyour question,again?1. Researchquestion from

theory2. Idealempirical design3. Feasible empirical

design/collection4. Results5. Adjustment of

theory/question/design6. Newresults7. …

A. WhatdatadowehaveB. Whatquestioncanthey

answerC. ResearchquestionD. Results


Allmodelsare wrong –butsome are useful

Two key goals1. Forecasting:individual behavior,policy

consequences,voting,ChampionsLeague,grades…Datascience/machine learning (butalsomacroeconomics)

2. Hypothesis testing,derived fromtheory´Traditional’socialscience


GeorgeBox

1. Forecasting• Example:Bankwants toforecast non-payment onloans(P_d:probability ofdefault)

• Couldn’t care less about theory• Rough”DataScience”: try topredict fromallavailabledata

• Suppose we findthat birth weight predicts default– Bankishappy,better fit (defer ethics etc)– Policy:does investing inpre-natal care reduce defaults?

• Inpractice: setofpredictors typically taken from(some)theory,even ifcasual

• Complications:ifcustomers knowthat P_d depends onbirth weight,would/should they disclose it?What ifloans only todisclosers?Would they tell thetruth?


2.Hypothesis testing• Theory (rationalchoice,sociology,biology,common sense,…)posits effect ofXonYA. Selection/typetheory:Peoplewho are impatient

cannot defer immediate pleasures ->smoke anddrinkwhile pregnant ->givesbirth sooner.Ifimpatient parents ->impatient children (whether bynatureornurture),we haveanexplanation.

B. Biological theory:low birth weight affects braindevelopment andneurological wiring forpatience.

• If(A),little role forpolicy;also,both can be trueatsametime

• Howtodistinguish:exogenous shock tobirthweight,butethically tricky...


Goodhart’s law

• Mostpopular:“Whenameasurebecomesatarget,itceasestobeagoodmeasure.”

• What he wrote:“Anyobservedstatisticalregularitywilltendtocollapseoncepressureisplaceduponitforcontrolpurposes.”


TargetsandMeasures

• You cannot be toldhow your bankconstructsyour P_d.Why?– Goodhart’s law:people will attempt tooutmaneuver measure

– (thought)example:spending onshoes goodindicator ofaccount overdraft ->shoe lovers willhaveothers buy forthem,ceases tobe agoodmeasure


CaseofGoogleFlu

• GoogleFlu:websearches forFlu symptomspredicted actual flu cases

• By-product ofGoogle’s main service• Butfrom2010,notsowell:overestimatedactual flu cases,partly asresult ofautosuggestfeature,partly becausemodelwas overfitted(we’ll return tothat)

• Bestpredictor:number ofcasespast week


roadmap




Effects ofcausesvs.

Causes ofeffects

Different questions• Effects ofcauses:intervention,what iseffectofpolicyXonoutcomeY

• Causes ofeffects:Why does Zoccur?


Effects ofcauses(forwardcausal questions)

• Narrow questions,sometimes (butnotalways)policyinterventions– Effect oftax change onbehavior– Effect ofregulation onrisk taking– Effect ofschooling onearnings– Effect ofsmokingonlung cancerpropensity– Effect ofpublichealth onschooling inAfrica– …

• Often,butnotalways,amenabletotreatments/randomization/experimentation


Causes ofeffects(reverse causal inference)

• Much harder,butoften moreinteresting–Why dosome people smoke?–What are thecauses ofdemocratization?–Why dosome people pursue aPhD why othersdropoutafter primary school?

–Why didGreece (almost)gobankrupt?• Tensionswith”effects ofcauses”– search forcauses sometimes derided as‘partychatter’


roadmap




What isthedatagenerating process?

Observational:endogenousdecisions,researcherpassivecollector ofdataRandomization:treatment-control(Some)exogeneity:policyinterventions,sometimeswithcomparisons,researcherssometimes involved

Important:moredatadoes notgivebetterresult/moreprecision if estimator isbiased


Datagenerating process

Randomizedexperiments• Distinguish– Labexperiments:traditionallycomputer-basedinecon,butalsoeyetracking/brainimages(fMRI)/physiological

– Surveyexperiments:assignsurveyrespondentstodifferentframes/treatments/primings,e.g.haveSocDems andLiberalssaysamethingandlookatsupport

– Fieldexperiments:experimentalcontrolintherealworld,e.g.bankschargingdifferentratestolearnaboutmobilityofcustomers;interventionsagainstteacherabsenteeisminIndia;…)

Different typesofdata 20

Randomizedexperiments

• Distinguish– Naturalexperiments(weatherinduced:effectsofpovertyonviolence,randomizationofnamesonelectionballots,…)

– Quasi-experiments(effectsofchangeinpolicy;effectoftaxreformontaxplanning;effectofimmigrantallocationoncrime)

• Throughout:exogenous(outsideoftheindividual)change



• Large,importantcurrentdebatein(development)economics

• CofE:whatareeffectsofpenaltiesonteachers’absenceinIndianvillageschools– evidencefromrandomizedexperiments

• Randomlyselectedteachersgetharshpenaltyforno-shows->differenceinabsenteeismcausaleffect ofpenalty

• (BroaderEofC Q:whyiseducationsectorinruralIndiasoinefficient?)



• Strongoninternalvalidity:fromrandomizationany effectonabsenteeismisfromharsherpenalties;goodfortestingtheory

• Weak(er)onexternalvalidity– wouldeffectbesimilarinAfrica?Wouldeffectfromlabworkoutsidelab?Why,whynot?

• (compare:medicineworksinsimilarwaysacrosslocations)



• Challenges– Limitstowhatcanbestudiedbyexperimentation( ethics;law;feasibility)

– Funding(fieldexperimentsexpensive,surveyexplessso)

– Oftenparticipationconstraint– voluntaryparticipants’gain>=0ornoincentive

– Subjectsleaveforvarious(systematic)reasons– Large-scalerandomizationcanbehardinfieldexperiments


Observationaldata

• Generatedwithoutexperimentalorexogenousintervention

• Typicallyrevealscorrelationsordescriptivepatternsthatcanbeinterestinginthemselves


Example:Inequality


Source:Piketty andSaez,Science2014,taxreturndata

Observationaldata

• Generatedwithoutexperimentalorexogenousintervention

• Typicallyrevealscorrelationsordescriptivepatternsthatcanbeinterestinginthemselves– Areinthemselvessilentaboutcausality– Theorymaybeprovidestructuretolearnaboutcausalmechanismunderstrongassumptions

– Mayconflatecorrelationandcausality


Observationaldata

• Exple:Doesbeinginprivateschoolsaffectgrades– Classic:CatholicschoolsandgradesinUS– Collectattendanceandgrades->runregression

• But:supposesomeparentsaremorefocusedonschoolingthanothers– Sendkidstoprivateschoolmore– Moreinvolvedinschool+homework

• Whatdohighergradesmeasure?– EffectofprivateschoolOReffectofinvolvedparents?


Observationaldata

• Whattodo?– Assignkids/parentsrandomlytoprivateschools?

• Morecomplicated–Waiting-listexperimentdesign:peoplewhosignuprevealthemselvesasschoolinterested,comparegradesbetweenthoseinprogramandonwaitinglist->muchnarrowerdesign

– Modeling(UScase):usefactthatCatholicsaremuchmorelikelytochooseCatholicschools


roadmap




Modesofdatacollection• (Ethnographic/participantobserver)• Survey– Interviewsurvey(inperson),phonesurvey,internetsurvey,…

• Administrativedata– Usedforadministrativepurposes– Somecountries:census,taxreturn– DK:CPR-registrybased

• (Primarycollection: texts,counting)• “Bigdata”:insocialsciencestypicallyaby-productofdigitalinformation


Modesofdatacollection

• Note:survey,admindata,bigdatacanallhaverandomized/exogenouselementsorbepurelyobservational

• OfteninLab/fieldexperiments:askaboutincome,educationetc – butmaybebiased

• Sometimes:combineexperimentaldatawithadminorbigdata(butrare)


Ethnographic

• Pros– Attempttounderstandsituationsfromparticipants’perspective

– Verydetailedobservations(e.g.dynamicsatameeting:whospeakswhen,wholistens,whonodsoffandflirtsetc)

• Cons– Verydifficulttogeneralize(ifeventhegoal)

– Typicallyverysmalln,notforstats

– Hardtoreproduce/replicate


Surveys• Pros

– Canbecheap– Elicitinfoonattitudes,

beliefs,expectations– Necessarywhennoother

meansexist– Combinewithopen-ended

info– Easilyanonymized (firms;

China)

• Cons– Canbeexpensive– Non-randomsamples,

sometimesverymuchso(paidsurveys)

– Cheaptalk– Diverseinterpretations

(e.g.1-10scales,Maasaiexample)

– Verydifferentquality:interviewvs.internet

– Notfullresearchercontrol:Interviewercompletions


Administrativedata

• Denmark,Norway,Sweden– Population-wide– Ex:Knowpopulation‘bypressingEnter’

• Mostothercountries:census(countingpeople),surveys,roughapproximations

– InDK,builtonCentralPersonRegistrynumber– Systemconstructedforsourcetaxationin1960s,nowusedasubiquitousidentifier

• WhydosomecountrieshaveCPR-likesystemsandsomenot?

BigDatainEconomics

Administrativedata

• Pros– Oftenfullpopulation– InDK:thirdpartyreported->noreportingbias,nosurveybias

– Verydetailed,nosurveyfatigue

– Oftenveryprecise,sinceusedforadminpurposes

• Cons– Nosoftdata(attitudes,expectations);canbelinkedtosurveys

– Privacyconcerns– Restrictedtowhatiscollectedforadminreasons,bothtypeandfrequency(e.g.annual)

BigDatainEconomics

Administrativedata

• LotsofworkinDanisheconutilizesregisterdata– Taxation– Education– Health– Financialdecisions– Labormarket

• Combinedwith– Personalitymeasures– Attitudes/politicalprefsfromsurveys

– Expectationsfromsurveys

– Biologicaldata(neuro-measures,genetics)

– Datafromexperiments

BigDatainEconomics

Viva la revolución?HarnessingtheDataRevolution

forGood

HumanDevelopmentReportOffice

Big data

BigDatainEconomics

NoagreedupondefinitionwhatBigDatais

• LargeN?• Highfrequency/muchdetail?

• Manydifferentmeasurements?

• Basedonwhatpeopledo(‘honestsignals’)– ctr surveys– Notalwayshonest

• Differenttodifferentpeople/traditions

• ToAmericans,Danishadmin/registerdataisbigdata

BigDatainEconomics

‘Bigdata’

• Pros– Oftenbasedonrealdecisions (asadmindata),butmoredetail,e.g.auctions

– Highfrequency (e.g.wifi),highgranularity->almost‘largeNethnographicdata’

– Sometimescheap/free

• Cons– Noestablishedprotocolforcollection

– Sometimesdubiousquality,selectionissues(bothknown/unknown)

– Start-upcosts– Evenmoreprivacyconcerns

– Corporategatekeepers->biasinaccess(Facebook,Google)

BigDatainEconomics

Characteristicsof‘bigdata’

• Structured(row/column-style)vs.unstructured(images/sound)

• Temporallyreferenced(date,time,frequency)• Geographicallyreferenced(wifi,bluetooth,Google)

• Personidentifiable(identifyvs.distinguishindividualsvs.notdistinguishindividuals)– Separatemedium(e.g.phone)fromowner

BigDatainEconomics

Example:SocialFabric

• Large-scale(N=1000)bigdataproject• HandedoutsmartphonestoDTUfreshmen• Collectedphone,SMS/text/email(notcontent),GPS,wifi,bluetooth data

• ->Where,when,withwhom• ->socialnetworks

BigDatainEconomics

Whyphonedata

• Phonesassociometers• Many/mostpeoplecarryphonewiththemallthetime

• WouldbeIMPOSSIBLEtohavepeoplereportindetailforevery10mineverydayforayear

• Forthisproject:tailoredsoftware,butrealizedthatmanyappscollectdetailedwifi-datawithouttelling

• Concern:take-upofphones

BigDatainEconomics


BigDatainEconomics

Phone locations0500hMondaymorning ->canpredictwherepeopleatgiventimewith85%accuracy


BigDatainEconomics

10minGPS wifi


BigDatainEconomics

Example:peereffectsineducationeconomics

• Studentsallocatedtostudyandsocialgroups,calledvectorgroups(randomly)

• Aretherepeereffects,i.e.arestudents’grades/healthbehavior/studybehavioraffectedbythegroup?

• Literature:sometimesyes,sometimesno;veryheterogeneous

• Why?Perhapsbeingallocatedtogroupisnot=toactuallymeeting/usinggroup

BigDatainEconomics

Example:peereffects• Thinkofallocationtogroupasintentiontotreat(similartoofferingtreatment)

• Interestingexample:Carrell etal,ECMA2013.Smallgroups,yespeereffects;largegroups:no/negativepeereffects– WHY?

• Usephonetomeasurefrequencyofgroupmembersbeingtogetherphysically,measuredbybluetooth

• Threeparts:(i)yestheyaremoretogether;(ii)moretogether=>workbettertogether;(iii)peereffects?

BigDatainEconomics

Broaderissue:Whomeets,andhowclosearethey?

• Again:usebluetooth signalstomeasuremeetings(duration,participants)

• Analyzes3.1mio meetingsovertwomonths• Someresults:– Women/womenpairs->closer– Facebookfriends->closer– Samestudy->closer– Differenceinbeauty->furtherapart– Oneoverweight,onenot->furtherapart

• Peoplewhostandvery(too)closetoothershavefewerfriends(!?)

BigDatainEconomics

Predictionvscausality

• Measureclassattendancefromphonedata(wifi/GPS/bluetooth)– Either:constructclustersatslotsknownasteachingtime;or:useadmininfoonclasslocationsandconstructGPSoverlays

• Facebookactivity

• Predictgrades

BigDatainEconomics

BigDatainEconomics

BigDatainEconomics

BigDatainEconomics

Predictionvscausality

Attendance->grades/comprehension– Peoplewhoattendmorelearnmore– PeoplewhospendlesstimeonFacebookhavemoretimeforstudying

AND/OR

Grades/comprehension->attendance– Findcourseshard->stayathome,moretemptedbyFacebook

BigDatainEconomics

Example:CSS

BigDatainEconomics

HeatmapofpeoplewithmobiledevicesonCSS(anonymous)

Example:DavidonSaturday

BigDatainEconomics

Example:DavidsomeSaturday

BigDatainEconomicsFleamarket

Example:howtomeasureconsumerspending

• Economicallyimportant:– Indicatorofhealthofeconomy– Importantforunderstandingindividualresponsestopolicy

– d.o.toeconomicshocks– Importantforconsumerprices->inflation->adjustmentsofwagesandtransfers

– Indevelopingcountries:importantforestimatesofpoverty,inequality

BigDatainEconomics

Example:consumerspending• Traditionalmethods:

– Consumerexpendituresurveys(DK:forbrugsundersøgelsen)

– Diaryorscanner– Errors,selection

• EconomistswantedaccesstoindividualspendingdatafromDankort foralongtime– Noluck

• Recently,StatisticsDenmarkgotaccesstoCOOP-carddatatomeasureinflation– Tobemadepublicsoon,

prettygoodfitwithexistingmeasures(andmuchfaster)

– Niceidea,incentivecompatible

– Indep ofpaymenttype– Butselection?

BigDatainEconomics

Example:consumerspending

• Attemptsindevelopingeconomics– Usesmartphonesasscannerormeansofpayment– whatcanweinferaboutindividualsfromsmartphoneuse(dedicatedusers)

– Selectionintowhohassmartphones– Butshouldbeseenagainstotherwaysofcollectingdata

• Qs:– Howcanweusesmartphonestoinferspendingbetter?– Whatkindsofeconomicallyinterestingdatacanwecollectviasmartphones?

BigDatainEconomics

StatisticalanalysisofBigData

• Manyobservations:whatdoesstatisticalsignificancemean?– Andwhatispracticalrelevance?Sizeeffects

• Multipletestingproblems?Ifbigdatageneratesmanyvariables,whynotrunthroughthemalltoseewhatissignificant?– Correctstandarderrors

• Insomecases,‘eyeballeconometrics’canbedifficult– Needsystematicapproach

BigDatainEconomics

Statistical/machinelearning

• Supposeyouhavenoorverylittletheorytoguideyou

• OLSisnotonlylinear,butalsopresumessomeideaofwhatactuallygoesinthereandhow

• Varian’sTitanicexample:whosurvivedtheTitanic– Twovariables:Classandage– Researcherdecide/guessvs.dataanalysisyieldmostlikely(decisiontree,butlotsmorecomplicated->Sebastian,later)

– Einav,Levin:Econshouldconsidermachinelearning

BigDatainEconomics

StatisticalanalysisofBigData

• Butwhatifyouhavetheory(orthinkyouhave)– e.g.combine econometricsandmachinelearning

• Goesbacktoolddebateineconomics– MiltonFriedman(1953): judgeamodelbyitspredictions,notitsassumptions

– Machinelearningmadeforpredictionnotforhypothesistestingandtheory(in)validation

BigDatainEconomics

roadmap




Strategicdatamanagementandproduction

• People/firms/governmentsdonotalwaysprovidetruthfuland/orcompletedata

• Example:Nopenaltyforlyinginsurveys– butnoreasonnottoeither

• Politicalreasonsforobscuringorinventingdata:GreeceinEU,Chineseeconomy

• Firms:Proprietaryinfo,competitionreasons,foolingcustomersandregulators(VW)

BigDatainEconomics

Strategicdatamanagementandproduction

• Individualdemandforprivacy(Wereturntothis)– Couldbeinstrumental:• lackofprivacydecreasesconsumersurplusbybetterestimateofreservationprice(e.g.Steering:Macvs PCwhenorderingonline)• Concernsaboutpoliticalissues

– Oranobjectiveinitself:Privacyasapoliticalgoal

BigDatainEconomics

Socialdesirability biasI

• Key concern insurveys,butmoregeneralproblem:What ifpeople answer soastoconformwithgeneralnotions ofwhat’s desirable?– Examples:Won’t admit tonotvoting orhavingsexually transmitted diseases,exaggerates income

– Reportsbuying healthy food vs unhealthy food– Important forasking/assessing sensitivequestions

BigDatainEconomics

Socialdesirability biasII

• Why?• Distinguish

a) self-deceptionb) impression management

• Example:What doyou value mostinapotentialmate?– Peoplesay:"kindandunderstanding”– Fromdatingdata:physical attractiveness,status– Biascould be both (a)and(b)

BigDatainEconomics

Documents

Social Data Science Data and Big data · (paid surveys) – Cheap talk – Diverse interpretations (e.g. 1-10 scales, Maasai example) – Very different quality: interview vs. internet