Upload
dataninja
View
739
Download
0
Embed Size (px)
DESCRIPTION
Introductory lesson about data journalism within science journalism and science communication during the International School of Science Journalism 2014 in Erice (June 10th, 2014).
Citation preview
ALESSIOCIMARELLIDatascientistatDataninja
[email protected]|@jenkin27
dtnj.it/erice14
InternationalSchoolofScienceJournalismTheDigitalWorld(Erice,June10th,2014)
akajenkin
PASTMasterDegreeinPhysicsattheUniversityofRome"LaSapienza"
MasterinScienceCommunicationattheInternationalSchoolforAdvancedStudies(SISSA-ISAS)inTrieste
PressofficerattheEuropeanLaboratoryforNon-LinearSpectroscopy(LENS)inFlorence
PRESENTFreelancedatajournalist,webdeveloper,opendataactivist,citizenscientist,...
Datajournalism&datavisualizationmadeinItaly
Youknowverywellhowitworks...:)
Astopic
Storiesabouttheedgeofscientificresearchandhumanknowledge.
Keyroleinrelationshipbetweenscienceandsociety.
Sciencejournalistcanbeawatchdogagainstfalsescienceand scientificfrauds.
Asmethod
Itwouldbeevidentin ,becausetheworkflowissimilartopoliceinquiriesorscientificresearch.
Manyinformationsfromdifferentsources,accountabilityproblems,hypothesisandproofs,trialanderrorcycles,andsoon.
Notonlyastory,butalsoadiscoveryitself...
Awordinabuzzwordsera
whenhisinvestigationisultimatelybasedon(ordrivenby)digitaldata,heacquiressuchprefix.
Ifajournalistwanttotelltheworld,andtheworldisnowmadeofdigitalandquantitativeinformations,hehastoacquireskillsinmanagementandinterpretationofdata,orhewillmissanopportunity.
Teamworkandmultidisciplinary
Nosefornews,publicinterest,intuitionbasedoncontestknowledge
Analyticalmind,mathematicalandstatisticalskills,intuitionbasedonscienceofnumbers
Teamworkandmultidisciplinary
Problemsolving,hi-techknowledgeinhardwareandsoftware,nerd(orgeek,ifyouprefer)mood
Artisticsensibilityandintuition,knowledgeinUserExperiencetheoryandtechniques
Miners,dustmen,researchers,andstorytellers
Publicsearchenginesordeepweb?Official5-starsopendataorwebspidersandscreenscrapers?Monitorandkeyboard,smartphoneandtouch,orbootsandmud?
Datashouldbereadbymachinesandnotbyhumans!Datasetscouldhideerrors,inconsistencies,lies...orshowonlyapartofastory.
Miners,dustmen,researchers,andstorytellers
Normalizationsandcomparisons,filtering,grouping,aggregation,correlations,...
Howtorepresentnumbersandrelationsamongnumbers?Yes,witharabicnumerals,butpicturesareworthathousandwords...aslongasyoukeepinmindthattherearefactsbehindthenumbers,and
(copyrightofTheGuardian).
Inmethod
Yourunintoadatasetandfeelthepresenceofapossiblenews...OR
...youhaveaninterest,anidea,athesis,soyouarelookingfordata.
Havingquantitativedataaboutaphenomenonmeansthatsomewherethereisa youhavetounderstand,test,verify...andinterpret!
Datathemselvescansuggestnewwaysforyourinvestigationorevenfalsifysomehypothesisorassumptions.
Commonsense,intellectualhonesty,professionalethics
Somerandomexamples
NewScientistAppstornadoeswarmingworldexoplanetsplancksealevel
TheTelegraphmapofwindfarm
SortingalgorithmsMeteorites
EarthJournalismNetwork
by GlobalEditorsNetworkHealth
(NYT)(ProPublica)
(ProPublica)Environment
(ProPublica)(CenterforPublicIntegrity)
(LaStampa)(LaStampa)
Astronomy(NYT)
Energy(PlanbureauvoordeLeefomgeving)
AmericanWayofBirth,CostliestintheWorldInsidetheGovernment'sDrugDataWhichEmergencyRoomWillSeeYoutheFastest?
NewYorkfloodsBreathlessandBurdenedWhenItalyisshakingItaly,adelicateland
Kepler’sTallyofPlanets
Biomassa
Researchdata,scienceworld,citizenscience
Hardsciencesandsocialsciences
Ok,neitherLHCpetabytesareforjournalists,norstatisticaldatafromepidemiologicsurveys.
But ,or(open),whynot?
Ifyouarenotspecializedinaspecifictopicorifyoulacktheknowledgeabouttheframework,youcanasktoanexpertyoutrust.
Youcanalsousenumbersnotinaninvestigation,buttotellacomplexstoryusinginfographicsandinteractivevisualizations.
Bibliographies,socialnetworksofscientists,infrastructures
Scienceisahumanactivityandanindustry(almost)likeanyother.
HowaretheEuropeanfundsinvestedinscientificresearch?Wherearethecentersspecializedinthetreatmentofspecificdiseases?Whysomewellknownmonitoringtechnologiesarenotusedinsomecountries?
Sensor-basedjournalism
Cheapelectronicsandsensors+
openhardware+
freeinformationsharing=
datafromstakeholdersotherthanscientists
It'searly,butpromising:SwissMakeOpenDataCampsJapanGeigermapat-a-glanceCitizenScience&Sensors
Ifyouhavedata,it'sbetterifyouknowhowtodealwiththem.
Ifyouthinkyoumayfindsomedata,it'sbetterifyouusethem.
Ifsomeoneusedata,it'sbetterifyoucancheckhisclaims.
Playwithdataisfunny!
Welcometothejungle!
Someexamples
PublicadministrationInternationalorganizationsNGOsCivicactivistsPressofficesLeaksSocialnetworksJournalisticsourcesSinglejournalistsOurselves...
Datamadepublicandreusable
(USA)(UK)
(Italy)(Indonesia)
...
Data.govData.gov.ukOpenDataHubOpenIR
Rememberthebuzzwordera?
Datafrombigscienceexperiments(Atlas,HumanBrainProject,...)
Socialnetworks(Facebook,Twitter,butalsoeBay,Amazon,...)
Maybeit'snotforjournalists,butit'sahottopic...
GoogleEarthEngine
Formachine,notforhuman
Thekeywordis !
Awell-formedtablerepresentastructureddataset.Alistoffacebookcomments,articlesofanewspaper,arecordedspeecharenotstructureddata(andsoarenotmachine-readable).
Italldependsontheformat
IfwehaveGladstoneGanderasbestfriend:spreadsheet(xls,xlsx,ods,csv,tsv);not-so-commongoodformats(xml,sql,json,shp,kml,...).
Ifwearenotsolucky:tablesorlistsinwebpages(html);simpletablesinwell-donepdfs(pdf).
IfwehaveMurphyasworstenemy:scannedimages,evenifinapdfwrapper(png,jpg,pdf);digitaldatabehindcomplexsearchengines.
Andifwehavethebestdataever,butunderclosedlicense?
Well-formeddatasets
Numbersarenumbers,stringsarestringsandnotnumbers,datetimemustalwayshaveasingleformat(ie.yyyy/mm/dd),localizationisimportant,nogendervaluesinnames'columnorsimilarmixings,everyelementsshouldbenamedwithaUniqueIdentifier(ID).
Datatypescomputerunderstands:integers(withsign,zeroincluded),floatingnumbers(withsign),datetime,charactersandstring(casesensitive),nullvalue(thestrangecaseofavaluethatstates"I'mnotavalue").
Andsimplecomparisonsarestrictlyequalities,alsoinstrings!
Aggregation,average,normalization,relativedifference,distribution,...
Asinglerule:correlationdoesnotimplycausation!Spuriouscorrelations:Correlated:
http://www.tylervigen.com/http://www.correlated.org/
Ataglance
Withgreatpowercomesgreatresponsibility
Thebasicideaisquitesimple:youhavequantitiesexpressedinnumbersandgeometricobjectsdefinedbydimensions(ie.radiusinacircle),soyoujusthavetodecidehowconnectyourquantitiestovisualdimensions.
Thereareseveral(un)commonchartsandendlesscombinations:scatterplots,lines,bars,areas,pies,donuts,bubblecharts,treemaps,wordclouds,alluvionaldiagrams,dendrograms,networks,streamgraphs,gauges,chorddiagrams,motioncharts,parallelcoordinates,sankeydiagrams,maps,choropleth,...
On thereisanendlesslistofexamples!d3js.orggallery
Buildingasimpledatasetoralargeandcomplexdatabasefocusedonatopicofpublicinterestleadstoavaluableproduct:thedatabaseitself,intendedasacollectionof(linked)dataplusmetadata.
Canapublicfrontendtosuchdatabase,designedforcitizens,journalists,stakeholders,beconsideredajournalisticoutcome?Ifjournalismisapublicgood,itcanbeaservice,notonlyaproduct...
Scraping"Copy&Paste"combo
forChromebrowserGoogleSpreadsheetfunction
forsimplepdfsPython(orotherlanguages)scriptsandlibraries
CleaningFiltersand"Find&Replace"toolsinspreadsheets
AnalysisPivottablesandsimplechartsinspreadsheetsDedicatedsoftwares(ie.open-source or )
Viz, , , , ,
, , , , , ,...
DataMinerIMPORTXML()Tabula
OpenRefine
QtiPlot QGIS
Datawrapper RAW GoogleFusionTables Tableau CartoDBinfogr.am easel.ly Timelinejs Timemapper StoryMap d3js
TinaCasagrand," ",TheOpenNotebook(2014)PaulBradshaw," ",Leanpub(2014)JohnMair,RichardLanceKeeble," ",abramis(2014)PaulBradshaw," "ClaireMiller," ",Leanpub(2013)NathanYau," ",Wiley(2013)SimonRogers," ",Faber&Faber(2013)JonathanGray," ",O'Reilly(2012)NathanYau," ",Wiley(2011)
Datajournalismforsciencejournalists
ScrapingforJournalistsDataJournalism
DataJournalismHeistGettingStartedwithDataJournalismDataPoints
FactsareSacredTheDataJournalismHandbook
VisualizeThis
Alessio"jenkin"[email protected]
@
Dataninja
Q&A
SWIM
jenkin27
www.dataninja.itschool.dataninja.it
dataninja.it/newsletter
school.dataninja.it/qa
sciencewritersinitaly.wordpress.com
Hacking+Marathon=Hackathon
ESPAD(Europeanstudentsanddrugs): http://www.espad.org/en/
RASFF(EUfoodsafety): http://ec.europa.eu/food/food/rapidalert/
http://ec.europa.eu/food/food/rapidalert/
TheRapidAlertSystemforFoodandFeed(RASFF)wasputinplacetoprovidefoodandfeedcontrolauthoritieswithaneffectivetooltoexchangeinformationaboutmeasurestakenrespondingtoseriousrisksdetectedinrelationtofoodorfeed.ThisexchangeofinformationhelpsMemberStatestoactmorerapidlyandinacoordinatedmannerinresponsetoahealththreatcausedbyfoodorfeed.
dtnj.it/rasff2013
http://www.espad.org/en/
Thisisthereportfromthefifthdata-collectionwaveoftheEuropeanSchoolSurveyProjectonAlcoholandOtherDrugs(ESPAD).Itisbasedondatafrommorethan100,000Europeanstudents.Overtheyearsabout500,000EuropeanstudentshaveansweredtheESPADquestionnaire.Atotalof36countriesandregionshavecontributeddatatothe2011ESPADDatabase.Drugslistincludescigarettes,alcohol,cannabis,otherillecitdrugs,tranquillantsandsedativeswithoutprescriptions.
dtnj.it/espad2011