Research on data journalism: What is there to investigate? Insights from a structured literature review

MEDIA & DESIGN

Research on data journalism: What is there to investigate? Insights from a structured literature review

Julian Ausserhofer1,2,3, Robert Gutounig1, Michael Oppermann2,

Sarah Matiasek1,2 & Eva Goldgruber1

1: FH Joanneum University of Applied Sciences, Graz2: University of Vienna 3: Humboldt Institute for Internet and Society, Berlin

NODA16 Academic Pre-Conference #NODA16 21.04.2016, University of Helsinki

MEDIA & DESIGN

*Tool evaluation partners

** *

Supported by:

MEDIA & DESIGN

Research Interest

MEDIA & DESIGN

@julauss

Research literature on data journalism

2013: "internalist tendencies at [... the] early stage of academic research" (Anderson, 2013, p. 1007)

↓

2015: "an explosion in data journalism-oriented scholarship" (Fink & Anderson, 2015, p. 476)*

"rapidly growing body" of scientific studies (Lewis, 2015, p. 322)* *cited via Loosen, Reimer & Schmidt (2015, p. 2)

4#NODA16

MEDIA & DESIGN

How is the research literature developing?

What are the research gaps?

Research questions

@julauss 5#NODA16

MEDIA & DESIGN

Method

MEDIA & DESIGN

Structured literature review

@julauss 7#NODA16

MEDIA & DESIGN

"to develop insights, critical reflections, future research paths and research questions" (Massaro, Dumay & Guthrie, forthcoming)

It adopts "a replicable, scientific and transparent process [...] that aims to minimize bias [...]" (Tranfield, Denyer & Smart, 2003)

Why a structured literature review?

@julauss 8#NODA16

MEDIA & DESIGNUndertaking a systematic literature review

9

Adapted from Massaro et al. (forthcoming)

Writing a literature review protocol

Developing insights and critique through analyzing the dataset

Developing future research paths and questions

Determining the type of studies and carrying out a comprehensive literature search

Coding dataDefining the questions that the literature review should answer

@julauss#NODA16

MEDIA & DESIGN

10





● Empirical research on DDJ ● Social science focus, but open to other

disciplines ● Published after 1995

IncludedJournal articlesBook sectionsConference papersReports (from industry and research projects)PhD theses

Not includedBachelor's and Master's thesesPress reportsBlog posts

@julauss#NODA16

MEDIA & DESIGN

11





● Preliminary search with “data-driven journalism”

● Extracting related terms from the keyword section of research papers

@julauss#NODA16

MEDIA & DESIGN

12





Search termsalgorithmic journalismcomputational journalismcomputer-assisted reportingdata journalism

data-driven journalismdata-driven reportingdatabase journalismdatajournalismdatenjournalismus

quantitative journalism

No search termsaccountability journalismcrowdsourced journalismdatavizdatavis

ddjdrone journalisminvestigative journalismonline journalismopen journalism

@julauss#NODA16

MEDIA & DESIGN

13





Scientific DatabasesACM Digital Sowiport

EBSCO Springer

IEEE SpringerLink

JSTOR Taylor & Francis Online

ProQuest Web of Science

Science Direct Wiley

Scopus Google Scholar

Sociological Abstracts

@julauss#NODA16

MEDIA & DESIGN

14





772 search results ↓ Assessment of title, abstract & keywords

- by two independently working researchers(Thomas et al., 2004)

↓ 33 research publications

@julauss#NODA16

MEDIA & DESIGN

15@julauss#NODA16

Records identified from scientific

databases (n= 772)

Further publications from expert poll of

data journalism researchers (n= 4)

Excluded after screening (n= 739)

Preliminary corpus: publications included

after screening of records (n= 33)

References from preliminary corpus

(n = 1151)

Final corpus: Publications included in the systematic

review (n=40)

Excluded after screening (n= 1148)

Further publications included after screening

of references (n = 3)

Adapted from Fecher, Friesike & Hebing (2015)

MEDIA & DESIGN

16@julauss#NODA16

software-assisted qualitative content analysis (Kaefer, Roper, & Sinha, 2015; Mayring, 2000; Schreier, 2012; QSR International, 2015)

computational analysis of structural aspects(Kreibich, 2016; Lopez, 2009)

MEDIA & DESIGN

Results

MEDIA & DESIGN

Development of the literature over time

n=40

@julauss 18#NODA16

MEDIA & DESIGN

Publications by type and citations

n=40 bubble size = number of citations in Google Scholar

@julauss 19#NODA16

MEDIA & DESIGNAffiliations & collaborations

@julauss 20#NODA16

MEDIA & DESIGN

1787-2015 n=1644

@julauss

References per year

21#NODA16

MEDIA & DESIGNMost-cited references

Publication Nr. of Citations

Meyer, P. (2002/1973). Precision journalism: A reporter’s introduction to social science methods (4th ed.). Oxford: Rowman & Littlefield. 15

Parasie, S., & Dagiral, E. (2013). Data-driven journalism and the public good: “Computer-assisted-reporters” and “programmer-journalists” in Chicago. New Media & Society, 15(6), 853–871.

15

Gray, J., Bounegru, L., & Chambers, L. (Eds.). (2012). The data journalism handbook: How journalists can use data to improve the news. Sebastopol: O’Reilly.

13

@julauss 22#NODA16

MEDIA & DESIGN

@julauss

Theoretical frames

● Science and technology studies

● Actor network theory(Ausserhofer, 2015; De Maeyer, Libert, Domingo, Heinderyckx, & Le Cam, 2015; Parasie & Dagiral, 2013; Parasie, 2015)

23#NODA16

MEDIA & DESIGN

@julauss

Research designs & data collection methods

Method Nr. of studies

In-depth interviews 25

Content analysis 21

Survey 5

Short-term observation 3

Newsroom ethnography 1

Note. Content analysis includes analysis of news, databases, blogs, job ads, visualizations, briefings, manuals, and more. Short-term observation encompasses visits to the newsroom and participation in meetings. A newsroom ethnography is defined as a detailed study of a newsroom over the course of several days.

24#NODA16

MEDIA & DESIGNGeographical scope

Country Number of studies

United States 16

United Kingdom 14

Germany 5

International 3

n/a 3

Sweden 2

Switzerland 2

Norway 2

Netherlands 2

… …

@julauss 25#NODA16

MEDIA & DESIGN

@julauss 26#NODA16

Research gaps in data journalism research

● Comparision of practices between countries (Appelgren & Nygren, 2014; Parasie & Dagiral, 2013)

● Long-term studies (Davenport, 2000; Knight, 2015)

● Newsroom ethnographies (Parasie & Dagiral, 2013)

● Software studies (Garrison, 1999; Lewis, 2013; Stavelin, 2013)

● Reader experience studies (Segel & Heer, 2010)

MEDIA & DESIGN

Conclusion

MEDIA & DESIGN

@julauss 28#NODA16

● data journalism and its investigation has been developing rapidly

● quality improvements in the research

● issues with the literature: few publications refer to theory or methodology, just report what has been investigated

Conclusion

MEDIA & DESIGN

@julauss 29#NODA16

● practices in small news organizations, freelancers, local and mobile data journalism etc.

● gender

● digital methods: investigating the field through its platforms

● theory

●…

Research opportunities

MEDIA & DESIGN

Explore the literature online at:

http://literature.validproject.at

Julian Ausserhofer1,2,3, [email protected] @julauss

Robert Gutounig1, @sextus_empirico

Michael Oppermann2, @oppermann_m

Sarah Matiasek1,2 & @sarahmatiasek

Eva Goldgruber1

@evagoldgruber

1: FH Joanneum University of Applied Sciences, Graz2: University of Vienna 3: Humboldt Institute for Internet and Society, Berlin

NODA16 Academic Pre-Conference #NODA16 21.04.2016, University of Helsinki

http://literature.validproject.at

http://twitter.com/julauss

http://twitter.com/sextus_empirico

http://twitter.com/oppermann_m

http://twitter.com/sarahmatiasek

http://twitter.com/evagoldgruber

MEDIA & DESIGN

References

31

MEDIA & DESIGN

@julauss 32#NODA16

Anderson, C. W. (2013). Towards a sociology of computational and algorithmic journalism. New Media & Society, 15(7), 1005–1021. doi: 10.1177/1461444812465137

Appelgren, E., & Nygren, G. (2014). Data journalism in Sweden: Introducing new methods and genres of journalism into “old” organizations. Digital Journalism, 2(3), 394–405. doi: 10.1080/21670811.2014.884344

Ausserhofer, J. (2015). „Die Methode liegt im Code”: Routinen und digitale Methoden im Datenjournalismus. In A. Maireder, J. Ausserhofer, C. Schumann, & M. Taddicken (Eds.), Digitale Methoden in der Kommunikationswissenschaft (pp. 87–111). Berlin: digitalcommunicationresearch.de. doi: 10.17174/dcr.v2.5

Davenport, L., Fico, F., & Detwiler, M. (2000). Computer–assisted reporting in Michigan daily newspapers: More than a decade of adoption. Presented at the Association for Education in Journalism and Mass Communication (AEJMC) National Convention, Phoenix, Arizona.

De Maeyer, J., Libert, M., Domingo, D., Heinderyckx, F., & Le Cam, F. (2015). Waiting for data journalism: A qualitative assessment of the anecdotal take-up of data journalism in French-speaking Belgium. Digital Journalism, 3(3), 432–446. doi: 10.1080/21670811.2014.976415

Fecher, B., Friesike, S., & Hebing, M. (2015). What drives academic data sharing? PLoS ONE, 10(2), e0118053. doi: 10.1371/journal.pone.0118053

Fink, K., & Anderson, C. W. (2015). Data journalism in the United States: Beyond the “usual suspects.” Journalism Studies, 16(4), 467–481. doi: 10.1080/1461670X.2014.939852

Garrison, B. (1999). Newspaper size as a factor in use of computer-assisted reporting. Newspaper Research Journal, 20(3). Retrieved from http://com.miami.edu/car/baltimore1.htm

Kaefer, F., Roper, J., & Sinha, P. (2015). A software-assisted qualitative content analysis of news articles: example and reflections. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 16(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/view/2123

Knight, M. (2015). Data journalism in the UK: A preliminary analysis of form and content. Journal of Media Practice, 16(1), 55–72. doi: 10.1080/14682753.2015.1015801

Kreibich, C. (2016). scholar.py. Retrieved from https://github.com/ckreibich/scholar.py

http://com.miami.edu/car/baltimore1.htm

http://www.qualitative-research.net/index.php/fqs/article/view/2123

https://github.com/ckreibich/scholar.py

MEDIA & DESIGN

@julauss 33#NODA16

Lewis, S. C. (2015). Journalism in an era of big data. Digital Journalism, 3(3), 321–330. doi: 10.1080/21670811.2014.976399

Loosen, W., Reimer, J., & Schmidt, F. (2015). When data become news: A content analysis of data journalism pieces. Presented at the The Future of Journalism 2015 Conference, Cardiff.

Lopez, P. (2009). GROBID: Combining automatic bibliographic data recognition and term extraction for scholarship publications. In M. Agosti, J. Borbinha, S. Kapidakis, C. Papatheodorou, & G. Tsakonas (Eds.), Research and Advanced Technology for Digital Libraries (pp. 473–474). Berlin: Springer. Retrieved from doi: 10.1007/978-3-642-04346-8_62

Massaro, M., Dumay, J. C., & Guthrie, J. (forthcoming). On the shoulders of giants: undertaking a structured literature review in accounting. Accounting, Auditing & Accountability Journal.

Mayring, P. (2000). Qualitative content analysis. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 1(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/view/1089

Parasie, S. (2015). Data-driven Revelation? Epistemological tensions in investigative journalism in the age of “big data.” Digital Journalism, 3(3), 364–380. doi: 10.1080/21670811.2014.976408

Parasie, S., & Dagiral, E. (2013). Data-driven journalism and the public good: “Computer-assisted-reporters” and “programmer-journalists” in Chicago. New Media & Society, 15(6), 853–871. doi: 10.1177/1461444812463345

QSR International. (2015). Nvivo. Retrieved from http://www.qsrinternational.com/ Schreier, M. (2012). Qualitative content analysis in practice. Thousand Oaks: SAGE. Segel, E., & Heer, J. (2010). Narrative visualization: Telling stories with data. IEEE Trans. Visualization and

Computer Graphics, 16(6), 1139–1148. doi: 10.1109/TVCG.2010.179 Stavelin, E. (2013). Computational journalism: When journalism meets programming (Dissertation).

University of Bergen, Bergen. Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed

management knowledge by means of systematic review. British Journal of Management, 14(3), 207–222. doi: 10.1111/1467-8551.00375

Data & Analytics

Research on data journalism: What is there to investigate? Insights from a structured literature review