Embed Size (px)
Citation preview
Scholarly Communication Research Group
Changing landscape of journals in social sciences and humanities in Poland
Zagreb, 5th of April 2018Emanuel Kulczycki
The lessons from Polish science policy
1. Evaluation of Scientific Institutions in Poland.
2. Polish Journal Ranking as the key instrument of evaluation.
3. The landscape of scholarly journals in Poland.
4. How has journal evaluation changed Polish journals in social sciences and humanities?
5. Toward a new model of evaluation.
6. Lessons from the Polish case.
Structure of Presentation
2
3
1.Evaluation of Scientific Institutions in Poland
4
The context and the scale of evaluation exercise
Warsaw
Poland
38 million citizens 430 higher education
institutions110.000 scholars
3.000 scientific journals
5
1991 1998 2010 2018
Poland started the evaluation of scientific institutions
Parametric model and journal evaluation
Polish Journal Ranking (Version 1 – old)
Polish Journal Ranking (Version 2 – new)
The History of Polish Research Evaluation
6
2Polish Journal Ranking as the key instrument of
evaluation
the 2016 Polish Journal Ranking
List (part) Points Number of
Journals What journals are indexed
A list 15–50 11,271 Journals indexed in the Journal Citation Reports (JCR)
B list 1–152,209(over
1,200 from the SSH)
Polish Journals without an Impact Factor and not indexed in the European Reference Index for the Humanities (ERIH)
C list 10–25 4,111Journals indexed in the European Reference Index for the Humanities (ERIH)
7
Formal Evaluation
Bibliometric Evaluation
Expert-based Evaluation
Three dimensions of evaluation(The B list) Parameters
Percentage of authors from foreign countriesPublishing without postponements greater than 6 months
10 parameters
for instance:
2 indicators Predicted Impact Factor (PIF)
Scimago Impact Factor (SIF)
Recommendations by the Committees of the Polish Academy of Sciences
8
0
10
20
30
40
50
Points per publication
50
252515
5
Book Chapter Article – the B list (max)Monograph Article – the C list (max)Article – the A list (max)
Journals from Social Sciences: 50 points (max) if indexed in the Journal Citation Reports (very high Impact Factor in the subject category)
Journals from Humanities: 25 points (max) if indexed in the ERIH and many citations in Scopus
10
2.Landscape of scholarly journals
in social sciences and humanities in Poland
11
Polish Journals
356Scopus**
257Web of Science**
over 3,000 Polish scientific journals in all fields of science
2476Polish Journal Ranking***
* ** ***
~10% of those journals is from social sciences and humanities~20% of those journals is from social sciences and humanities~60% of those journals is from social sciences and humanities
Landscape of Polish scientific journals
Social sciences and humanities
Humanities
Social Sciences
15
Active Polish scientific journals in 2017N
umbe
r of J
ourn
als
0
10
20
30
40
50
60
70
80
90
100
Year of journal founding (1st year of publishing)
182018731879188718931897190219081911191419191922192519281931193419371940194719501953195619591962196519681971197419771980198319861989199219951998200120042007201020132016
16
Editorial standards of Polish Journals: situation at December 2017Sh
are
of jo
urna
ls
0%10%20%30%40%50%60%70%80%90%
100%
A number of points on the Polish Journal Ranking0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 20 25 30 40
ORCIDDOIAnti-plagiarism systemJournal management and publishing system
63.4%
26.4%
68.3%
61.8%
78.7%
45.9%25.8%
17.2%
Percentage of peer reviewed publications in English, local language(s) and other languages in the social sciences and humanities in 2014
EnglishLocal language(s)Other language(s)
Norway
Finland
Poland
Czech Republic
Flanders (Belgium)
Slovakia
Slovenia
Denmark
DOI :10.1007/s11192-018-2711-0
% of publications in English
4.How has journal evaluation changed Polish journals in
social sciences and humanities
It is easier to establish a new journal than publish a good paper
From to 2012 to 2015, nearly 600 Polish journals were added to the Polish Journal Rankings.
2012
2013
2015
0 750 1 500 2 250 3 000
2 212
1 807
1 639
19
20
What is the actual effect of science policy oriented on internationalization?
% of members of the editorial advisory board from foreign countries
% of authors from foreign countries
% of reviewers from foreign countries
% of articles published in the so-called congress languages, i.e. English, German, French, Spanish, Russian, and Italian
Journals submitted information for the last two years in each edition.
Science policy’s goal: Internationalization
In all groups of sciences, the primary effect of science policy has been a change in the composition of editorial advisory boards.It was the easiest way to play the game.
Journals from all sciences increased % of reviewers from foreign countries.
% of authors and % articles in the congress languages did not change significantly in the analyzed period.
21
Effects of Science Policy
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
2012 2013 2015 2012 2013 2015 2012 2013 2015 2012 2013 2015
Social SciencesHumanitiesHard Sciences
Internationalization of Authors
Internationalization of Reviewers
Internationalization of Languages
Internationalization of Editorial Boards
22
5.Toward a new model of evaluation
24
1. We need a bibliometrics indicator (not formal criteria) to assess journals from all fields.
2. Journals from the humanities need to be able achieve the highest number of points.
3. A part of local Polish journals need funding to improve their editorial practices.
Three main assumptions of the new model
25
Evolution of the Polish Journal Ranking
The A list
Journals indexed in the Journal Citation Reports
Polish journals without the Impact Factor
Journals indexed in the ERIH.
The B list
The C list
Old model New model
Journals indexed in SCOPUS (SNIP)
+Journals indexed in WoS and not in Scopus
+
up to 250 local Polish journals mostly from SSH
15–50 points
1–15 points
10–25 points
Three lists One list
Extra funding(grant program)
6.Lessons from the Polish case
Lessons
Each country has its own starting point (e.g. the share of publications in English) and the evaluation criteria must be fitted to it.
Web of Science is not appropriate for the evaluation of SSH journals.
If a bibliometric indicator is needed, SNIP is one of the best options for the SSH.
Using formal criteria does not allow to achieve science policy goals.
Local SSH journals need stable funding (for years!) to improve their editorial practices.
Expert-based assessment (made by the members of the Polish Academy of Sciences’ committees) was a total failure.
27
Recommendations for policy makers
A system of journal evaluation should acknowledge two types of journals in the SSH:
➡ those that are actually prestigious and internationally oriented or might become such;
➡ those that are actually locally oriented and have implemented the highest standards of editorial practices.
A system of journal evaluation should encourage small journals (e.g. from a single faculty or region) to consolidate.
A system of evaluation can promote good practices (establishing journal websites) but – most often – it promotes playing the game. Thus, criteria have to be clear, well-matched, and systematically revised.
28
Presented results were based mostly on below publications:
1. Kulczycki, E. (2017). ‘Assessing publications through a bibliometric indicator : The case of comprehensive evaluation of scientific units in Poland’, Research Evaluation, 45/1: rvw023. DOI: 10.1093/reseval/rvw023
2. Kulczycki, E. , Korzeń, M., & Korytkowski, P. (2017). ‘Toward an excellence-based research funding system: Evidence from Poland’, Journal of Informetrics, 11/1: 282–98. DOI: 10.1016/j.joi.2017.01.001
3. Kulczycki, E. , Engels, T. C. E., Pölönen, J., Bruun, K., Dušková, M., Guns, R., Nowotniak, R., et al. (2018). ‘Publication patterns in the social sciences and humanities: evidence from eight European countries’, Scientometrics. DOI: 10.1007/s11192-018-2711-0
4. Sorokowski, P., Kulczycki, E. , Sorokowska, A., & Pisanski, K. (2017). ‘Predatory journals recruit fake editor’, Nature, 543/7646: 481–3. DOI: 10.1038/543481a
5. Kulczycki, E. , & Rozkosz, E. A. (2017). ‘Does an expert-based evaluation allow us to go beyond the Impact Factor? Experiences from building a ranking of national journals in Poland’, Scientometrics, 111/1: 417–442. Springer Netherlands. DOI: 10.1007/s11192-017-2261-x
