W HERE IS O PEN A CCESS IN THE ARC' S ERA? John Lamp School of Information Systems

WHERE IS OPEN ACCESS IN THE ARC'S ERA?John Lamp

School of Information Systems

ERA assessment measures ERA ranking context ERA ranking analysis

Information Systems journals Business journals OA journals (DOAJ data)

Conclusions The next round

OUTLINE

A CHEQUERED CAREER

Projectionist Printer Public sector Pundit Provocateur Programmer Pedagogy Pontificator

NEW TECHNOLOGY AND THE AUSTRALIAN PRINTING INDUSTRY

Produced by the PKIU in 1979

Arguably the first examination by a union of the impact of information technology on jobs

GOODBYE GUTENBERG

Produced in 1980 Anthony Smith was

Director of the British Film Institute

A VIEW FROM 1980 … it is into this empty terrain that the new electronic information systems will make their appearance during their pioneering period.

Peer review Books Book chapters Journal articles Conference publications Non-traditional research output

Esteem Editor prestigious works of reference Membership of Learned Academy Category 1 research fellowships Membership of statutory committees Australia Council grants or fellowships

ERA MEASURES

Applied Patents Registered designs Plant breeder’s rights NHMRC endorsed guidelines Research commercialisation income

Citation Analysis Relative Citation Impact Centile analysis Relative Citation Impact classes

ERA MEASURES

HERDC Research Income Category 1 Category 2 Category 3 Category 4

Volume and Activity Analysis Eligible researchers profiled by level Traditional research outputs by type Non-traditional research outputs by type

ERA MEASURES

Ranked Outlets Journals Conferences

ERA 2010 Discipline Matrix, http://www.arc.gov.au/era/key_docs10.htm

ERA MEASURES

What is it? Quality? Influence? Prestige?

Papers determine journal rankings, not vice versa

Approaches Metrics Didactic pronouncements Consensus

OUTLET RANKING

“A journal ranking index is currently being developed, and the list will be finalised in consultation with the sector. More than 17,000 journals have been ranked across 100 disciplines. This is a larger number of journals than either the Thompson ISI or Scopus index use in their respective databases which contain approximately 15,000 journals.”

Consultation Paper, June 2008, p11

WHY HAVE AN ERA RANKING?

Organic Chemistry Seven (12%) journals at A* Publish fortnightly - 182 issues/year Many people have many A* papers

Information Systems Nine (5%) journals at A*

0806, less HCI, CS journals – see ACPHIS list Publish quarterly - 36 issues/year Few people have A* papers

RANKING IS DISCIPLINE SPECIFIC …

ERA WOS RANKINGInternational focus required for ranking > B?

It’s the mix of publications that matter Remember all disciplines are different

Looking at a marker for local relevance Peer reviews override mathematical formulas

ERA VIEWS

C

A*

A

B

“Journal articles are defined as either indexed or non-indexed. An ‘indexed journal article’ is an article that has been published in a journal indexed by Scopus and has an electronic identifier (EID). For disciplines where citation analysis is used … a ‘non-indexed journal article’ is an article that has been published in a journal which is not indexed by Scopus and does not have a Scopus EID.”

ERA 2010 Submission Guidelines, December 2009, p37

INDEXED ARTICLE

“For disciplines where citation analysis is used, the low volume threshold is 50 indexed journal articles. This means that no evaluation will be conducted for the relevant FoR for a given institution if the number of indexed journal articles over the six year research outputs reference period is fewer than 50 in any four- or two-digit FoR.”

ERA 2010 Submission Guidelines, December 2009, p12

VOLUME THRESHOLD

ACPHIS JOURNALS IN ERA & SCOPUS

ScopusIn Out

A* 26 0A 35 2B 42 10C 34 31Total 137 43

ERA JOURNALS

Total number of journals ranked: 20,712 Total number of DOAJ journals in ERA: 1466

http://lamp.infosys.deakin.edu.au/era/ Access via ANZFoR Access via journal/conference title Access via conference acronym Draft listings also available 70,000 hits per month ARMS Excellence in Research Management

Award, 2010

ERA DISTRIBUTION OF OA JOURNALS (N=1466)

Rank All DOAJA* 5% 1%A 15% 5%B 28% 20%C 52% 74%

ERA RANKING AND AGE: INFORMATION SYSTEMS (N=541)

ERA RANKING AND AGE: BUSINESS (N=1045)

OA JOURNALS BY START DECADE

ERA RANKING AND AGEOA JOURNALS (N=1453)

TEN CHALLENGES FOR OA JOURNALS

Peter Suber – the Scholarly Publishing and Academic Resources Coalition

SPARC OA Newsletter, November 20091. Measuring up – journal metrics2. Opening up – use of CC license3. Closing up – prestige vs quality4. Doubts about quality – peer review used?5. Doubts about preservation – dead OA journals6. Doubts about honesty – especially fee based OA7. Doubts about publication fees – corrupting peer review8. Doubts about sustainability – bills vs workload vs

quality9. Doubts about redirection – of funds from trad. journals10. Doubts about strategy – how to make them a success

CONCLUSIONS

Remember to take the issue of journal ranking in context

Rankings aren’t everything Their interpretation is questionable The profiles are discipline specific Publication frequency is hidden

No evidence of discrimination Darwin vs Machiavelli

Natural selection assumes open choice Social Darwinism Ranking Eugenics!

Assuming no change to ERA program Assessment in 2010 Call for comments etc in 2011 Next ERA Ranking list in 2012 More open/transparent process

Blogging comments on journals?

THE NEXT ROUND