View
6
Download
0
Category
Preview:
Citation preview
Benchmarking Research Performance using Incites Citation-based intelligence for research performance evaluation
Subhasree Nag, PhDSenior Solutions Consultant
2
Speaker Profile
Dr. Subhasree Nag is a senior solution consultant for the scientific and scholarly research division at Clarivate Analytics. She completed her M.Pharm from ICT, Mumbai, her PhD from Texas Tech University Health Sciences Center, USA and her post-doctoral training from Pacific Northwest National Laboratory, USA where she worked on anticancer drug discovery and pharmacokinetic modeling. She has 28 peer reviewed publications and more than 600 citations. As part of Clarivate Analytics, she is instrumental in carrying out author workshops and designing research capacity building solutions in different institutions all over the country. Her other interests include scientometrics and bibliometric evaluation
Trends in scholarly research
Efforts at objective approaches to allocation of credit
Changing nature of scholarly journal publishing
Global competition in the Sciences – BIG SCIENCE
Reputation management & demonstration of achievement
Funding pressures
A need for information tools to advance informed decision makingResearch Analytics– using metrics to understand research activities– addresses the need for administrators to answer the following:
• How does my institution compare to peer institutions (down to the paper and individual level)?
• What are the strongest fields at my institution? Which ones need improvement?• How many papers did my institution produce?• Which authors are rising stars?• Is my institution's research focus changing?• Who is collaborating with whom inside and outside the institution? How often? And where
is it most impactful?
What is INCITES?
InCites™ is a citation-based research evaluation tool on the Web that enables you to analyze institutional productivityand benchmark your output against peers worldwide.
This comprehensive resource supplies all the data and tools you need to easily produce targeted, customized reports...all in one place. You can conduct in-depth analyses of your institution's role in research, as well as produce focused snapshots that showcase particular aspects of research performance
How Clarivate Analytics can helpWith InCites Benchmarking & Analytics, you can:• Analyze institutional productivity and benchmark your output against peers worldwide.• Pinpoint influential researchers and emerging research trends.• Showcase strengths and identify potential areas for growth. • Monitor collaboration activity and track new collaboration opportunities. • Support accreditation activity, funding proposals, legislative agendas, alumni appeals, faculty and student
recruitment.• Plan a research strategy with metrics that can be tracked over time.• Pull multiple datasets (of 50,000) from the Web of Science Core Collection for deeper analysis.
Basic indicators – Type of Citation Metrics used in Incites
6
Productivity # papers
Total influence
# citations
H-index
Efficiency
Avg. citation rate
Percent of papers cited
Relative Impact/Bench-marking
Journal actual/expected citation rateCategory actual/ expected citation ratePercentile in categoryand mean percentile% papers in top 10% of theirfield% papers in top 1% of theirfield
Aggregated PerformanceIndicator
Specialization Disciplinarity index Interdisciplinarity index
Can be applied to an institution,a researcher, a research group,etc.
Key Metrics used in Incites
METRIC MEASURE IDENTIFYTimes Cited Total cites to an authors papers Authors with highest /lowest total cites to their papers
WOS documents Total number of papers by an author indataset Authors with highest/ lowest numberof publications
Average cites per document
Efficiency (or average impact) of author papers Authors with highest/lowestaverage impact
h-index An authors research performance. Publications are ranked in descending order by the times cited. The value of h is equal to the number of papers (N) in the list that have N or more citations
Authors with highest impact and quantity of publications in a single indicator
JournalActual/Expected Citations
Average ratio for authors papers. Ratio is relationship between actual citations to each paper to what is expected for papers in same journal/ publication year and document type
Authors who’s papers perform above (1) or below what is expect in their respective journals. Useful when comparing authors in different fields/ career length
Category Actual/Expected Citations (Category Normalized Citation Impact)
Average ratio for authors papers. Ratio is relationship between actual citations to each paper to what is expected for papers in same category/ publicationyear and document type
Authors who’s papers perform above (1) or below what is expected in their respective subject categories. Useful when comparing authors in differentfields/ career length
Average percentile Average Percentile for set of authors papers. Percentile is assigned to a paper within a set of papers from same subject category/year/ document type ordered most cited (0%) to least cited (100%)
Authors who’s papers are performing at the top or bottom of their respectivefields
7
Web of Science Core Collection: The InCites Data Source
Web of Science Core Collection: The InCites Data Source
• Cited references for all papers, back to 1900
• All author names and addresses captured for all papers
• Funding data from 2008-present
For many years Clarivate Analytics has provided a wide range oftools and services supporting accurate and effective research evaluation.
Our specialists work with Web of Science data and ensure maximum standardization and unification before delivery to customers.
Clarivate Analytics offers not just simple counts and averages, butreal “metrics” founded on baselines for comparison and normalized statistics.
Clarivate AnalyticsExpertise and Processing
Research Analytics Resources
Data
Address UnificationData Cleansing
& Standardization
Normalization and Baselines
Web of Science
Web of Science Core Collection Journal Selection Policy
Journals are selected by experts with:
150 years of experience
Fluency in 12 languages
0 affiliations with any journals or publishers
Publishing Standards
Editorial Content
International Focus
Citation Analysis
Four Points of Evaluation
• A complex process: no one factor is considered in isolation.
• Each journal is evaluated on its own merits, with an objective, unbiased approach.
• Covered titles are monitored to ensure that they maintain performance, and de-selected .
More information available in the Journal Selection Process Essay
InCites & the Web of Science Core Collection
INCITES PLATFORMWEB OF SCIENCE PLATFORM
Journal Citation Reports• Impact Factor - uses data from most
recent JCR data year + prior 2 years
Essential Science Indicators• Data from most recent 10 year period• Hot & Highly Cited papers in 22
categories
Benchmarking & Analytics• Data from most recent 35 year period
Web of Science Core Collection• Science Citation Index Expanded • Social Sciences Citation Index• Arts & Humanities Citation Index• Conference Proceedings Citation Index• Book Citation Index
• Emerging Sources Citation Index
Web of Science Core CollectionThe world’s most trusted citation index.
Master Journal List | Conferences List | Book List
Powerful bibliographic and cited reference search capabilities, together with the benefits of cited reference linking and navigation.
Cover-to-cover indexing of over 18,000 journals
Over 12,000 conferences covered, annually
Over 80,000 scholarly books
Science Citation
Index Expanded
Social Sciences Citation
Index
Arts and Humanities
Citation Index
Emerging Sources Citation
Index
Conference Proceedings
Citation Index
Book Citation
Index
Multidisciplinary International Influential
Save to InCites from Web of Science Core Collection
— See normalized metrics for each paper in your results set to put citation counts into context by field & journal
— Identify experts & centers of excellence on your topic of interest— View publication trends by country to understand the global research
landscape specific to your area of interest— Datasets: 50,000 documents (search results) or 50,000 documents from
Marked List.• 50,000 counted against your sort on the WoS results page • Content restricted to the Core Collection 1980 to present
20Custom datasets
per account
InCites Benchmarking & Analytics Current Dataset (February 2018)The data included in InCites Benchmarking & Analytics:
• Data updated monthly
• Web of Science Core Collection content indexed through the week of December 31, 2017
• Standard Address Unification (Organization-Enhanced)
Parameters Values
Source Editions SCI-E, SSCI, A&H, CPCI, CPCI-H, BKCI, BKCI-H
Citing Editions ALL
Document Types ALL
Source & Citing Years(publication date) 1980-2017
17
Which academic, government and industry groups produce the most impactful work in my specialty?
Research Impact Analysis & Visualization
Top 10 organizations publishing in immuno-oncology by Category Normalized Citation Impact (CNCI)
This graph shows organizations that published at least 50 oncology papers during the past 10 years (2007-2018) ranked by Category Normalized Citation Impact. All are producing research cited a rate far above average. Research from Bristol Meyers Squibb has been cited at over 11 times the average expected rate.1.0 = average impact in
field
18
Where are the centers of excellence in a particular region?Research Impact Analysis & Visualization
Top 10 Chinese universities publishing in nanomaterials by Category Normalized Citation Impact (CNCI)
This tree map shows organizations in China that published at least 200 nanomaterials papers during the past 10 years (2007-2018) ranked by Category Normalized Citation Impact.
You can use this information to target the top institutions in any region, globally, for partnership.
19
Which organizations produce the hottest research? Who is funding the hottest research?
Research Impact Analysis & Visualization
Hot Papers are papers published in the last two years, that have accrued the most citations in the past two months–they represent the top 0.1% of recently cited papers.
You can use this data to understand where the leading research in your area is being produced, and who is funding it.
Organizations publishinghot papers in immuno-oncology
Number of Hot Papers
Harvard University 7UNICANCER 7Yale University 6Gustave Roussy 6
VA Boston Healthcare System 6
University of California Los Angeles 5
Dana-Farber Cancer Institute 5Memorial Sloan Kettering Cancer Center 5
University of Chicago 4University of North Carolina Chapel Hill 4
Vanderbilt University 4
UTMD Anderson Cancer Center 4
Merck & Company 4
Organizations funding*hot papers in immuno-oncology
Number of Hot Papers
National Institutes of Health (NIH) - USA 14NIH National Cancer Institute (NCI) 8Merck & Company 3KWF Kankerbestrijding 2European Research Council (ERC) 2
*Funding data is derived from the funding acknowledgments in the journal articles.
20
Which KoLs have produced the most impactful work?KoL Assessment & Reporting
Top 10 Global Authors publishing on fuel efficiency by CNCI
1.0 = average impact in field
Rank Name Affiliation CountryWeb of Science
Documents
Category Normalized
Citation Impact
H-Index
1 Yang, Peidong Lawrence Berkeley National Laboratory USA 13 13.23 10
2 Baek, Jong-Beom Ulsan National Institute of Science & Technology SOUTH KOREA 11 12.3 9
3 Yang, Peidong University of California Berkeley USA 15 12.11 114 Dai, Liming Case Western Reserve University USA 27 10.63 205 Qiao, Shi Zhang University of Adelaide AUSTRALIA 11 10.52 106 Qu, Liangti Beijing Institute of Technology CHINA 12 9.55 97 Yu, Shu-Hong University of Science & Technology of China CHINA 17 8.86 138 Klemes, Jiri Jaromir University of Pannonia HUNGARY 10 8.62 79 Lewis, Nathan S. California Institute of Technology USA 27 8.09 1410 Jaroniec, Mietek Kent State University USA 12 8.06 11
This table shows authors that have published at least 10 papers on fuel efficiency between 2008-2017, ranked by Category Normalized Citation Impact. All are producing research cited a rate far above average.
You can use this data to inform: Recruitment Strategic partnerships
21
Who are the KoLs in a particular region?KoL Assessment & Reporting
Quickly locate experts publishing on any topic.
You can focus your search to any country or region.
22
What is the funding landscape for my specialty?Research Funding Analysis & Visualization
*Data derived from the funding acknowledgments in the journal articles
Top 10 organizations funding* immuno-oncology research by number of funded documents
You can instantly see which agencies, foundations and corporations fund work in your area.
Over 900 unified funders are currently available for analysis, with more added monthly.
Funding details are available for material from 2008-present.
23
Which institutions and authors are benefitting from which agency?Research Funding Analysis & Visualization
*Data derived from the funding acknowledgments in the journal articles
CNCI of Top 10 organizations funding* immuno-oncology research
BMS-Funded Institutions (Top 10 by output)
Web of Science
DocumentsMemorial Sloan Kettering Cancer Center 52
Harvard University 41Bristol Myers Squibb 40Johns Hopkins University 35VA Boston Healthcare System 34
Dana-Farber Cancer Institute 31Johns Hopkins Oncology Center 30
H Lee Moffitt Cancer Center & Research Institute 23
UTMD Anderson Cancer Center 22
Cornell University 201.0 = average impact in field
24Collaboration Analysis & Visualization
Industry Academia Collaboration-Which collaborations have the most impact?
BASF• Collaborating organizations
defined as co-authorship• Ranked clockwise by # co-
authored papers• Displaying CNCI
Top BASF collaborations in nanomaterials researchby number of co-authored documents
Quickly visualize your existing partnerships & determine which have the highest impact in science.
25Collaboration Analysis & Visualization
Industry Academia Collaboration-Which academic institutions is my competitor working with? Which collaborations have the most impact?
DuPont• Collaborating organizations defined
as co-authorship• Ranked clockwise by # co-authored
papers• Displaying CNCI
Top Dupont collaborations in nanomaterials researchby number of co-authored documents
Are you missing out on expertise that your competitors are exploiting?
26Collaboration Analysis & Visualization
Which corporations is a university working with? Which collaborations have the most impact?
Johns Hopkins Oncology Center • Collaborating organizations defined as co-
authorship• Ranked clockwise by # co-authored papers• Displaying CNCI
Top Johns Hopkins Oncology Center Collaborations in immuno-oncology by number of co-authored documents
View the extent of a university’s existing corporate partnerships to gauge opportunity.
Understand how your organization fits in with other research activities at the target.
27
InCites Examples with Screenshots
You can easily visualize what filter you have applied to your analysis
You can clear your filter in one click
All filter sections are expandable
You can choose two different metrics to:- Rank your results- Generate the visualization
Use this to e.g. Avoid having to use a thresholdAn item can be searched in the
list.Use this to search your
institution
Time period selector is a slider.Use this to avoid “unfinished” years
such as 2019
Define a threshold of number of WoS Documents or Times cited.
Very useful to avoid meaningless reports based on too few documents/citations
You can of course combine filters to demonstrate the infinite amount of angles one can use
I want to find all German institutions, NOT
corporateand THE ranked, that have
published at least 30Karticles and reviews in the
past five years
Remove any item from a list, just tick the box next to it and select “exclude from results”
This is particularly useful when one of the institution listed is part of the analyzed institution (e.g. University of California System)
For example, you can analyze the journals where an institution has published.
Instead of typing its name in the “Journals” section, you can simply click on its name in the “organizations” section, then refocus on “Journals”.
Demonstrating how you can bring a selection of documents to another section can be very impressive.
In this list of five institutions that have published at least 200K papers in the last
ten years, which one has the highest Normalized citation Impact?
In this list of five institutions that have published at least 200K papers in the last
ten years, which one has the highest proportion of Cited Documents?
Make sure you click on this to access a multitude of additional indicators
Click on “Browse Indicators” to add indicators to your report
In this list of five institutions that have published at least 200K papers in the
last ten years, which one has the biggest proportion of international
collaborations?
In this list of five institutions that have published at least 200K
papers in the last ten years, which one has the biggest H Index?
Filtering by research areas is now possible with a multitude of schema
In this list of five institutions that have published at least 200K papers in the last ten years,
which one is the most influential, in Physics, as defined by ESI?
!You need to “PIN TO TOP” these 5 institutions before setting the document
threshold back to zero!
Which one has the biggest Open Access output?
Which one has the most papers in
specific journals?
Which one has published articles
with one of the three 2013
chemistry Nobel prices?
Which one gets the biggest impact
when collaborating with
India?
you have to enter its name in the “Collaborations with Organizations”
section
OR
You can simply click on the name of the institution in a given list, then refocus the
analysis using “collaborating organizations”.
To study collaborations with a specific institution
!! When you use “Refocus”, try to do it from a list coming from a small set of filters, otherwise the refocus will take too long and/or not work at all !!
The German and UK institutions that collaborated with CNRS in the past 10 years
From this list, I can generate reports on as much as 51 indicators
The 10 journals in which University of California
Berkeley has published the most.
How much of these papers got cited?
If Tetrahedron decided to not
publish any author from a German
institution, what would happen?
What happened when Tetrahedron published a Nobel
Prize winner’s articles?
Where did this person publish in the last 10
years?
What are the journals classified
in the FAPESP Chemistry
category, and what are their impact performances?
The 10 countries publishing the most of Physics Condensed
Matter papers.
Are these papers coming from collaborations with industry
sector?
The 5 countries publishing the most of Physics
Condensed Matter papers.
Do they publish a lot of Highly Cited Papers?
What are the countries
publishing in Nature?
What are the countries
collaborating with Chinese
institutions?What are the
performances of these
collaborations?
Does UC Berkeley collaborate
internationally?Does it help in
terms of impact?
Are these 6 countries
publishing in this OECD subject
category?
The 10 disciplines (SCADC) in which Chinese institutions
publish the most.
What is their performance in terms of percentiles?
The 10 disciplines (SCADC) in which Chinese institutions publish the most.
Did they recently produce a lot of Hot Papers?
Did the world publish more in
Organic or Inorganic
Chemistry?Which subject get
more impact?
In which disciplines did this group (department?)
publish the most?
In Which OECD category can we
find UPMC’s publications?
Any strength or weakness?
In which subject did these 4 major players publish at least 300K papers
in the last 10 years?
Subhasree Nag, Sr. Solutions Consultant| subhasree.nag@clarivate.com | clarivate.com
Recommended