Upload
lorna-wildgaard
View
132
Download
0
Embed Size (px)
Citation preview
For at ændre ”Enhedens navn” og ”Sted og dato”:
Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod
Lorna Wildgaard Royal School of Library and Information Science
Birger LarsenDepartment of Communication, AAU-CPH
(Scaling analysis) of author-level bibliometric indicators.
For at ændre ”Enhedens navn” og ”Sted og dato”:
Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod
CONTRIBUTE TO THE DISCUSSION:
TODAY’S MEEThttps://todaysmeet.com/STI2014
sign in with your email, create a password, confirm and log in
OR
Log in with Gmail
Dias 202/05/23
For at ændre ”Enhedens navn” og ”Sted og dato”:
Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod
PURPOSE OF THE INVESTIGATION
Quantifiable and objective alternative to other metrics when evaluating faculty members for academic advancement.
Dias 302/05/23
For at ændre ”Enhedens navn” og ”Sted og dato”:
Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod
WHAT IS A RESEARCHER?
Dias 402/05/23
For at ændre ”Enhedens navn” og ”Sted og dato”:
Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod
MOTIVATION
Dias 502/05/23
DATA
2154 scholars identified in online questionnaire.
793 working links to online CVs identified in
sampling strategy across 4 disciplines and 5
seniorities
Astronomy n203:
PhD n15Post Doc n49Assis Prof n27Assoc Prof n72Prof n40
Environment n203:
PhD n3Post Doc n18Assis Prof n42Assoc Prof n85Prof n55
Philosophy n250:
PhD n9Post Doc n23Assis Prof n49Assoc Prof n82Prof n87
Public Health n137:
PhD n9Post Doc n14Assis Prof n31Assoc Prof n53Prof n30
Data collection start date: 13th June 2013.
Publication data of 750 researchers included. Data collection completed: July 10th 2013
Astronomy n203:
PhD n15Post Doc n48Assis Prof n26Assoc Prof n67Prof n37
Environment n203:
PhD n3Post Doc n17Assis Prof n39Assoc Prof n85Prof n51
Philosophy n250:
PhD n9Post Doc n22Assis Prof n45Assoc Prof n75Prof n78
Public Health n137:
PhD n9Post Doc n14Assis Prof n30Assoc Prof n50Prof n29
CVs and publication data of 793 scholars collected from Google Scholar, via Publish or Perish.
Excluded n43:Dead links n12Not included discipline: n15Duplicates: n1No publication list: n13No identifiable seniority: n1Impossible to find in POP: n1
SUBJECTIVE GROUPING OF 54 INDICATORS
Dias 802/05/23
"A review of the characteristics of 108 author-level bibliometric indicators", Scientometrics, DOI: 10.1007/s11192-014-1423-3
METHOD 1: IDENTIFICATION OF CENTRAL INDICATORS
Discipline Index Calculation nCorr.
Astronomy Hg The square root of (h multiplied by g). 25
Environ. Sci. H, H2
Publications are ranked in descending order after number of citations. H is where number of citations and rank is the same. H2 is where the square of the number of papers is equal to the number of citations.
26
Philosophy IQP
IQP= expected average performance of scholar in the field, amount of papers that are cited more frequently than average and how much more than average they are cited (Tc>a)
28
Pub. Health G
Publications are ranked in descending order after number of citations. G is where the the square root of the cumulative sum of citations is equal to the rank
23
MDS SCALING: SIMILARITY/DISSIMILARITY OF INDICES
ASTRO Enviro. Sci
25%
47% 38 %
24%
ENVIRO
PHIL PUB. HEALTH
EXPLORATIVE FACTOR ANALYSIS
Dias 1102/05/23
Discipline Publication & recognition
Normalized for field or time
Miscellaneas
Astronomy 57.3 % (0.78) 11.8 (0.49) 8.3 (-0.028)
Environ. Sci 57.2% (0.77) 6.2 (0.04) 10.4 (0.89)
Philosophy 53.6 (0.82) 7.0 (0.50) 10.4 (0.03)
Public Health 56.2 (0.77) 6.6 (0.00) 12.1 (0.59)
24-32 indicators in dimension 1 4-9 indicators in dimension 2 3-15 in dimension 3
For at ændre ”Enhedens navn” og ”Sted og dato”:
Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod
REASSESSING THE METHOD
Purpose: Quantifiable and objective alternative to other metrics when evaluating faculty members for academic advancement.
What we have learnt so far:1. Publication and citation data is highly skewed2. Transforming the variables with log, inverse, sqrt did not
improve the normality assumption of the data or improve the MDS or the Factor Analysis,
3. Recoding the variables into categorical groups resulted in lack of detail and still not significant results (a lot of work, inconclusive results
So we returned to non-parametric and descriptive analyses of the data – simple seems to be more informative when we have skewed data that builds on publications and citations.
Dias 1202/05/23
For at ændre ”Enhedens navn” og ”Sted og dato”:
Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod
DIFFERENCE IN MEDIAN PUBLICATIONS BETWEEN SENIORITIES
PublicationsMedian
Post Doc-PhD
Assis Prof – Post Doc
Assoc. Prof – Assis Prof
Prof.-Assoc Prof
Mean difference
Astronomy 12.5 20 22 28.5 20.7Environment 5 9 11 22.5 11.8Philosophy 3 2.5 0.5 11 4.25Public Health 5 11 21 33 17.5
Dias 1302/05/23
DIFFERENCE IN MEDIAN CITATIONS BETWEEN SENIORITIES
CitationsMedian
Post Doc-PhD
Assis Prof – Post Doc
Assoc. Prof – Assis Prof
Prof.-Assoc Prof
Mean difference
Astronomy 51.1 500.5 512 675 434.7Environment 7 107 178 109 100.2Philosophy 7.5 -1.5 1.5 21 7.1Public Health 20.5 86.5 351 436 223.5
For at ændre ”Enhedens navn” og ”Sted og dato”:
Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod
P & C INCREASE WITH SENIORITY. DO OTHER INDICATORS?
Dias 1402/05/23
DISCIPLINE OUTPUT EFFECT OF OUTPUT
IMPACT OVER TIME
QUALIFY IMPACT TO FIELD
RANK PORTFOLIO
Astronomy PC, sc, nnc, Sig, Csc, Fc,
Cage, AWCR, AWCRpa, AW, AR
Sum pp top ncits, IQP, NprodP
Millers H, h, A, R, g, hg, e, Q2, POPh
Enviro. Sci. P, FpC, CPP, Sc, FracCPP, nnc, Sig, Csc, Fc
Cage, AWCR, AWCRpa, AW, AR
Mcs, sum pp top ncits, mean mjs mcs, max mjs mcs, IQP, NprodP
Millers h, h, m, A, R, g, hg, e, Q2, H2, POPh
Philosophy P, Fp C, Sc, nnc, Sig, Csc, Fc Cage, AR NprodP m,A,R,g,e,
H2
Pub. Health P, Fp C, Sc, nnc, Sig, Csc, Fc
AWCR, AWCRpa, AW, AR
Mcs, Sum pp rop ncits, Sum pp top prop, NprodP
Millers h, m, A, R, g, hg, e, Q2, H2, PopH
For at ændre ”Enhedens navn” og ”Sted og dato”:
Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod
ARE PUBLICATION & CITATION COUNT EFFECTED BY GENDER?
nMales nFemales Md P, male
Md P, female
Md C, male
Md C, female
Astronomy 162 30 48 39 881 518Environ. Sci 160 35 29 18 321 135Philosophy 179 43 9 8 12 8
Pub. Health 79 53 31 29 311 353
Dias 1502/05/23
Environmental Science: Significant difference in the amount of publications produced by male and female researchers, U=2036, z=-2.525, p=0.012, r=0.18. Significant difference in the amount of citations male and female researchers receive, U=2056, z=-2.460, p=0.014, r=0.176
For at ændre ”Enhedens navn” og ”Sted og dato”:
Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod
ARE PUBLICATION & CITATION COUNT EFFECTED BY ORIGIN?
Dias 1602/05/23
For at ændre ”Enhedens navn” og ”Sted og dato”:
Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod
ARE PUBLICATION & CITATION COUNT EFFECTED BY ACADEMIC AGE OR SENIORITY?
Purpose: How well do seniority and academic age predict number of publications? How much of the variance in publication scores can be explained by scores on these two scales?
Method: Multiple Regression
Results (ALL FIELDS):The model which controls for seniority and academic age explains between 22-36.2% of the variance in publications (A=36%, E=36%, P=30%, PH=22%) and 1-22% of the variance in citations, (A=18%, E= 19%, P=0,9%, PH=22%.
Conclusions:Academic Age makes the largest unique contribution as a predictor of publications or citations, Seniority makes very little contribution .
Dias 1702/05/23
For at ændre ”Enhedens navn” og ”Sted og dato”:
Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod
Purpose:Controlling for the effect of gender and origin, is our set of variables (academic age and seniority) still able to predict a significant amount of the variance in publication count?
Method: Hierarchical Regression (ATT: high correlated data, assumptions of normality violated)
Results (All Disciplines):Only Academic age and seniority made a statistically significant contribution to the model. With academic age recording a higher beta value (.30-.46) than seniority (.18-.25) in each discipline.
Dias 1802/05/23
ARE PUBLICATION & CITATION COUNT EFFECTED BY ACADEMIC AGE OR SENIORITY WHEN CONTROLLING FOR GENDER AND ORIGIN?
For at ændre ”Enhedens navn” og ”Sted og dato”:
Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod
FINDINGS: RECOMENDING PUBLICATION AND CITATION INDICATORS
CONCLUSIONS (SO FAR)
1. Indicator values are effected to varying degrees, dependent on discipline, by gender, origin, seniority and academic age (database & version).
2. Academic age is dependent on how it is calculated. Here is highly dependent on database coverage. Seniority is more understandable. But does it make sense?
3. Don’t have to wrap data in algorithims. More informative to summarize patterns between indicators.
4. It is important to report the database and which version of the database was used to collect the data.
For at ændre ”Enhedens navn” og ”Sted og dato”:
Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod
CONCLUSIONS (SO FAR)
5.Variance in amount of publications between scholars differs from discipline to discipline. Clear difference in amount of publications and citations.
6.The indicators are estimates. Report confidence intervals and range to contextualize the values.
7.Strong correlation between indicators. Central and isolated indicators need further investigation allowing for confounders.
8.No one indicator can stand alone. Work continues to identify indicators suitable for discipline and seniority.