Measuring and ranking scienti c productiondimetic.dime-eu.org/dimetic_files/dimetic_ranking.pdf · Measuring scienti c production Ranking Bibliometrics A eld born with the its object

Measuring scientific production Ranking

Measuring and ranking scientific production

Nicolas Carayol

Observatoire des Sciences et Techniques, Paris

Dimetic, July 2010, Pecs 2010


Bibliometrics

A field born with the its object : scientific production.

A distributional approach : with Bradford, Zipf and Lotka1926-> 35

RK Merton influence : measuring production and credit.

D. J. de Solla Price

E. Garfield

A connection with economics through Stigler’s influence


Bibliometrics





E. Garfield



Bibliometrics





E. Garfield



Bibliometrics





E. Garfield



Bibliometrics





E. Garfield



Bibliometrics





E. Garfield



Bibliometrics

Measuring scientific production

How to rank ?


Bibliometrics

Measuring scientific production

How to rank ?


Sections :

1 Measuring scientific productionVolumeImpactInfluenceMeasures for both quality and quantity

2 RankingThe axiomatic approachThe extended stochastic dominance approach

Core background theoryIllustrative Results


Bibliometrics

Define the agent ?

Q : quantity

q : quality


Bibliometrics

Define the agent ?

Q : quantity

q : quality


Bibliometrics

Define the agent ?

Q : quantity

q : quality


Volume

The basic data

The structured set of i ’s publications is given by

Si := (s1, s2, ..., s′a, ..., sa, ..., sni ),

with ni the number of articles to be associated to agent i .


Volume

Ground zero

Total counts :

Ti ,a = # {i mentioned in a}

Total counts by domain :

T ki ,a = # {i mentioned in a} × 1 {j(a) associated to k}

with j(a) is the journal in which a were published.


Volume

Ground zero

Total counts :

Ti ,a = # {i mentioned in a}

Total counts by domain :

T ki ,a = # {i mentioned in a} × 1 {j(a) associated to k}

with j(a) is the journal in which a were published.


Volume

Accounting for coauthorship

An article a, referencing at least one address associated toinstitution i , brings a score of :

pki ,a =

# {i mentioned in a}# { j | j mentioned in a}

× 1 {j(a) associated to k} ,

where 1 {.} is the indicator function and # {.} denotes thecardinal of the set defined into brackets.


Volume

Allocate over disciplines

An article a, referencing at least one address associated toinstitution i , brings a score of :

pki ,a =

# {i mentioned in a}# { j | j mentioned in a}

× 1 {j(a) associated to k}# {k | j(a) associated to k}

,

where 1 {.} is the indicator function and # {.} denotes thecardinal of the set defined into brackets. j(a) is the journal inwhich a were published.


Volume

Other basic corrections

# pages

...


Volume

Other basic corrections

# pages

...


Volume

Authors’ rank

(Assimakis & Adam, scientometrics forthcoming)The linearcontribution index :

lpia = c − δria,

where ria the rank of author i among the na authors of articlea and with c a normalization parameter c = na+1

2 δ + 1na

sothat authros contribution sum to one.

The geometriccontribution index

gpia = κ · λ−na+ri ,

with 0 < λ < 1, and κ he normalization parameter.


Volume

Authors’ rank

(Assimakis & Adam, scientometrics forthcoming)The linearcontribution index :

lpia = c − δria,

where ria the rank of author i among the na authors of articlea and with c a normalization parameter c = na+1

2 δ + 1na

sothat authros contribution sum to one.The geometriccontribution index

gpia = κ · λ−na+ri ,

with 0 < λ < 1, and κ he normalization parameter.


Impact

Quality is critical

Rely on peer review

Rely on citation data


Impact

Quality is critical

Rely on peer review

Rely on citation data


Impact

Counting citations

The Number of direct citations received :

sa = # { j | tj ∈ w(a) and j cites a} ,

with tj the year of publication of j , and w(a) the citationwindow of article a.


Impact

The impact Factor

Impact Factor of the journal :

s ′a =# { j | tj ∈ w(a) and j cites i ∈ j(a)}

# { i | i ∈ j(a)}.


Impact

Correction across fields

Relative Impact Factor of the journal :

s ′′a =s ′a

〈s ′a〉subfield of a

.


Influence

Prestige centrality

Narin & Pinski 1976

Leibowitz and Palmer 1986

Page rank (Page & al, 1998)

Katz → prestige


Influence

Prestige centrality

Narin & Pinski 1976



Katz → prestige


Influence

Prestige centrality

Narin & Pinski 1976



Katz → prestige


Influence

Prestige centrality

Narin & Pinski 1976



Katz → prestige


Influence

A unified writing of centrality

A = (ai ) with ai the number of articles published in/of i .

C = (cij) with cij the number of citations i received from j (orthe number of references made in j to i).

The counting method of Bush, Hamelman & Staaf (1974) :

φi = λ

∑j cij/ai∑

k

∑j ckj/ak

.

value is proportional to the average number of citations madeto its articles

The counting modified method :

φi = λ

∑j cij/ai × φj∑

k

∑j ckj/ak × φj

.



Influence





φi = λ

∑j cij/ai∑

k

∑j ckj/ak

.



φi = λ


k

∑j ckj/ak × φj

.



Influence





φi = λ

∑j cij/ai∑

k

∑j ckj/ak

.



φi = λ


k

∑j ckj/ak × φj

.



Influence





φi = λ

∑j cij/ai∑

k

∑j ckj/ak

.



φi = λ


k

∑j ckj/ak × φj

.



Influence


The Leibowitz and Palmer (1986) method :

φi = λ

∑j 6=i cij/ai∑

k

∑j 6=k ckj/ak

,

the value is proportional to the average number of citationsmade to its articles.

In matrix form, this becomes :

φ = λA−1Cφ

‖A−1Cφ‖

with ‖X‖ =∑

k ‖xk‖ the 1 norm of X .


Influence


The Leibowitz and Palmer (1986) method :

φi = λ

∑j 6=i cij/ai∑

k

∑j 6=k ckj/ak

,

the value is proportional to the average number of citationsmade to its articles.


φ = λA−1Cφ

‖A−1Cφ‖

with ‖X‖ =∑

k ‖xk‖ the 1 norm of X .


Influence


The Narin and Pinski (1976) method :

φi = λ∑j 6=i

cij/ai

cj/ajφj ,

the value is a weighted average of the centrality of thejournals citing it.


φ = λA−1CD−1C Aφ

with DC the diagonal matrix having the sum of the of thereferences in the diagonal (ci ). The matrix CD−1

C is thenormalized matrix of C and is a stochastic matrix (sums toone in each column).


Influence


The Narin and Pinski (1976) method :

φi = λ∑j 6=i

cij/ai

cj/ajφj ,

the value is a weighted average of the centrality of thejournals citing it.


φ = λA−1CD−1C Aφ

with DC the diagonal matrix having the sum of the of thereferences in the diagonal (ci ). The matrix CD−1

C is thenormalized matrix of C and is a stochastic matrix (sums toone in each column).


Influence

Prestige centrality (reminder)

Prestige depends of the average prestige of its neighbors asystem of equations :

pi (g) =∑j 6=i

gij

ηj (g)pj (g) .


Measures for both quality and quantity

Quantity and quality

The number of articles published ni , the average 〈s〉i and thetotal number 〈s〉i ni of citations received.

The first synthetic measure of both quality and quantity wasproposed by (Lindsay, 1978) :

Qi = 〈s〉i × (〈s〉i ni )1/2 = 〈s〉3/2

i n1/2i .



Quantity and quality

The number of articles published ni , the average 〈s〉i and thetotal number 〈s〉i ni of citations received.

The first synthetic measure of both quality and quantity wasproposed by (Lindsay, 1978) :

Qi = 〈s〉i × (〈s〉i ni )1/2 = 〈s〉3/2

i n1/2i .



The h-index

Each article published is denoted by an index a ∈ N\ {0} andis characterized by an associated impact measure (herecitations) sa ∈ R+ (here sa ∈ N).

The structured set of agent i ’s publications isSi := (s1, s2, ..., s

′a, ..., sa, ..., sni ), a vector assumed to be

(decreasingly) ordered according to the number of citationsreceived : a > a′ → s ′a ≤ sa.



The h-index

Each article published is denoted by an index a ∈ N\ {0} andis characterized by an associated impact measure (herecitations) sa ∈ R+ (here sa ∈ N).

The structured set of agent i ’s publications isSi := (s1, s2, ..., s

′a, ..., sa, ..., sni ), a vector assumed to be

(decreasingly) ordered according to the number of citationsreceived : a > a′ → s ′a ≤ sa.



The h-index

Hirsch (2005) proposes the h-index as a synthetic a uniquemeasure for both the # of citations and the # ofpublications. More, it a basic measure for the wholepublication/citation sequence.

The h-index of agent i is a measure computed from her/hisstructured set Si as follows :

h := maxa

(a ≤ sa).



The h-index

Hirsch (2005) proposes the h-index as a synthetic a uniquemeasure for both the # of citations and the # ofpublications. More, it a basic measure for the wholepublication/citation sequence.

The h-index of agent i is a measure computed from her/hisstructured set Si as follows :

h := maxa

(a ≤ sa).



The h-index

Example : ni = 7, Si = (8, 6, 5, 4, 2, 2, 1).

S

rank 1 2 3 4 5 6 7

8

S

rank 1 2 3 4 5 6 7

8

h=4

sh=



The h-index

Example : ni = 7, Si = (8, 6, 5, 4, 2, 2, 1).

S

rank 1 2 3 4 5 6 7

8

S

rank 1 2 3 4 5 6 7

8

h=4

sh=



The h-index

Example : ni = 7, Si = (8, 6, 5, 4, 2, 2, 1).

S

rank 1 2 3 4 5 6 7

8

S

rank 1 2 3 4 5 6 7

8

h=4

sh=



The h-index and beyond

Main shortcoming of the h-index : citations have no impact onthe index outside the h-core

The derivatives :

the g-index (Egghe, 2006) : g := maxa a2 ≤∑

j=1...a sj

the r-index (Jin et al., 2007)the tapered h-index (Anderson et al., 2008)the w-index (Woeringer, 2008)...





The derivatives :


j=1...a sj






The derivatives :


j=1...a sj






The derivatives :


j=1...a sj

the r-index (Jin et al., 2007)

the tapered h-index (Anderson et al., 2008)the w-index (Woeringer, 2008)...





The derivatives :


j=1...a sj

the r-index (Jin et al., 2007)the tapered h-index (Anderson et al., 2008)

the w-index (Woeringer, 2008)...





The derivatives :


j=1...a sj

the r-index (Jin et al., 2007)the tapered h-index (Anderson et al., 2008)the w-index (Woeringer, 2008)

...





The derivatives :


j=1...a sj





Hirsch self defense :

He argues his aim is precisely to NOT consider both i) lowimpact papers (is intended to capture a maintained flow ofinfluential contributions) AND ii) the high volume of citationsthat some papers sometime receive (could unduly grant someco-authors of highly cited papers).Hirsch argues the h-index has a strong predictive power on thearrival of future citations (Hirsch, 2007).

Main implicit advantage : easy to compute and thereforerobust to errors.

From a theoretical point of view, it is not a defense of theimplicit value judgements it incorporates.





He argues his aim is precisely to NOT consider both i) lowimpact papers (is intended to capture a maintained flow ofinfluential contributions) AND ii) the high volume of citationsthat some papers sometime receive (could unduly grant someco-authors of highly cited papers).

Hirsch argues the h-index has a strong predictive power on thearrival of future citations (Hirsch, 2007).

























Sections :

1 Measuring scientific productionVolumeImpactInfluenceMeasures for both quality and quantity

2 RankingThe axiomatic approachThe extended stochastic dominance approach

Core background theoryIllustrative Results


Ranking : what for ?

Any ranking incorporate value judgements.

An axiomatic approach which renders explicit the valuejudgements incorporated in any ranking based on some index(as Arrow 1955).

A normative approach directly applied to the structured set ofpublications (as Rothshild and Stiglitz 1969, Atkinson, 1970).












The axiomatic approach


→ An axiomatic approach which renders explicit the valuejudgements incorporated in measures :

Palacios Huerta & Volij 2004 (econometrica) →axiomatization of Pinski & Narin influence measure and someothers

Woeringer 2008 (math soc science) & Marchant 2009(scientometrics) → axiomatization of h-index and some othermeasures




→ An axiomatic approach which renders explicit the valuejudgements incorporated in measures :

Palacios Huerta & Volij 2004 (econometrica) →axiomatization of Pinski & Narin influence measure and someothers

Woeringer 2008 (math soc science) & Marchant 2009(scientometrics) → axiomatization of h-index and some othermeasures




Palacios Huerta & Volij 2004 :

Let J be the set of ranked agents

Axioms :

1 Invariance with respect to reference intensity : the ranking isnot modified by any modification of the reference intensity ofagents (each one has a vote one) : for any diagonal matrix Λwith strictly positive diagonal entries λj , if C ′ = C Λ, theranking is unchanged.

2 Weak homogeneity : in a two agents ranking problem issueonly, for any two agents with identical number of references(ai = aj) : φi

φj=

cij

cji.






Axioms :



φj=

cij

cji.






Axioms :



φj=

cij

cji.






Axioms :



φj=

cij

cji.






Axioms :



φj=

cij

cji.




3 Weak consistency : if for any set of agents with identicalnumber of references (ai = aj ,∀i , j ∈ J) :

∀k ∈ J, φiφj

=φk

i

φkj

, ∀i , j ∈ J\ {k}, with φki the rank of i with the

modified citation matrix C k such thatckij = cij + ckj

cik∑h∈J\{k} chk

. (the relative valuations shall be

proportional of the relative influences without any otheragent, the influences of which are redistributed to the agentsit cites (proportionally).




4 Invariance to the splitting of agents : if for all rankingproblems, and any splitting of agent i∈ J into nj identicalagents (same profile of citation and references :

anj

j = aj/nj , cninj

ij = cij/(ninj)) : φiφj

=φ

nii

φnjj

, ∀ni , nj .




Theorem

The Narin and Pinski (1976) measure of centrality is the uniquemeasure that may rank any set of agents while satisfying the 4axioms.




Woeringer 2008 axiomatization of h − index and some othermeasures

Si := (s1, s2, ..., s′a, ..., sa, ..., sni )

S ∈ Ψ the set of all possible positive entry vectors ofdimension 1× n(n > 0).

An index is a function φ : Ψ→ N with φ(0, 0, ..., 0) = 0





Si := (s1, s2, ..., s′a, ..., sa, ..., sni )







Si := (s1, s2, ..., s′a, ..., sa, ..., sni )







Si := (s1, s2, ..., s′a, ..., sa, ..., sni )






Axioms

a Monotonicity : for all vectors S of size 1× n and S ′ of size m,such that m ≤ n and sk ≥ s ′k ,∀k ≤ m then φ(S) > φ(S ′).

b If the vector S of size 1× (n + 1) can be built from vector S ′

of size 1× n by simply adding a paper with φ(S) citations,then φ(S ′) ≤ φ(S).




Axioms







Axioms







Axioms

c If the vector S of size 1× n can be built from vector S ′ of size1× n by simply adding a citation to a paper (∃h ≤ n sts ′h = sh + 1 and ∀k ≤ n st k 6= h then s ′h = sh) thenφ(S) ≤ φ(S ′) + 1.

d If the vector S of size 1× (n + 1) can be built from vector S ′

of size 1× n by simply adding an article with φ(S) citations,and then increasing the number of citations of paper by atleast one (∀h ≤ n st sh ≥ s ′h) then φ(S) > φ(S ′).




Axioms







Axioms







Theorem

An index φ : Ψ→ N satisfies the four axioms a, b, c and d if andonly if it is the h-index.


The extended stochastic dominance approach

→ A normative approach directly applied to the structured set ofpublications.



→ A normative approach directly applied to the structured set ofpublications.



The main goal

There is a well known concept often used in economics :stochastic dominance.

Applied to the theory of choice under uncertainty (Stiglitz &Rothschild, 1970) and income distribution (Atkinson 1970).

But :

This is designed to compare distributions only (care only aboutquality and quality distribution).The assumptions associated to second order stochasticdominance (concavity) will not be systematically consistent.

→ A need an adaptation of the technique which would value bothimpact and volume of publication.



The main goal



But :





The main goal



But :





The main goal



But :

This is designed to compare distributions only (care only aboutquality and quality distribution).

The assumptions associated to second order stochasticdominance (concavity) will not be systematically consistent.




The main goal



But :





The main goal



But :





The main goal



But :





The main goal

→ we hereby introduce a theory and develop a methodologyto compare the scientific production of institutions.

1 Dominance relations are established.

2 Dominance relations are used to build dominance networks,rankings and reference classes.



The main goal






The main goal






General problem

→ in principle that theory could apply to the comparison ofthe outcome of a set agents for which :

1 The outcome is composite,

2 Each element can be described by a “quality” index,

3 The modeler is concerned by both quantity and quality,

4 The modeler does not know exactly the function whichtransforms the “quality” in valued outcome, and thusrefers to classes of functions.



General problem








General problem








General problem








General problem








General problem : Potential applictions

Departments of economics

Museums

Social clubs

Night clubs

...





Museums

Social clubs

Night clubs

...





Museums

Social clubs

Night clubs

...





Museums

Social clubs

Night clubs

...





Museums

Social clubs

Night clubs

...



Implicit to explicit valuation of the scientific output

Let f ki (s) :=

∑j=1,...,ni

1 {sj = s} be the publicationperformance of i with “impact” s in discipline k .

0 1 2 3 4 s

f(0)

f(1) f(2)

f(3)



The scientific output : articles and citations

Then the vector of structured publication outcome ofinstitution i in domain k is computed by summing over allarticles a and filtering out the article scores in the sum on theright side of previous equation. It gives :

f ki (s) =

∑j=1,...,ni

pki ,a × 1 {sj = s} .

Assume the exists some s such that ∀i , f ki (s) = 0 if s ≥ s.

The “value” (for the agent, the consumer, the holder, thesponsor...) of the whole publication production of institution iin domain k is assumed to be symmetric and additiveseparable given by :

V k (Si ) = ωk

∫ s

0v(s)f k

i (s) ds,

with ωk the discipline dependent normalization parameter.Computation of the “value” of the whole publicationproduction of institution i in domain k is conditional to thevaluation function v(s).





f ki (s) =

∑j=1,...,ni

pki ,a × 1 {sj = s} .



V k (Si ) = ωk

∫ s

0v(s)f k

i (s) ds,






f ki (s) =

∑j=1,...,ni

pki ,a × 1 {sj = s} .



V k (Si ) = ωk

∫ s

0v(s)f k

i (s) ds,

with ωk the discipline dependent normalization parameter.

Computation of the “value” of the whole publicationproduction of institution i in domain k is conditional to thevaluation function v(s).





f ki (s) =

∑j=1,...,ni

pki ,a × 1 {sj = s} .



V k (Si ) = ωk

∫ s

0v(s)f k

i (s) ds,




Implicit valuation : examples

0 1 2 3 4 T s

A

B C D

E F



Various assumptions

Assumption 1. v ≥ 0 (A1)

Assumption 2. v ′ ≥ 0 (A2)

Assumption 3. v ′′ ≥ 0 (A3)

Assumption 4. v ′′ ≤ 0 (A4)

According to your goals and/or beliefs, you can rely uponwell defined dominance relations that can be used to producerankings and references classes.



Various assumptions








Various assumptions








Various assumptions








Various assumptions








Various assumptions








Standard measures using publications and citations

The number of articles published : v(s) = θ > 0

The total number of citations : v(s) = s

The average number of citations per article :vi (s) = s/

∫fi (s)ds

The h-index respects (A1) & (A2) but violates (A3) &(A4).







∫fi (s)ds








∫fi (s)ds








∫fi (s)ds




Strong dominance

Definition

The scientific production of institution i in field k stronglydominates the one of institution j , noted i Ik j , if, for anypositive function v (·) (A1),

∫ s0 v(s)f k

i (s) ds ≥∫ s

0 v(s)f kj (s) ds.

Lemma

i Ik j if and only if ∀x ∈ [0,∞[ , f ki (x)− f k

j (x) ≥ 0.



Strong dominance

Definition

The scientific production of institution i in field k stronglydominates the one of institution j , noted i Ik j , if, for anypositive function v (·) (A1),

∫ s0 v(s)f k

i (s) ds ≥∫ s

0 v(s)f kj (s) ds.

Lemma

i Ik j if and only if ∀x ∈ [0,∞[ , f ki (x)− f k

j (x) ≥ 0.



Implicit valuation of strong dominance : positive functions

0 1 2 3 4 T s

A

B C D

E F



Strong dominance



But often :



Dominance

Definition

The scientific production of institution i in field k dominates theone of institution j , noted i Bk j , if, for any positive and nondecreasing function v (·) (A1 & A2),∫ s

0 v(s)f ki (s) ds ≥

∫ s0 v(s)f k

j (s) ds.

Lemma

i Bk j , if and only if ∀x ∈ [0, s[ ,∫ sx

[f ki (s)− f k

j (s)]

ds ≥ 0.



Dominance

Definition

The scientific production of institution i in field k dominates theone of institution j , noted i Bk j , if, for any positive and nondecreasing function v (·) (A1 & A2),∫ s

0 v(s)f ki (s) ds ≥

∫ s0 v(s)f k

j (s) ds.

Lemma

i Bk j , if and only if ∀x ∈ [0, s[ ,∫ sx

[f ki (s)− f k

j (s)]

ds ≥ 0.



Implicit valuation of dominance : positive and increasingfunctions

0 1 2 3 4 T s

B C

E



Dominance



Dominance



Dominance



Dominance



Dominance



Weak dominance

Definition

The scientific production of institution i in field k weaklydominates the one of institution j , noted i Dk j , if, for anypositive, non decreasing function and weakly convex functionv (·) (A1, A2 & A3),

∫ s0 v(s)f k

i (s) ds ≥∫ s

0 v(s)f kj (s) ds.

Lemma

i Dk j , if and only if ∀x ∈ [0, s[ ,∫ sx s[f ki (s)− f k

j (s)]

ds ≥ 0.



Weak dominance

Definition

The scientific production of institution i in field k weaklydominates the one of institution j , noted i Dk j , if, for anypositive, non decreasing function and weakly convex functionv (·) (A1, A2 & A3),

∫ s0 v(s)f k

i (s) ds ≥∫ s

0 v(s)f kj (s) ds.

Lemma

i Dk j , if and only if ∀x ∈ [0, s[ ,∫ sx s[f ki (s)− f k

j (s)]

ds ≥ 0.



Implicit valuation of weak dominance : positive, increasingand weakly convex functions

0 1 2 3 4 T s

B

F



Weak Dominance

fik (s)

fjk (s)



Weak Dominance

fik (s)

fjk (s)



Weak Dominance

fik (s)

fjk (s)



Weak Dominance



Weak concave dominance

Definition

The scientific production of institution i in field k weaklyconcavely dominates the one of institution j , noted iBk j , if, forany positive, non decreasing function and weakly concavefunction v (·) (A1, A2 & A4),

∫ s0 v(s)f k

i (s) ds ≥∫ s

0 v(s)f kj (s) ds.

Lemma

iBk j if and only if ∀x ∈ [0, s[ ,∫ x

0 s[f ki (s)− f k

j (s)]

ds ≥ 0.



Weak concave dominance

Definition

The scientific production of institution i in field k weaklyconcavely dominates the one of institution j , noted iBk j , if, forany positive, non decreasing function and weakly concavefunction v (·) (A1, A2 & A4),

∫ s0 v(s)f k

i (s) ds ≥∫ s

0 v(s)f kj (s) ds.

Lemma

iBk j if and only if ∀x ∈ [0, s[ ,∫ x

0 s[f ki (s)− f k

j (s)]

ds ≥ 0.



Upward dominance

Definition

Let sφk be the smallest visibility a paper may exhibit among the φ%most visible papers in the field k (in the world).

Strong dominance at order φ : i Iφk j .

Dominance at order φ : i Bφk j .

Weak dominance at order φ : i Dφk j .

Upward dominance relations are obviously generalization ofthe dominance relations which are equivalent to upwarddominance relations at order 1.



Upward dominance

Definition








Upward dominance

Definition








Upward dominance

Definition








Upward dominance

Definition








Upward Dominance

fik (s)

fjk (s)

fik (s)

fjk (s)

sφ



Upward Dominance

fik (s)

fjk (s)

fik (s)

fjk (s)

sφ



The Interdisciplinary scientific output

Θi =(Θk

i

)k=1,...|K |, the vector of publication informed by

citations performance in all disciplines/domains k ∈ K

The value of the publication performance of institution i asthe weighted and valued sum of the articles of thatinstitution :

V (Θi ) =∑k

ωk

∫ s

0v(s)f k

i (s) ds.

We propose to set ωk as the inverse of the average number ofcitations made by articles in field k .

The value function is not symmetric anymore.




Θi =(Θk

i




V (Θi ) =∑k

ωk

∫ s

0v(s)f k

i (s) ds.






Θi =(Θk

i




V (Θi ) =∑k

ωk

∫ s

0v(s)f k

i (s) ds.






Θi =(Θk

i




V (Θi ) =∑k

ωk

∫ s

0v(s)f k

i (s) ds.






Definition

The scientific production of institution i inter-disciplinary (a)strongly dominates

(i Iφ j

), (b) dominates

(i Bφ j

)or (c) weakly

dominates(i Dφ j

)at order φ ∈ ]0, 1] the one of institution j , if∫ s

sφk

v(s)f ki (s) ds ≥

∑k

∫ ssφk

v(s)f kj (s) ds for any (a) positive

function (b) positive and non-decreasing (c) positive andnon-decreasing and weakly convex function v (·) .




Lemma

The three following statements hold :i) i Iφ j iff, ∀x = (xk)k=1...|K | st

∀k , xk ∈[sφk , s

[,∑

k ωk

[f ki (xk)− f k

j (xk)]

ds ≥ 0;

ii) i Bφ j iff, ∀x = (xk)k=1...|K | st

∀k, xk ∈[sφk , s

[,∑

k ωk

∫ sxk

[f ki (s)− f k

j (s)]

ds ≥ 0;

iii) i Dφ j iff ∀x = (xk)k=1...|K | st

∀k, xk ∈[sφk , s

[,∑

k ωk

∫ sxk

s[f ki (s)− f k

j (s)]

ds ≥ 0.



Properties of dominance relations

Lemma

All the dominance relations introduced (upward strong dominance,upward dominance and upward weak dominance) are transitive : ifi � j and j � h, then i � h, where � potentially accounts forIφ

k ,Bφk or Dφ

k , with ∀φ ∈ ]0, 1].



Relations between dominance relations

Definition

A dominance relation � is stronger than dominance relation �′,noted ��′, if, ∀i , j , i � j implies i �′ j .

Theorem

∀φ, φ′ ∈ [0, 1] such that φ ≥ φ′, then

- Iφk�Bφ

k�Dφk ,

- Iφk�Iφ′

k , Bφk�Bφ′

k and Dφk�Dφ′

k ,- Iφ�Bφ�Dφ, and- Iφ�Iφ′ , Bφ�Bφ′ and Dφ�Dφ′ .

Thus the weaker the dominance relation, the more complete,that is the more dominance relations it is possible to establisha given set of institutions.




Definition


Theorem


- Iφk�Bφ

k�Dφk ,

- Iφk�Iφ′

k , Bφk�Bφ′

k and Dφk�Dφ′






Definition


Theorem


- Iφk�Bφ

k�Dφk ,

- Iφk�Iφ′

k , Bφk�Bφ′

k and Dφk�Dφ′





Publication data

OST in-house ISI-WOS database recording of publications andcitations

Large disciplines : 1/ Fund. bio., 2/ Medicine, 3/ Ap.bio/ecol., 4/ Chem., 5/ Physics, 6/ Science univ., 7/ Eng.sciences, 8/ Maths (Humanities and social sciences areexcluded.

Multidisciplinary sciences papers published in PNAS, Science& Nature have been allocated to their reference discipline fordisciplinary comparisons.

3-years citations moving window for all indexes.

Linearization of the distribution.



Publication data








Publication data








Publication data








Publication data








Data set

French universities :

129 institutionsThe institutions checked the validity of signing patterns.

US research universities :

112 best ranked in the ARWU Shanghaı ranking (30% of PhDgranting Univ.)



Data set


129 institutions

The institutions checked the validity of signing patterns.





Data set







Data set







Data set







Networks of dominance relations

Let us consider �, which could be any one of the dominancerelations examined above (Iφ

k ,Bφk or Dφ

k , with ∀φ ∈ ]0, 1]).

Let’s build the dominance directed network ~g associated todominance relation � and the institutions set I by writing adirect link from institution i to institution j if i � j .

In this network, transitive triplets are uninformative since thetransitivity property holds. Therefore, let’s build the network~g ′ derived from ~g by deleting all such triplets : ∀i , j , k ∈ I , ifij , ik, jk ∈ ~g and kj /∈ ~g then ik /∈ ~g ′.





k ,Bφk or Dφ

k , with ∀φ ∈ ]0, 1]).


In this network, transitive triplets are uninformative since thetransitivity property holds. Therefore, let’s build the network~g ′ derived from ~g by deleting all such triplets : ∀i , j , k ∈ I , ifij , ik, jk ∈ ~g and kj /∈ ~g then ik /∈ ~g ′.





k ,Bφk or Dφ

k , with ∀φ ∈ ]0, 1]).


In this network, transitive triplets are uninformative since thetransitivity property holds. Therefore, let’s build the network~g ′ derived from ~g by deleting all such triplets : ∀i , j , k ∈ I , ifij , ik , jk ∈ ~g and kj /∈ ~g then ik /∈ ~g ′.



Dominance network : an example



Reference classes

Definition

∀k ∈ I , j ∈ c�i (⊆ I ) the reference class of institution i associatedto dominance relation � if i � j and j � i or if i � j and j � i

Reading example : j ∈ cBki means it is not always possible to

rank strictly and in a unique manner the scientific productionof i and j in domain k relying on any implicit valuationfunction of articles according to their impact which would beboth positive and increasing with impact.



Reference classes

Definition

∀k ∈ I , j ∈ c�i (⊆ I ) the reference class of institution i associatedto dominance relation � if i � j and j � i or if i � j and j � i

Reading example : j ∈ cBki means it is not always possible to

rank strictly and in a unique manner the scientific productionof i and j in domain k relying on any implicit valuationfunction of articles according to their impact which would beboth positive and increasing with impact.



Dominance Networks : Top French universities in FundBio, φ = 1 - citations

Paris 6

Paris 11

Strasbourg 1

Paris 5 Montpellier 2

Aix Marseille 2

Paris 7 Lyon 1

Grenoble 1



Dominance Networks : Top French universities in FundBio, φ = .1 - citations

Paris 6

Paris 11

Strasbourg 1

Paris 5 Montpelier 2 Aix Marseille 2

Paris 7

Lyon 1 Grenoble 1



Dominance Networks : Top French universities in FundBio, φ = .1 - impact factor

Paris 6

Paris 11

Strasbourg 1

Paris 5

Montpellier 2

Aix Marseille 2

Paris 7

Lyon 1

Grenoble 1



Complete dominance relations and ranking

Definition

A dominance relation � is said to be I -complete if ∀i , j ∈ I , i � jor j � i .

Definition

A (complete) dominance ranking R�I can be constructed overinstitutions set I on the basis of dominance relation � if and onlyif � is an I -complete dominance relation.



Complete dominance relations and ranking

Definition

A dominance relation � is said to be I -complete if ∀i , j ∈ I , i � jor j � i .

Definition

A (complete) dominance ranking R�I can be constructed overinstitutions set I on the basis of dominance relation � if and onlyif � is an I -complete dominance relation.



Complete dominance ranking

Definition

For each type of dominance (strong dominance, dominance, weakdominance), the smallest and strictly positive value of φ for whichthe corresponding dominance relations are I -complete are calledmax-I -complete dominance relations.

1 2 3 4 5 6 7 8

I .009 .004 .005 .022 .016 .019 .002 .009

B .009 .004 .008 .024 .016 .026 .002 .009

D .009 .004 .008 .024 .052 .026 .007 .009

The largest φ such that such dominance relation is I -complete overthe set of 129 French higher Education and research institutions




Definition


1 2 3 4 5 6 7 8

I .009 .004 .005 .022 .016 .019 .002 .009

B .009 .004 .008 .024 .016 .026 .002 .009

D .009 .004 .008 .024 .052 .026 .007 .009





Definition


1 2 3 4 5 6 7 8

I .009 .004 .005 .022 .016 .019 .002 .009

B .009 .004 .008 .024 .016 .026 .002 .009

D .009 .004 .008 .024 .052 .026 .007 .009




Pseudo dominance ranking

Definition

For any dominance relation and any set of institutions I , a pseudodominance ranking can be established on the basis of the score ofeach institution i , si = # {j |ij ∈ ~g }, the number of dominancerelations which emanate from institution i .



Pseudo dominance ranking : : US universities

Fund Biology I1 B1 D1

citations ri si ri si ri si

Harvard 1 111 1 111 1 111

John Hopkins 3 95 2 106 4 108

UCSF 7 91 3 105 3 109

Pennsylvania 3 95 4 103 8 104

UCLA 2 96 4 103 9 103

UCSD 5 92 6 100 6 105

Yale 11 87 6 100 5 107

Stanford 15 81 6 100 2 110

UW Seattle 5 92 9 99 12 99

Columbia 11 87 10 98 6 105



Conclusion

Combinations of influence, quality and quality.

Ranking can also be good way of doing economics (and notjust playing a childish game).



Conclusion

Combinations of influence, quality and quality.

Ranking can also be good way of doing economics (and notjust playing a childish game).

Documents

Measuring and ranking scienti c productiondimetic.dime-eu.org/dimetic_files/dimetic_ranking.pdf · Measuring scienti c production Ranking Bibliometrics A eld born with the its object