Measuring and ranking scienti c productiondimetic.dime-eu.org/dimetic_files/dimetic_ranking.pdf ·...

Measuring scientific production Ranking

Measuring and ranking scientific production

Nicolas Carayol

Observatoire des Sciences et Techniques, Paris

Dimetic, July 2010, Pecs 2010

Bibliometrics

A field born with the its object : scientific production.

A distributional approach : with Bradford, Zipf and Lotka1926-> 35

RK Merton influence : measuring production and credit.

D. J. de Solla Price

E. Garfield

A connection with economics through Stigler’s influence

Bibliometrics

E. Garfield

Bibliometrics

E. Garfield

Bibliometrics

E. Garfield

Bibliometrics

E. Garfield

Bibliometrics

E. Garfield

Bibliometrics

Measuring scientific production

How to rank ?

Bibliometrics

Measuring scientific production

How to rank ?

Sections :

1 Measuring scientific productionVolumeImpactInfluenceMeasures for both quality and quantity

2 RankingThe axiomatic approachThe extended stochastic dominance approach

Core background theoryIllustrative Results

Bibliometrics

Define the agent ?

Q : quantity

q : quality

Bibliometrics

Define the agent ?

Q : quantity

q : quality

Bibliometrics

Define the agent ?

Q : quantity

q : quality

Volume

The basic data

The structured set of i ’s publications is given by

Si := (s1, s2, ..., s′a, ..., sa, ..., sni ),

with ni the number of articles to be associated to agent i .

Volume

Ground zero

Total counts :

Ti ,a = # {i mentioned in a}

Total counts by domain :

T ki ,a = # {i mentioned in a} × 1 {j(a) associated to k}

with j(a) is the journal in which a were published.

Volume

Ground zero

Total counts :

Ti ,a = # {i mentioned in a}

Total counts by domain :

T ki ,a = # {i mentioned in a} × 1 {j(a) associated to k}

with j(a) is the journal in which a were published.

Volume

Accounting for coauthorship

An article a, referencing at least one address associated toinstitution i , brings a score of :

pki ,a =

# {i mentioned in a}# { j | j mentioned in a}

× 1 {j(a) associated to k} ,

where 1 {.} is the indicator function and # {.} denotes thecardinal of the set defined into brackets.

Volume

Allocate over disciplines

An article a, referencing at least one address associated toinstitution i , brings a score of :

pki ,a =

# {i mentioned in a}# { j | j mentioned in a}

× 1 {j(a) associated to k}# {k | j(a) associated to k}

where 1 {.} is the indicator function and # {.} denotes thecardinal of the set defined into brackets. j(a) is the journal inwhich a were published.

Volume

Other basic corrections

# pages

Volume

Other basic corrections

# pages

Volume

Authors’ rank

(Assimakis & Adam, scientometrics forthcoming)The linearcontribution index :

lpia = c − δria,

where ria the rank of author i among the na authors of articlea and with c a normalization parameter c = na+1

2 δ + 1na

sothat authros contribution sum to one.

The geometriccontribution index

gpia = κ · λ−na+ri ,

with 0 < λ < 1, and κ he normalization parameter.

Volume

Authors’ rank

(Assimakis & Adam, scientometrics forthcoming)The linearcontribution index :

lpia = c − δria,

where ria the rank of author i among the na authors of articlea and with c a normalization parameter c = na+1

2 δ + 1na

sothat authros contribution sum to one.The geometriccontribution index

gpia = κ · λ−na+ri ,

with 0 < λ < 1, and κ he normalization parameter.

Impact

Quality is critical

Rely on peer review

Rely on citation data

Impact

Quality is critical

Rely on peer review

Rely on citation data

Impact

Counting citations

The Number of direct citations received :

sa = # { j | tj ∈ w(a) and j cites a} ,

with tj the year of publication of j , and w(a) the citationwindow of article a.

Impact

The impact Factor

Impact Factor of the journal :

s ′a =# { j | tj ∈ w(a) and j cites i ∈ j(a)}

# { i | i ∈ j(a)}.

Impact

Correction across fields

Relative Impact Factor of the journal :

s ′′a =s ′a

〈s ′a〉subfield of a

Influence

Prestige centrality

Narin & Pinski 1976

Leibowitz and Palmer 1986

Page rank (Page & al, 1998)

Katz → prestige

Influence

Prestige centrality

Narin & Pinski 1976

Katz → prestige

Influence

Prestige centrality

Narin & Pinski 1976

Katz → prestige

Influence

Prestige centrality

Narin & Pinski 1976

Katz → prestige

Influence

A unified writing of centrality

A = (ai ) with ai the number of articles published in/of i .

C = (cij) with cij the number of citations i received from j (orthe number of references made in j to i).

The counting method of Bush, Hamelman & Staaf (1974) :

φi = λ

∑j cij/ai∑

∑j ckj/ak

value is proportional to the average number of citations madeto its articles

The counting modified method :

φi = λ

∑j cij/ai × φj∑

∑j ckj/ak × φj

Influence

φi = λ

∑j cij/ai∑

∑j ckj/ak

φi = λ

∑j ckj/ak × φj

Influence

φi = λ

∑j cij/ai∑

∑j ckj/ak

φi = λ

∑j ckj/ak × φj

Influence

φi = λ

∑j cij/ai∑

∑j ckj/ak

φi = λ

∑j ckj/ak × φj

Influence

The Leibowitz and Palmer (1986) method :

φi = λ

∑j 6=i cij/ai∑

∑j 6=k ckj/ak

the value is proportional to the average number of citationsmade to its articles.

In matrix form, this becomes :

φ = λA−1Cφ

‖A−1Cφ‖

with ‖X‖ =∑

k ‖xk‖ the 1 norm of X .

Influence

The Leibowitz and Palmer (1986) method :

φi = λ

∑j 6=i cij/ai∑

∑j 6=k ckj/ak

the value is proportional to the average number of citationsmade to its articles.

φ = λA−1Cφ

‖A−1Cφ‖

with ‖X‖ =∑

k ‖xk‖ the 1 norm of X .

Influence

The Narin and Pinski (1976) method :

φi = λ∑j 6=i

cij/ai

cj/ajφj ,

the value is a weighted average of the centrality of thejournals citing it.

φ = λA−1CD−1C Aφ

with DC the diagonal matrix having the sum of the of thereferences in the diagonal (ci ). The matrix CD−1

C is thenormalized matrix of C and is a stochastic matrix (sums toone in each column).

Influence

The Narin and Pinski (1976) method :

φi = λ∑j 6=i

cij/ai

cj/ajφj ,

the value is a weighted average of the centrality of thejournals citing it.

φ = λA−1CD−1C Aφ

with DC the diagonal matrix having the sum of the of thereferences in the diagonal (ci ). The matrix CD−1

C is thenormalized matrix of C and is a stochastic matrix (sums toone in each column).

Influence

Prestige centrality (reminder)

Prestige depends of the average prestige of its neighbors asystem of equations :

pi (g) =∑j 6=i

ηj (g)pj (g) .

Measures for both quality and quantity

Quantity and quality

The number of articles published ni , the average 〈s〉i and thetotal number 〈s〉i ni of citations received.

The first synthetic measure of both quality and quantity wasproposed by (Lindsay, 1978) :

Qi = 〈s〉i × (〈s〉i ni )1/2 = 〈s〉3/2

i n1/2i .

Quantity and quality

The number of articles published ni , the average 〈s〉i and thetotal number 〈s〉i ni of citations received.

The first synthetic measure of both quality and quantity wasproposed by (Lindsay, 1978) :

Qi = 〈s〉i × (〈s〉i ni )1/2 = 〈s〉3/2

i n1/2i .

The h-index

Each article published is denoted by an index a ∈ N\ {0} andis characterized by an associated impact measure (herecitations) sa ∈ R+ (here sa ∈ N).

The structured set of agent i ’s publications isSi := (s1, s2, ..., s

′a, ..., sa, ..., sni ), a vector assumed to be

(decreasingly) ordered according to the number of citationsreceived : a > a′ → s ′a ≤ sa.

The h-index

Each article published is denoted by an index a ∈ N\ {0} andis characterized by an associated impact measure (herecitations) sa ∈ R+ (here sa ∈ N).

The structured set of agent i ’s publications isSi := (s1, s2, ..., s

′a, ..., sa, ..., sni ), a vector assumed to be

(decreasingly) ordered according to the number of citationsreceived : a > a′ → s ′a ≤ sa.

The h-index

Hirsch (2005) proposes the h-index as a synthetic a uniquemeasure for both the # of citations and the # ofpublications. More, it a basic measure for the wholepublication/citation sequence.

The h-index of agent i is a measure computed from her/hisstructured set Si as follows :

h := maxa

(a ≤ sa).

The h-index

Hirsch (2005) proposes the h-index as a synthetic a uniquemeasure for both the # of citations and the # ofpublications. More, it a basic measure for the wholepublication/citation sequence.

The h-index of agent i is a measure computed from her/hisstructured set Si as follows :

h := maxa

(a ≤ sa).

The h-index

Example : ni = 7, Si = (8, 6, 5, 4, 2, 2, 1).

rank 1 2 3 4 5 6 7

The h-index

Example : ni = 7, Si = (8, 6, 5, 4, 2, 2, 1).

rank 1 2 3 4 5 6 7

The h-index

Example : ni = 7, Si = (8, 6, 5, 4, 2, 2, 1).

rank 1 2 3 4 5 6 7

The h-index and beyond

Main shortcoming of the h-index : citations have no impact onthe index outside the h-core

The derivatives :

the g-index (Egghe, 2006) : g := maxa a2 ≤∑

j=1...a sj

the r-index (Jin et al., 2007)the tapered h-index (Anderson et al., 2008)the w-index (Woeringer, 2008)...

The derivatives :

j=1...a sj

The derivatives :

j=1...a sj

The derivatives :

j=1...a sj

the r-index (Jin et al., 2007)

the tapered h-index (Anderson et al., 2008)the w-index (Woeringer, 2008)...

The derivatives :

j=1...a sj

the r-index (Jin et al., 2007)the tapered h-index (Anderson et al., 2008)

the w-index (Woeringer, 2008)...

The derivatives :

j=1...a sj

the r-index (Jin et al., 2007)the tapered h-index (Anderson et al., 2008)the w-index (Woeringer, 2008)

The derivatives :

j=1...a sj

Hirsch self defense :

He argues his aim is precisely to NOT consider both i) lowimpact papers (is intended to capture a maintained flow ofinfluential contributions) AND ii) the high volume of citationsthat some papers sometime receive (could unduly grant someco-authors of highly cited papers).Hirsch argues the h-index has a strong predictive power on thearrival of future citations (Hirsch, 2007).

Main implicit advantage : easy to compute and thereforerobust to errors.

From a theoretical point of view, it is not a defense of theimplicit value judgements it incorporates.

He argues his aim is precisely to NOT consider both i) lowimpact papers (is intended to capture a maintained flow ofinfluential contributions) AND ii) the high volume of citationsthat some papers sometime receive (could unduly grant someco-authors of highly cited papers).

Hirsch argues the h-index has a strong predictive power on thearrival of future citations (Hirsch, 2007).

Sections :

1 Measuring scientific productionVolumeImpactInfluenceMeasures for both quality and quantity

2 RankingThe axiomatic approachThe extended stochastic dominance approach

Core background theoryIllustrative Results

Ranking : what for ?

Any ranking incorporate value judgements.

An axiomatic approach which renders explicit the valuejudgements incorporated in any ranking based on some index(as Arrow 1955).

A normative approach directly applied to the structured set ofpublications (as Rothshild and Stiglitz 1969, Atkinson, 1970).

The axiomatic approach

→ An axiomatic approach which renders explicit the valuejudgements incorporated in measures :

Palacios Huerta & Volij 2004 (econometrica) →axiomatization of Pinski & Narin influence measure and someothers

Woeringer 2008 (math soc science) & Marchant 2009(scientometrics) → axiomatization of h-index and some othermeasures

→ An axiomatic approach which renders explicit the valuejudgements incorporated in measures :

Palacios Huerta & Volij 2004 (econometrica) →axiomatization of Pinski & Narin influence measure and someothers

Woeringer 2008 (math soc science) & Marchant 2009(scientometrics) → axiomatization of h-index and some othermeasures

Palacios Huerta & Volij 2004 :

Let J be the set of ranked agents

Axioms :

1 Invariance with respect to reference intensity : the ranking isnot modified by any modification of the reference intensity ofagents (each one has a vote one) : for any diagonal matrix Λwith strictly positive diagonal entries λj , if C ′ = C Λ, theranking is unchanged.

2 Weak homogeneity : in a two agents ranking problem issueonly, for any two agents with identical number of references(ai = aj) : φi

Axioms :

3 Weak consistency : if for any set of agents with identicalnumber of references (ai = aj ,∀i , j ∈ J) :

∀k ∈ J, φiφj

, ∀i , j ∈ J\ {k}, with φki the rank of i with the

modified citation matrix C k such thatckij = cij + ckj

cik∑h∈J\{k} chk

. (the relative valuations shall be

proportional of the relative influences without any otheragent, the influences of which are redistributed to the agentsit cites (proportionally).

4 Invariance to the splitting of agents : if for all rankingproblems, and any splitting of agent i∈ J into nj identicalagents (same profile of citation and references :

j = aj/nj , cninj

ij = cij/(ninj)) : φiφj

, ∀ni , nj .

Theorem

The Narin and Pinski (1976) measure of centrality is the uniquemeasure that may rank any set of agents while satisfying the 4axioms.

Woeringer 2008 axiomatization of h − index and some othermeasures

Si := (s1, s2, ..., s′a, ..., sa, ..., sni )

S ∈ Ψ the set of all possible positive entry vectors ofdimension 1× n(n > 0).

An index is a function φ : Ψ→ N with φ(0, 0, ..., 0) = 0

Si := (s1, s2, ..., s′a, ..., sa, ..., sni )

Axioms

a Monotonicity : for all vectors S of size 1× n and S ′ of size m,such that m ≤ n and sk ≥ s ′k ,∀k ≤ m then φ(S) > φ(S ′).

b If the vector S of size 1× (n + 1) can be built from vector S ′

of size 1× n by simply adding a paper with φ(S) citations,then φ(S ′) ≤ φ(S).

Axioms

c If the vector S of size 1× n can be built from vector S ′ of size1× n by simply adding a citation to a paper (∃h ≤ n sts ′h = sh + 1 and ∀k ≤ n st k 6= h then s ′h = sh) thenφ(S) ≤ φ(S ′) + 1.

d If the vector S of size 1× (n + 1) can be built from vector S ′

of size 1× n by simply adding an article with φ(S) citations,and then increasing the number of citations of paper by atleast one (∀h ≤ n st sh ≥ s ′h) then φ(S) > φ(S ′).

Axioms

Theorem

An index φ : Ψ→ N satisfies the four axioms a, b, c and d if andonly if it is the h-index.

The extended stochastic dominance approach

→ A normative approach directly applied to the structured set ofpublications.

The main goal

There is a well known concept often used in economics :stochastic dominance.

Applied to the theory of choice under uncertainty (Stiglitz &Rothschild, 1970) and income distribution (Atkinson 1970).

This is designed to compare distributions only (care only aboutquality and quality distribution).The assumptions associated to second order stochasticdominance (concavity) will not be systematically consistent.

→ A need an adaptation of the technique which would value bothimpact and volume of publication.

The main goal

This is designed to compare distributions only (care only aboutquality and quality distribution).

The assumptions associated to second order stochasticdominance (concavity) will not be systematically consistent.

The main goal

→ we hereby introduce a theory and develop a methodologyto compare the scientific production of institutions.

1 Dominance relations are established.

2 Dominance relations are used to build dominance networks,rankings and reference classes.

The main goal

General problem

→ in principle that theory could apply to the comparison ofthe outcome of a set agents for which :

1 The outcome is composite,

2 Each element can be described by a “quality” index,

3 The modeler is concerned by both quantity and quality,

4 The modeler does not know exactly the function whichtransforms the “quality” in valued outcome, and thusrefers to classes of functions.

General problem

General problem : Potential applictions

Departments of economics

Museums

Social clubs

Night clubs

Museums

Social clubs

Night clubs

Museums

Social clubs

Night clubs

Museums

Social clubs

Night clubs

Museums

Social clubs

Night clubs

Implicit to explicit valuation of the scientific output

Let f ki (s) :=

∑j=1,...,ni

1 {sj = s} be the publicationperformance of i with “impact” s in discipline k .

0 1 2 3 4 s

f(1) f(2)

The scientific output : articles and citations

Then the vector of structured publication outcome ofinstitution i in domain k is computed by summing over allarticles a and filtering out the article scores in the sum on theright side of previous equation. It gives :

f ki (s) =

∑j=1,...,ni

pki ,a × 1 {sj = s} .

Assume the exists some s such that ∀i , f ki (s) = 0 if s ≥ s.

The “value” (for the agent, the consumer, the holder, thesponsor...) of the whole publication production of institution iin domain k is assumed to be symmetric and additiveseparable given by :

V k (Si ) = ωk

0v(s)f k

i (s) ds,

with ωk the discipline dependent normalization parameter.Computation of the “value” of the whole publicationproduction of institution i in domain k is conditional to thevaluation function v(s).

f ki (s) =

∑j=1,...,ni

pki ,a × 1 {sj = s} .

V k (Si ) = ωk

0v(s)f k

i (s) ds,

f ki (s) =

∑j=1,...,ni

pki ,a × 1 {sj = s} .

V k (Si ) = ωk

0v(s)f k

i (s) ds,

with ωk the discipline dependent normalization parameter.

Computation of the “value” of the whole publicationproduction of institution i in domain k is conditional to thevaluation function v(s).

f ki (s) =

∑j=1,...,ni

pki ,a × 1 {sj = s} .

V k (Si ) = ωk

0v(s)f k

i (s) ds,

Implicit valuation : examples

0 1 2 3 4 T s

Various assumptions

Assumption 1. v ≥ 0 (A1)

Assumption 2. v ′ ≥ 0 (A2)

Assumption 3. v ′′ ≥ 0 (A3)

Assumption 4. v ′′ ≤ 0 (A4)

According to your goals and/or beliefs, you can rely uponwell defined dominance relations that can be used to producerankings and references classes.

Various assumptions

Standard measures using publications and citations

The number of articles published : v(s) = θ > 0

The total number of citations : v(s) = s

The average number of citations per article :vi (s) = s/

∫fi (s)ds

The h-index respects (A1) & (A2) but violates (A3) &(A4).

∫fi (s)ds

Strong dominance

Definition

The scientific production of institution i in field k stronglydominates the one of institution j , noted i Ik j , if, for anypositive function v (·) (A1),

∫ s0 v(s)f k

i (s) ds ≥∫ s

0 v(s)f kj (s) ds.

i Ik j if and only if ∀x ∈ [0,∞[ , f ki (x)− f k

j (x) ≥ 0.

Strong dominance

Definition

The scientific production of institution i in field k stronglydominates the one of institution j , noted i Ik j , if, for anypositive function v (·) (A1),

∫ s0 v(s)f k

i (s) ds ≥∫ s

0 v(s)f kj (s) ds.

i Ik j if and only if ∀x ∈ [0,∞[ , f ki (x)− f k

j (x) ≥ 0.

Implicit valuation of strong dominance : positive functions

0 1 2 3 4 T s

Strong dominance

But often :

Dominance

Definition

The scientific production of institution i in field k dominates theone of institution j , noted i Bk j , if, for any positive and nondecreasing function v (·) (A1 & A2),∫ s

0 v(s)f ki (s) ds ≥

∫ s0 v(s)f k

j (s) ds.

i Bk j , if and only if ∀x ∈ [0, s[ ,∫ sx

[f ki (s)− f k

j (s)]

ds ≥ 0.

Dominance

Definition

The scientific production of institution i in field k dominates theone of institution j , noted i Bk j , if, for any positive and nondecreasing function v (·) (A1 & A2),∫ s

0 v(s)f ki (s) ds ≥

∫ s0 v(s)f k

j (s) ds.

i Bk j , if and only if ∀x ∈ [0, s[ ,∫ sx

[f ki (s)− f k

j (s)]

ds ≥ 0.

Implicit valuation of dominance : positive and increasingfunctions

0 1 2 3 4 T s

Dominance

Weak dominance

Definition

The scientific production of institution i in field k weaklydominates the one of institution j , noted i Dk j , if, for anypositive, non decreasing function and weakly convex functionv (·) (A1, A2 & A3),

∫ s0 v(s)f k

i (s) ds ≥∫ s

0 v(s)f kj (s) ds.

i Dk j , if and only if ∀x ∈ [0, s[ ,∫ sx s[f ki (s)− f k

j (s)]

ds ≥ 0.

Weak dominance

Definition

The scientific production of institution i in field k weaklydominates the one of institution j , noted i Dk j , if, for anypositive, non decreasing function and weakly convex functionv (·) (A1, A2 & A3),

∫ s0 v(s)f k

i (s) ds ≥∫ s

0 v(s)f kj (s) ds.

i Dk j , if and only if ∀x ∈ [0, s[ ,∫ sx s[f ki (s)− f k

j (s)]

ds ≥ 0.

Implicit valuation of weak dominance : positive, increasingand weakly convex functions

0 1 2 3 4 T s

Weak Dominance

fik (s)

fjk (s)

Weak Dominance

fik (s)

fjk (s)

Weak Dominance

fik (s)

fjk (s)

Weak Dominance

Weak concave dominance

Definition

The scientific production of institution i in field k weaklyconcavely dominates the one of institution j , noted iBk j , if, forany positive, non decreasing function and weakly concavefunction v (·) (A1, A2 & A4),

∫ s0 v(s)f k

i (s) ds ≥∫ s

0 v(s)f kj (s) ds.

iBk j if and only if ∀x ∈ [0, s[ ,∫ x

0 s[f ki (s)− f k

j (s)]

ds ≥ 0.

Weak concave dominance

Definition

The scientific production of institution i in field k weaklyconcavely dominates the one of institution j , noted iBk j , if, forany positive, non decreasing function and weakly concavefunction v (·) (A1, A2 & A4),

∫ s0 v(s)f k

i (s) ds ≥∫ s

0 v(s)f kj (s) ds.

iBk j if and only if ∀x ∈ [0, s[ ,∫ x

0 s[f ki (s)− f k

j (s)]

ds ≥ 0.

Upward dominance

Definition

Let sφk be the smallest visibility a paper may exhibit among the φ%most visible papers in the field k (in the world).

Strong dominance at order φ : i Iφk j .

Dominance at order φ : i Bφk j .

Weak dominance at order φ : i Dφk j .

Upward dominance relations are obviously generalization ofthe dominance relations which are equivalent to upwarddominance relations at order 1.

Upward dominance

Definition

Upward dominance

Definition

Upward dominance

Definition

Upward dominance

Definition

Upward Dominance

fik (s)

fjk (s)

fik (s)

fjk (s)

Upward Dominance

fik (s)

fjk (s)

fik (s)

fjk (s)

The Interdisciplinary scientific output

Θi =(Θk

)k=1,...|K |, the vector of publication informed by

citations performance in all disciplines/domains k ∈ K

The value of the publication performance of institution i asthe weighted and valued sum of the articles of thatinstitution :

V (Θi ) =∑k

0v(s)f k

i (s) ds.

We propose to set ωk as the inverse of the average number ofcitations made by articles in field k .

The value function is not symmetric anymore.

Θi =(Θk

V (Θi ) =∑k

0v(s)f k

i (s) ds.

Θi =(Θk

V (Θi ) =∑k

0v(s)f k

i (s) ds.

Θi =(Θk

V (Θi ) =∑k

0v(s)f k

i (s) ds.

Definition

The scientific production of institution i inter-disciplinary (a)strongly dominates

(i Iφ j

), (b) dominates

(i Bφ j

)or (c) weakly

dominates(i Dφ j

)at order φ ∈ ]0, 1] the one of institution j , if∫ s

v(s)f ki (s) ds ≥

∫ ssφk

v(s)f kj (s) ds for any (a) positive

function (b) positive and non-decreasing (c) positive andnon-decreasing and weakly convex function v (·) .

The three following statements hold :i) i Iφ j iff, ∀x = (xk)k=1...|K | st

∀k , xk ∈[sφk , s

[f ki (xk)− f k

j (xk)]

ds ≥ 0;

ii) i Bφ j iff, ∀x = (xk)k=1...|K | st

∀k, xk ∈[sφk , s

∫ sxk

[f ki (s)− f k

j (s)]

ds ≥ 0;

iii) i Dφ j iff ∀x = (xk)k=1...|K | st

∀k, xk ∈[sφk , s

∫ sxk

s[f ki (s)− f k

j (s)]

ds ≥ 0.

Properties of dominance relations

All the dominance relations introduced (upward strong dominance,upward dominance and upward weak dominance) are transitive : ifi � j and j � h, then i � h, where � potentially accounts forIφ

k ,Bφk or Dφ

k , with ∀φ ∈ ]0, 1].

Relations between dominance relations

Definition

A dominance relation � is stronger than dominance relation �′,noted ��′, if, ∀i , j , i � j implies i �′ j .

Theorem

∀φ, φ′ ∈ [0, 1] such that φ ≥ φ′, then

- Iφk�Bφ

k�Dφk ,

- Iφk�Iφ′

k , Bφk�Bφ′

k and Dφk�Dφ′

k ,- Iφ�Bφ�Dφ, and- Iφ�Iφ′ , Bφ�Bφ′ and Dφ�Dφ′ .

Thus the weaker the dominance relation, the more complete,that is the more dominance relations it is possible to establisha given set of institutions.

Definition

Theorem

- Iφk�Bφ

k�Dφk ,

- Iφk�Iφ′

k , Bφk�Bφ′

k and Dφk�Dφ′

Definition

Theorem

- Iφk�Bφ

k�Dφk ,

- Iφk�Iφ′

k , Bφk�Bφ′

k and Dφk�Dφ′

Publication data

OST in-house ISI-WOS database recording of publications andcitations

Large disciplines : 1/ Fund. bio., 2/ Medicine, 3/ Ap.bio/ecol., 4/ Chem., 5/ Physics, 6/ Science univ., 7/ Eng.sciences, 8/ Maths (Humanities and social sciences areexcluded.

Multidisciplinary sciences papers published in PNAS, Science& Nature have been allocated to their reference discipline fordisciplinary comparisons.

3-years citations moving window for all indexes.

Linearization of the distribution.

Publication data

Data set

French universities :

129 institutionsThe institutions checked the validity of signing patterns.

US research universities :

112 best ranked in the ARWU Shanghaı ranking (30% of PhDgranting Univ.)

Data set

129 institutions

The institutions checked the validity of signing patterns.

Data set

Networks of dominance relations

Let us consider �, which could be any one of the dominancerelations examined above (Iφ

k ,Bφk or Dφ

k , with ∀φ ∈ ]0, 1]).

Let’s build the dominance directed network ~g associated todominance relation � and the institutions set I by writing adirect link from institution i to institution j if i � j .

In this network, transitive triplets are uninformative since thetransitivity property holds. Therefore, let’s build the network~g ′ derived from ~g by deleting all such triplets : ∀i , j , k ∈ I , ifij , ik, jk ∈ ~g and kj /∈ ~g then ik /∈ ~g ′.

k ,Bφk or Dφ

k , with ∀φ ∈ ]0, 1]).

In this network, transitive triplets are uninformative since thetransitivity property holds. Therefore, let’s build the network~g ′ derived from ~g by deleting all such triplets : ∀i , j , k ∈ I , ifij , ik, jk ∈ ~g and kj /∈ ~g then ik /∈ ~g ′.

k ,Bφk or Dφ

k , with ∀φ ∈ ]0, 1]).

In this network, transitive triplets are uninformative since thetransitivity property holds. Therefore, let’s build the network~g ′ derived from ~g by deleting all such triplets : ∀i , j , k ∈ I , ifij , ik , jk ∈ ~g and kj /∈ ~g then ik /∈ ~g ′.

Dominance network : an example

Reference classes

Definition

∀k ∈ I , j ∈ c�i (⊆ I ) the reference class of institution i associatedto dominance relation � if i � j and j � i or if i � j and j � i

Reading example : j ∈ cBki means it is not always possible to

rank strictly and in a unique manner the scientific productionof i and j in domain k relying on any implicit valuationfunction of articles according to their impact which would beboth positive and increasing with impact.

Reference classes

Definition

∀k ∈ I , j ∈ c�i (⊆ I ) the reference class of institution i associatedto dominance relation � if i � j and j � i or if i � j and j � i

Reading example : j ∈ cBki means it is not always possible to

rank strictly and in a unique manner the scientific productionof i and j in domain k relying on any implicit valuationfunction of articles according to their impact which would beboth positive and increasing with impact.

Dominance Networks : Top French universities in FundBio, φ = 1 - citations

Paris 6

Paris 11

Strasbourg 1

Paris 5 Montpellier 2

Aix Marseille 2

Paris 7 Lyon 1

Grenoble 1

Dominance Networks : Top French universities in FundBio, φ = .1 - citations

Paris 6

Paris 11

Strasbourg 1

Paris 5 Montpelier 2 Aix Marseille 2

Paris 7

Lyon 1 Grenoble 1

Dominance Networks : Top French universities in FundBio, φ = .1 - impact factor

Paris 6

Paris 11

Strasbourg 1

Paris 5

Montpellier 2

Aix Marseille 2

Paris 7

Lyon 1

Grenoble 1

Complete dominance relations and ranking

Definition

A dominance relation � is said to be I -complete if ∀i , j ∈ I , i � jor j � i .

Definition

A (complete) dominance ranking R�I can be constructed overinstitutions set I on the basis of dominance relation � if and onlyif � is an I -complete dominance relation.

Complete dominance relations and ranking

Definition

A dominance relation � is said to be I -complete if ∀i , j ∈ I , i � jor j � i .

Definition

A (complete) dominance ranking R�I can be constructed overinstitutions set I on the basis of dominance relation � if and onlyif � is an I -complete dominance relation.

Complete dominance ranking

Definition

For each type of dominance (strong dominance, dominance, weakdominance), the smallest and strictly positive value of φ for whichthe corresponding dominance relations are I -complete are calledmax-I -complete dominance relations.

1 2 3 4 5 6 7 8

I .009 .004 .005 .022 .016 .019 .002 .009

B .009 .004 .008 .024 .016 .026 .002 .009

D .009 .004 .008 .024 .052 .026 .007 .009

The largest φ such that such dominance relation is I -complete overthe set of 129 French higher Education and research institutions

Definition

1 2 3 4 5 6 7 8

I .009 .004 .005 .022 .016 .019 .002 .009

B .009 .004 .008 .024 .016 .026 .002 .009

D .009 .004 .008 .024 .052 .026 .007 .009

Definition

1 2 3 4 5 6 7 8

I .009 .004 .005 .022 .016 .019 .002 .009

B .009 .004 .008 .024 .016 .026 .002 .009

D .009 .004 .008 .024 .052 .026 .007 .009

Pseudo dominance ranking

Definition

For any dominance relation and any set of institutions I , a pseudodominance ranking can be established on the basis of the score ofeach institution i , si = # {j |ij ∈ ~g }, the number of dominancerelations which emanate from institution i .

Pseudo dominance ranking : : US universities

Fund Biology I1 B1 D1

citations ri si ri si ri si

Harvard 1 111 1 111 1 111

John Hopkins 3 95 2 106 4 108

UCSF 7 91 3 105 3 109

Pennsylvania 3 95 4 103 8 104

UCLA 2 96 4 103 9 103

UCSD 5 92 6 100 6 105

Yale 11 87 6 100 5 107

Stanford 15 81 6 100 2 110

UW Seattle 5 92 9 99 12 99

Columbia 11 87 10 98 6 105

Conclusion

Combinations of influence, quality and quality.

Ranking can also be good way of doing economics (and notjust playing a childish game).

Conclusion

Combinations of influence, quality and quality.

Ranking can also be good way of doing economics (and notjust playing a childish game).

Measuring and ranking scienti c productiondimetic.dime-eu.org/dimetic_files/dimetic_ranking.pdf ·...

Documents

Scienti ccomputingintheLatin America

“Gateway” - Denver · 2015-06-25 · Ranking 1 to 10 Weight x Ranking; Ranking 1 to 10. Weight x Ranking; Ranking 1 to 10 Weight x Ranking. Ranking 1 to 10 Weight x Ranking; Ranking

Scienti c presentations: Critical errorsmath.usask.ca/fvk/Presentation21.pdf · Scienti c presentations: Critical errors From \The craft of scienti c presentations" by Michael Alley

Optimizing And Tuning Scienti c Codes

Work Control Process and Measuring Alignment...Range ranking of Work Implementers – Max minus Min ranking for each of the alignment survey statements – The larger the “spider

RESEARCH OpenAccess Semanticindexingwithdeeplearning:a ... · cation and information retrieval. Document ranking largely depends on measuring the semantic similarity of query-document

Scienti c Computing: Solving Nonlinear Equations

QS Ranking – World University Ranking & Ranking by Subject€¦ · QS Ranking – World University Ranking & Ranking by Subject Herausgeber: Quacquarelli Symonds (USA, England)

Algorithms in Scienti c Computing II

Testing Scienti c Software: A Systematic Literature Reviebieman/Pubs/kanewalaBiemanSystematic.ICST... · Testing Scienti c Software: A Systematic Literature Review ... Scienti c software,

Scienti c Python Tutorial - Boston University · Scienti c Python Tutorial Scienti c Python Yann Tambouret Scienti c Computing and Visualization Information Services & Technology

Liability law under scienti c uncertaintybusiness-school.exeter.ac.uk › media › ... › events › lawandambiguityw… · Liability law under scienti c uncertainty Luigi Alberto

Thermo Scienti c NESLAB ThermoFlex TM · Thermo Scienti c NESLAB ThermoFlex TM ... tm

Measuring economic journals’ citation eﬃciency: A data ...ranking reports (Kiel Institute internals ranking and ABS quality ranking report) in a DEA setting and in order to derive

Object-Oriented Programming for Scienti c Computing · Object-Oriented Programming for Scienti c Computing STL Containers and Iterators Ole Klein Interdisciplinary Center for Scienti

ScienTI Network: aims and perspective Lisbon, November 10th 2005 BIREME/PAHO/WHO Executive Secretariat of ScienTI

A ranking is a ranking, a ranking is not …

Scienti c Computing II · 2019. 4. 7. · Scienti c Computing II Parallel Methods Jens Saak and Martin K ohler Summer Term 2019 ... Jens Saak Scienti c Computing II 3/348. Parallel

Computational Modelling 102 (Scienti c Programming ...jenlow/files/como102_lectures.pdf · Computational Modelling 102 (Scienti c Programming) ... 1.3 Introduction to MATLAB ... Scienti

Introduction to PythonIntroduction to Python for Scienti · PDF fileIntroduction to PythonIntroduction to Python for Scienti c Computingfor Scienti c Computing ... (originally at CWI,