Changing the Structures of Research from the Perspective of an Active Scientist Evaluation and...

Changing the Structures of Research from the Perspective of

an Active Scientist

Evaluation and Dissemination-

Factors Affecting the Creativity Process

http://w3.rz-berlin.mpg.de/ac/ac.html

R. Schlögl

Structure-Function Relations

020004000020400200040000306002004006002461001503

Å

Å

The atomic structure of model systems transform reversibly upon subjecting the model to the desired function

no extrapolations from ideal to real states

no methods to investigate dynamics

no basic understanding of underlying chemistry

“complexity”

Heterogeneous Catalysis

Research in this and in many other fields requires per se an interdisciplinary approach

R.

Interdisciplinarity

This is converted into a generic strategy

Would never show up in interdisciplinary accounts, as publicised within the catalysis community

Evaluation targets

• Projects (Resources)

•Appointments

•Institutions

•National-International science politics

Hypothesis:

Science is lacking an external regulatory pressure and happens by enlarge outside of the social context of the financing society.

Consequence:

Suitable evaluation is essential

Projects

• Megaprojects (HGP, space station, LHC, astronomy etc.

•Group and network activities (scientifically required or politically forced)

•Long-term projects (astronomy, humanities, physics)

•Individual projects (extremely diverse)

•Industry - Academia relations (chemistry, materials science)

Evaluation criteriaNon-accountable criteria:

* qualitative arguments, rely on in-depth content information

* require expert knowledge

* are always debatable

Accountable criteria

* transparent

* intersubjective

* context insensitive

* superficial

Impact Factors of Journals

Selected journals from our daily work

note distortions between practical relevance and impact ranking

R.

The relevance of impact factors

J. Am. Soc. Inf. Sci., 47, (1996), 458

The journals of analytical chemistry all fall within a very narrow range on the score index

The relevance of impact factors

J. Am. Soc. Inf. Sci., 47, (1996), 458

Using a statistical manipulation of the SCI JCR data one ends up with a totally different score board than with the Garfield parameter.

The relevance of impact factors

Scientometrics 1 (1979), 359

SCI Journals Citation Reports (1993)

The SCI JCR parameter is an equilibrating ratio method precluding a lot of information on the relevance of a journal for the scientific dissemination in the field.

The relevance of impact factors

Analyt. Chim. Acta, 360, (1998), 215

Impact factors are used as steering factors for selection of analytical methods in food chemistry.

Scientometrics

ISI, National Science Indicator/ 1995-1999

Grand accumulations (here 293.000 papers) are used to evaluate national strengths in science

Accountability

• keyword from economics (“economisation” of science)

• essential for controlling and distributing resources

• replaces discourse over contents

The charm of quantitative measure for quality has two dangerous consequences:

It affects the problem choosing strategy of scientists (fashions)

It allows decision makers to atomise responsibility (“independent authority ISI”)

Speed in Science• no implicit quality of science

•different from “critical mass of resources”

• keyword from sports (is sports a model for science?)

• accountable quantity, “objective measure”

• dangerous for quality:–inhibits in-depth research–calls for repetition and fragmentation–creates fashions–produces superficial and glamorous statements

Quality in Science

No unambiguous definition given so far

• contribution to knowledge base

• results as “product”:–cost effective–innovative–useful for application–speed

Quality in Science

• originality of work

• novelty and generic character (“seminal”)

• general value–for mankind–for social structures–for environment–..........

Changes in Evaluation

• reduce the load of evaluation, deepen its quality

• transform from threat into challenge

• emphasise the content-oriented evaluation

• recognise the negative effects of accountability

adapt the evaluation procedure to the desired purpose

Do not use (formal) evaluation as excuse for neglecting the necessary discussion about targets and priorities in science!

Dissemination

Should be of genuine concern to each scientist as it is outlet and input to any scientific production

we face a revolution of dissemination by the existence of reliable internet functions

MPG has taken measures to give access to electronic journals to all its co-workers and prepares for

innovative uses of the e-media

Dissemination

Work flow and its change by the Internet

Dissemination - non traditional

E-media exhibit new properties over the “replacement” of paper media:

• highly parallel access to complex information

• unhindered access to interdisciplinary information

• non-discriminating access to information

• interactive access to information

E-media, emerging possibilities

Much is under evolution, it will transform the working habits of science in all disciplines

E-media and evaluation

• play a major role in directing evaluation towards content-oriented judgement (vs. formal and prejudice oriented)

• world-wide access to achievements and strategic considerations of individuals and institutions (no numerical oversimplification)

• flexible medium to accommodate the variety of work habits in scientific

communities

• easy access to tutorial information

Dissemination - print media

• are expected to become more important as review and teaching medium

• books need to be written with more original thought than today (quick review collections)

• conference print media are obsolete and should be banned from inflicted subsidies by fees

Dissemination in MPG

e

Vision of MPG system

ZIM in foundation to transform traditional system

Changes in Dissemination• re-consider division of work between industry

and academia

• recognise the internet as prime information for content (exceeding that of papers)

• recognise internet as novel medium for interactive work

• archive selectively only final documents and databases

•maintain the minimum regulatory level in the internet (on the expense of perfection)

Generalisation

• science is organising itself by the actions of individuals in diverse communities beyond the traditional disciplines (“interdisciplinary”)

• allow for flexible organisational framework

• do not attempt to administer this process, refrain from standardising procedures of dissemination and evaluation

Generalisation•Dissemination is an intrinsic affair of scientists

and their organisations (get involved!)

•e-media reflect the diversity of scientific activity and need support from within science on the generic level (tools)

• formal standards for traditional and innovative internet science products are needed

• archiving is an international affair (organisation)

• there are awareness and acceptance problems

Conclusions

•science at large is no business

•correctness and credibility are fundamentals of science

•define targets and priorities on the basis of facts not on formalities nor pre-conceptions

•allow for a mix in evaluation criteria and rapidly use e-media for dissemination and evaluation

creativity and diversity require each other

beware from limiting any of the two!