The Social Cost of Scientific Misconduct and Questionable Research Practices

William Gardner, PhD

Center for Innovation in Pediatric Practice, Center for Innovation in Pediatric Practice, Columbus Children’s Research InstituteColumbus Children’s Research InstituteCenter for Innovation in Pediatric Practice, Center for Innovation in Pediatric Practice, Columbus Children’s Research InstituteColumbus Children’s Research Institute

Departments of Pediatrics, Psychology, & PsychiatryDepartments of Pediatrics, Psychology, & PsychiatryThe Ohio State UniversityThe Ohio State University

Departments of Pediatrics, Psychology, & PsychiatryDepartments of Pediatrics, Psychology, & PsychiatryThe Ohio State UniversityThe Ohio State University

Supported by NIH/ORI: 1R01 NS49591.

Social Costs of Research Misconduct (RM)

• How much should we care about RM?

• “A lot, because integrity is essential to science.”

• But why should non-scientists care about this?

• Consequentialist answer: “A lot, because research misconduct may have a large social cost.”

• My talk is conceptual, not empirical. I’ll define this social cost and suggest how to study it.

Direct, Visible Economic Costs of Research Misconduct

• Steneck: What does the NIH lose from RM? • ‘Direct’ Cost Loss = Grant Award $s. • ‘Visible’ Cost Use only known RM cases.• Loss / Year = (N RM Cases / Year) x Cost / Case.• Cost / Case = $427K (Avg 2005 NIH award).• N RM Cases = 10 cases / Year. • Therefore, Loss / Year = $4.3M.

Direct, Invisible, Economic Costs of Research Misconduct

• ‘Invisible’ Cost Impute rate of RM cases based on hypothesized incidence of RM.

• N of NIH Grants (2005) = 47,345.

Prevalence of RM Imputed 2005 Loss

1/1000 Grants $20M

1/500 Grants $40M

1/250 Grants $81M

1/100 Grants $202M

Indirect Economic Costs of Research Misconduct

• ‘Indirect’ Cost Loss = monetary value of benefits that society would have received from research, but did not.

• Challenge 1: Measure monetary value of science.• Challenge 2: Identify the benefits that society did

not receive because of RM.

Two Thought Experiments

• Did research misconduct delay the start of smoking prevention efforts?

• Does research misconduct reduce the efficiency of pharmaceutical research?

Thought Experiment: Scientific Misconduct & Smoking

US Cigarette Tobacco Consumption (Pounds per capita) by Year

0

2

4

6

8

10

12

1895 1905 1915 1925 1935 1945 1955 1965 1975 1985 1995

Year

Po

un

d p

er c

apit

a

1964 Surgeon General's Report

Doll & Hill (1950), BMJ

Massive behavior change, catalyzed by dissemination of scientific research.

Reduction of CVD Mortality Attributable to Smoking Reduction

Based on Cutler (2001).

4.5 years improvement in life expectancy over 50 years due to reduction in CVD mortality.

Of that, 1.5 years reflects behavioral change.

Of that, assume .75 years is smoking reduction.

Additional Life-Years From CVD Mortality Reduction Attributable to Smoking Reduction

Accumulate the per capita gain in chance of surviving a year across population.

Annual Value (in $B) of CVD Mortality Reduction Attributable to Smoking Reduction

Valuing a year of life at $100,000 (Murphy & Topel, 2001).

What if the Surgeon General Spoke Out in 1959?

Scientific debate about smoking was rife with misconduct.

Counterfactual curve:

•Suppose that RM & QRP had not clouded the debate.

•Suppose that prevention efforts had started 5 years earlier?

Additional Benefit From Reduced CVD Mortality Attributable to Earlier Smoking Reduction

Shaded area is the value of the reduction of CVD mortality, had we started 5 years earlier ( > $100B).

Under these counterfactuals, these are the social losses attributable to RM.

Moving prevention efforts forward 1 year saves $27B.

Thought Experiment: Misconduct, QRP, & New Drugs

InvestmentDrug Development Process

Reduced Morbidity & Mortality

RM & QRP results in the direct loss of investment.

However, RM in the drug development may also reduce N of new drugs / $ invested.

Fewer new effective drugs means less reduction in mortality. Indirect Cost = Value of life-years not saved.

The Drug Development Process

• Drug development is a multiple stage process, from pre-clinical investigations through multiple phases of CTs.

• Later stages of development involving animals and humans are far more expensive than earlier stages.

Clinically Useful Compounds Tested at

Stage t

Useless Compounds Tested at Stage t

(N Useful Compounds at t-1) x (Sensitivity at t-1)

(N Useless Compounds at t-1) x (False Positive Rate at t-1)

Misconduct & The Drug Development Process

• Misconduct increases the False Positive Rate in the drug development process.

• Small increases in the False Positive Rate in drug testing will substantially increase the proportion of useless compounds tested in expensive later stages.

• This wastes research dollars, a direct cost of RM.• However, it also increases the investment cost of

developing a useful drug fewer drugs are developed.

The Value of the Missing Drugs

• The indirect cost of RM in drug development are those missing drugs.

• Econometric models suggest that better drugs for CVD have a larger impact than smoking reduction.

• Hence the annual value of life-years saved due to improved CVD medication alone is worth tens of $B.

• Small inefficiencies in the drug development process will result in losses that are likely to be scaled in $B.

Summary: Social Cost of Fraud in Pharmaceutical Research

• RM and QRP make it harder to eliminate bad ideas early, and may induce other researchers to pursue false leads, reducing research efficiency.

• Therefore, RM and QRP increase the cost of discovery, reduce the rate of discovery, and reduce the rate of growth of life expectancy.

• RM & QRP are like securities fraud: Society loses from both the loss of the investment and reduced economic growth from the misallocation of capital.

So we have proved what, exactly?

Buck Mulligan, in the Martello Tower, speaking of Stephan Dedalus:

“It’s quite simple. He proves by algebra that Hamlet’s grandson is Shakespeare’s grandfather and that he himself is the ghost of his own father.”

James Joyce, Ulysses

I am waving my hands. The point of waving my hands is to point out a new question, and suggest how we might try to answer it.

Limitations of Thought Experiments

• Our examples do not include data on misconduct.

• Our arguments rely on economic models that are controversial.

• We consider only medical examples.

Conclusion to Thought Experiments

• We have clarified the meaning of an indirect social cost of research misconduct.

• We have also clarified the stakes in RRI.• Steneck’s estimate of the direct costs of RM were

expressed in $M. • I think the value of indirect social costs of RM

should be expressed in $B (3 orders of magnitude).

Conclusion: The Social Costs of Scientific Misconduct

• Because RM is relatively rare, the direct costs of RM are likely small.

• The social value of science, however, is titanic.• Inefficiencies in technology transfer caused by

RM should be projected across this large value. • Social costs of RM are greater than are

imaginged, and vastly greater than the cost of research on RI.