Upload
malcolm-carr
View
212
Download
0
Embed Size (px)
Citation preview
What counts as good evidence?
Dylan EvansDepartment of Mechanical Engineering
University of Bath
It’s hard to know what is bogus!
• NASA invests $1m in antigravity machine
• US Patent 6,362,718 issued for a motionless electromagnetic generator
• Both of these machines are physically impossible
• But scientists have advocated both!
Seven warning signs
• Robert L Park is a Professor of Physics at the University of Maryland
• He has identified seven warning signs that indicate that a scientific claim is bogus.
• Remember, these are only warning signs – they are not infallible.
1. Pitching to the media
• The integrity of science rests on the willingness of scientists to expose new ideas and findings to the scrutiny of other scientists before anyone else.
• This is called ‘peer review’.• Going to the media first suggests that
the work is unlikely to stand up to peer review. Eg. Pons & Fleischmann, 1989.
2. Conspiracy and cover-ups
• The discoverer says that a powerful establishment is trying to suppress his or her work.
• Mainstream science is made out to be a part of a larger conspiracy that includes industry and the government.
• Examples: GM potatoes, cold fusion.
3. At the limit of detection
• The scientific effect is said to be very hard to detect.
• All scientific measurements must contend with some level of background noise.
• But if the signal-to-noise ratio cannot be improved, even in principle, the effect is probably not real.
• Examples: parapsychology.
4. Evidence is anecdotal
• Anecdotes and narrative have a very strong emotional impact.
• Statistics have a weak emotional impact.
• Frauds tend to use anecdotes and not statistics such as RCTs.
• Scientists have to learn to put statistics above anecdote. Eg. bloodletting.
5. Ancient wisdom
• The discoverer says a belief is credible because it has endured for centuries.
• ‘Our ancestors possessed wonderful advanced knowledge that has been forgotten.’ Eg. crystal healing.
• Science was born when people began to reject blind tradition in favour of experiment and observation (eg. Bacon)
6. Splendid isolation
• The image of the lone genius who struggles in secrecy in an attic laboratory is rarely accurate.
• Scientific breakthroughs nowadays are almost always syntheses of the work of many scientists.
7. New laws of nature
• The discoverer proposes a new law of nature that conflicts with what is already well-established.
• Weigh up the evidence for each alternative: eg. homeopathy and the meta-analysis in the BMJ.
On the other hand…
• Sometimes a scientist MAY go to the media first!
• Sometimes there ARE powerful forces trying to suppress a good idea!
• Some scientific effects ARE very hard to detect!
On the other hand… (contd)
• Anecdotal evidence is better than NO evidence at all!
• Some ancient knowledge HAS been forgotten!
• Some scientists DO work in isolation!
• Sometimes a new law of nature IS correct!
So how can we tell?
• No ‘recipe’ or ‘algorithm’ for assessing science.
• No substitute for thinking!
• But there are some general guidelines on how to be critical. Apply them to your own work and to that of others.
Whenever you read a paper…..(even your own!)..
• What is the hypothesis being tested?
• Read the methods section – is there enough information to allow the experiment or observation to be repeated?
• Read the results section – is there a proper statistical analysis?