Gary SchwitzerPublisher
Adjunct Assoc. Prof., Univ. of Minnesota School of Public Health
Knight Science Journalism Fellowships at MITMedical Evidence Boot Camp
December 3, 2013
Journalistic peaks of excellence
But the valleys between these occasional peaks are becoming wider and deeper⌠âŚthe daily drumbeat of dreck
Every day for 7 years3,500 media messages reviewed
Two elements to our work â common theme
Review news stories that make claims about treatments, tests, products, procedures. ⢠Objective⢠Systematic⢠Criteria-driven
Comment on health care advertising, marketing, public relations, broader communications issues at intersection of medicine & media⢠Like most blogs - more
subjective, opinionated⢠Allows us to show broader
influence of industry - pharma, device manufacturers, hospitals â on the public dialogue
Our criteria: Does the story explainâŚ
⢠Whatâs the total cost?
⢠How often do benefits occur?
⢠How often do harms occur?
⢠How strong is the evidence?
⢠Are there alternative choices?
⢠Is the condition exaggerated?
⢠Is this really a new approach?
⢠Is it available?
⢠Whoâs promoting this?
⢠Do they have a financial conflict of interest?
69%
66%
65%
61%
57%
Percent unsatisfactory after 1,900 story reviews â 7 years
News stories often paint a kid-in-candy-store picture
of U.S. health care
TerrificRisk-freeWithout a price tag
Exaggerate or emphasize benefits
Ignoreor
Minimize Potential Harms
Common flaws: too much stenography â not independent vetting of studies in journals
Glorifying journals - Publication in a medical journal does not guarantee the findings are true (or even important).
Not ready for prime time. Never intended to be sources of daily news â journals meant for conversation among scientists
So if youâre going to feed off them daily, you simply must be aware of the landscape:
⢠Publication bias for positive results. Rose-colored view of progress in research.
⢠Retractions, research fraud, fabrication, falsification of data
⢠Unpublished data (BMJ special edition on âthe extent, causes and consequences of unpublished evidenceâ)
⢠Ghostwriting of journal articles (The Public Library of Science hosts a âGhostwriting Collectionâ on its website.)
Tip: Set up Google Alert for John Ioannidis about pitfalls of steady diet of journal stories
PLoS Med 2005; 2(8): e124
Journals complicit in miscommunication
⢠Editors of the HEART Group journals stated that âinappropriate word choice to describe results can lead to scientific inaccuracy.â
â J AM COLL CARDIOL, Vol. 60, No. 23, 2012
⢠âAre we making a mountain out of a mole hill? A call to appropriate interpretation of clinical trials and population-based studiesâ
â Am J Obstet Gynecol, published online 11/29/12
⢠âSpin and Boasting in Research Articles.â- Commentary in Arch Pediatr Adolesc Med:
[published online October 2012]
Risk reduction must be stated in absolute, not just relative, terms
⢠NBC News reported, âIn women, aspirin reduces the rate of stroke blood clots in the brain, the most common type, by 24% but had little effect on heart attack risk. In men, the daily aspirin cuts heart attack risk by 32 percent but had little effect on stroke risk.â
â HealthNewsReview.org comment: â24 percent of what? And 32
percent of what? The journal article upon which the story is based did state the absolute values for both benefits and risks. Those absolute values show that, over about a six and half year period, daily aspirin is predicted to result in 2 fewer strokes and 2.5 more major bleeds per 1000 women; and 8 fewer heart attacks and 3 more major bleeds per 1000 men. To many viewers, those numbers will mean more than 32% or 24%.â
The limits of observational studies should always be stated clearly
⢠The Associated Press reported âlots of red meat increases mortality risk.â HealthNewsReview.org noted: âThe language of the story suggests that cause-and-effect has been established when this kind of observational study simply can not establish causation. It can only point to statistical association.â
⢠When CNN reported, âCoffee may cut risk for some cancers,â frustrated readers reacted with these comments posted online:â "The statistics book in a class Iâm taking right now uses coffee as an example of
statistics run amok. It seems coffee has caused all the cancers and cures them at the same time."
â "âŚcorrelation IS NOT causation!!!! So people that drink 4 or more cups of coffee have a lower incidence of two certain types of head and neck cancers, and this is supposed to mean that coffee is actually âwarding offâ these cancers???â
The tyranny of the anecdote⢠Stories may use only positive, glowing patient anecdotes but fail to capture trial
dropouts, compliance problems, patient dissatisfaction or the choice to pursue less aggressive options.
⢠In one example, ABC, CBS, NBC, WebMD all used the same âbreakthroughâ patient anecdote of one manâs reported improvement from a heart failure stem cell experiment.
â This suggests another common problem: public relations people spoon-feed journalists the patient anecdotes that put an intervention in the most positive light.
Single source stories & journalism via news release are unacceptable
⢠ABCâs Good Morning America asked, âCould a cure for obesity be just a pill away?â HealthNewsReview.org analyzed the sourcing in this story: â âIn a bizarre twist, the reporter used the potential conflicts of interest of the doctor he
interviewed in the studio as evidence of his expertise. He noted that the doctor is a consultant to the manufacturer of the pill and then said, âYouâre the right man to talk to.â Really? In a recent journal publication, this researcher listed financial ties to 17 drug and device makers, including the maker of the (pill in question)⌠viewers were not clearly told that all of the results reported in the story came from a trial controlled by the company that has not been independently reviewed.â
⢠The Wall Street Journal reported, âDepression treatment Cymbalta effective as pain reliever, Lilly says.â HealthNewsReview.org wrote: â âThis 189-word story failed on almost every count: â no independent perspective;â a quote apparently lifted from a company news release;â The company news release actually did a better job of explaining the study than did this
news story.â
Failure to explain limits of surrogate markers
⢠USA Today: âNew drug âmay turn back the clock on heart disease.â ââ HealthNewsReview.org analysis: âThe story focuses excessively on the
âunprecedentedâ changes in cholesterol achieved with the drug. ⌠these are surrogate markers that may not reflect any real benefit on the outcomes that matter to patients.â
⢠WebMD: âBeet Juice Good for Brain: Drinking Beet Juice Increases Blood Flow to Brain and May Fight Dementia.â â HealthNewsReview.org analysis: âSurrogate endpoints do not necessarily
reflect on overall health outcomes. And especially not from a study of 14 people over four days. And from this we get the headline that âdrinking beet juice may fight dementiaâ ???â
⢠HealthNewsReview.org linked to 10 stories that framed a tiny short-term study of a biochemical marker of bone turnover as if it were definitive advice to women that, for example, âTwo glasses of wine a day ends menopause misery.â
Screening stories should explain the tradeoffs of benefits and harms
⢠The worst single episode was news coverage of the November 2009 revised mammography recommendations from the U.S. Preventive Services Task Force. We wrote: âThere is an undeniable and clear bias in many news stories, reporters and news organizations for promoting screening â evidence be damned.â
⢠The HealthNewsReview.org story, âA clear pattern of imbalanced screening stories by ABC News,â identified 10 examples in recent years involving cardiovascular screening, and screening for lung, prostate and breast cancer
⢠The HealthNewsReview.org story, âPrevention magazine editor promotes ovarian cancer screening despite lack of evidence,â explained that neither the American Cancer Society nor the National Cancer Institute support the transvaginal ultrasound screening advocated by the magazine.
⢠We have criticized the Chicago Sun-Times, the New York Daily News, the NBC station in Minneapolis, and other news organizations for promoting public prostate cancer screening events, disregarding evidence-based recommendations.
⢠The project has also identified imbalanced screening stories by CNN, CBS, Fox News, NBC News, Minneapolis Star Tribune, Saint Paul Pioneer Press, Chicago Sun-Times, Washington Post, and HealthDay among others.
Unquestioning news coverage of disease awareness campaigns
⢠Prostate cancer awareness is one focus of the âMovemberâ publicity campaign subtitled âChanging the Face of Menâs Health.â In the WCCO-TV (CBS owned-and-operated) television station newsroom, several members of the news staff grew mustaches âin support of the cause.â Exactly which cause, and how growing a mustache helps, is unclear. This campaign has spawned countless prostate cancer screening events, conflicting with many evidence-based recommendations.
Moâ Movember âawarenessâ
⢠The NBC station in Atlanta had newsmen wear johnnies to promote prostate screening.
⢠The Chicago Sun-Times promoted Movember with misinformation from Dr. Laura Berman, self-proclaimed âstarâ of âIn The Bedroom with Dr. Laura Berman.â
Fawning coverage of new technologies (proton beam therapy & robotic surgery)
⢠KERA public radio in Dallas reported on both technologies in its âBattlefield Breakthroughâ series. â Headline: âZapping Cancer With A Ray-Gun.â We wrote: âThe
story presents not one word of caveats, concerns or limitations. There was no independent source, no data about efficacy.â Instead, there was fascination: âlooks like something out of Star Wars.â
â Headline: âRobots Slice Time Under The Knife.â
We wrote: âStar Wars was a theme in that one as well. In fact, the formula was the same as for part one with proton â more battlefield breakthroughs. ⌠Maybe we need to wait a bit to see how this arms race plays out with evidence and data before proclaiming battlefield breakthroughs just yet.â
Going easy on health business stories⢠Reuters
â Headline: Medtronic tests stents for erectile dysfunction. Our comment: âAn initial feasibility study in 30 men is worthy of news coverage? With a company-sponsored investigator as the only source? Not in our view.â
⢠Chicago Tribuneâ Headline: Area firmâs pneumonia treatment passes test. Our comment: âIt features the company presidentâs
positive assessment of the study results but does not seek independent comment. ⌠It fails to make clear that the results of the trial have not been peer-reviewed or published and that results of a related trial have not been released at all.â 56
⢠New York Timesâ Headline: âTest of Eye Drug Is Said To Show Success in Elderly.â Our comment: âThe piece ⌠shouldâve
slapped a bigger warning label on this cocktail of leaked summaries of secret results, speculation, and an inadequate dash of independent evaluation.â
⢠Wall Street Journalâ Headline: âAstraZeneca Says Brilinta Beats Plavix in Clinical Trial.â Our comment: â⌠lets a drug company get
away with making superiority claims without releasing data. ⌠All information in the story appears to come from company sources.â
⢠Los Angeles Timesâ Headline: âDrug for menstrual cramps in the works.â Our comment: âThis story reports that data were
presented at a scientific meeting. But it didnât tell readers what the data showed. But it did allow a company VP â the only person quoted â to say this could be a âbreakthrough.â Wow.â
⢠Minneapolis Star Tribune â HealthNewsReview.org found four stories within six weeks in which the Star Tribune established a clear
pattern of writing nice things about local medical device industries, but did not lead in reporting negative stories.
Whom to believe? Framing matters.
⢠Dueling news stories on brain wave scan for ADHD. Important tool? Or waste of money?
â NJ Star-Ledger, Fox News, NY Daily News, Bloomberg â all gushing praise such as âthis should be an important toolâ quote from MD in the study
vs.
â NY Times quoted independent expert âskeptical about the testâ
â LA Times quoted independent expert âThe current scientific research really doesnât support (this) as a diagnostic tool. I would caution people.â
â ABC News quoting independent expert: âI donât know that this going to help...itâs going to make people spend money needlessly. They can charge for it and it gives you a pseudo-scientific basis for diagnosis.â
Whom to believe? Framing matters.Dueling news stories on melanoma detection device. Helping doctors? Or causing dissent/polarizing the field?
⢠Cleveland Plain Dealer: âhelps doctors detect deadly cancerâ⢠WTVF Nashville & KFSN Fresno â among the stations using a syndicated news
serviceâs story that claimed âthe Cleveland Clinic named the device one of its top medical innovations of 2013.â
⢠KDVR Denver allowed a doctor to call it the biggest advance in melanoma in his 26-year career
vs.⢠NY Times quoted 3 independent experts:
1. âThis should still be considered to be in the developmental stage.â2. A dermatologist on the FDA panel who voted against approval3. Biostatistician: â(The test) just says everything is positive. I donât think it
helps an aggressive doctor and unaggressive doctors could do just as well if they were more diligent without the device.â
Whom to believe? Framing matters.
⢠Dueling news stories on meds for mild-moderate depression. Give pause about prescribing? Or no reason to avoid meds?
â HealthDay: âSeverely depressed gain most from antidepressants: But thatâs no reason to avoid meds for moderate cases.â
vs.
â LA Times: âStudy finds medication of little help to patients with mild, moderate depressionâ
â USA Today (quoting independent expert): Such findings âdemonstrate a failure in the system: These drugs were not thoroughly tested in mild to moderate depression prior to their approval.â
Whom to believe? Framing matters.
⢠Dueling news stories on Alzheimerâs drug. BY THE SAME NEWS SOURCE! Reverses disease in mice? Or a flop?
â MedicalDaily.com: âCancer drug may help reverse Alzheimerâs.â
AND
â âTargretin a flop: Potential Alzheimerâs drug fails retrial.â
Whom to believe? Framing matters.
⢠Dueling news stories on limitations of colonoscopy. Miss many cancers? Or concerns overblown?
â NY Times: âColonoscopies miss many cancers, study finds.â
VS.
â ABC News: âThe Case for Keeping Colonoscopy: Cancer Experts Say Concerns Over Colonoscopy Effectiveness May Be Overblown.â
â CBS Newsâ Katie Couric: âDonât use this study as an excuse not to get screened.â
Whom to believe? Framing matters.
⢠Dueling news stories on prostate cancer screening study. Reduces deaths? Or isnât saving lives/value questioned?
â MSNBC: âRegular prostate screening reduces deathsââ HealthDay: âPSA testing cuts cancer death riskââ Bloomberg: âPSA screening reduces deathsâ
vs.
â Associated Press: âprostate cancer screening isnât saving lives, study findsâ
â CNN.com: âValue of mass prostate cancer screenings questioned.â
Framing
⢠Star Tribune: âBoy, 12, dies after historic transplant to treat HIV, leukemia.â â âHe was in line to become the second person in
the world to be cured of both deadly illnesses by the extraordinary type of bone marrow transplant, doctors said.â
â Maybe. But in fact he was in line to die from a transplant that was not historic because it did not treat his HIV & leukemia.
âMediaâs most troublesome trendâ award
THE WINNER IS:
SCREENING TESTS(often involving this gland)
Award for best supporting roles:⢠Breasts⢠Lungs⢠Arteries
I am not anti-screening
⢠But I do criticize: â Stories that donât discuss tradeoffs involved in all
screening test decisionsâ Framing screening as a mandate, not a choiceâ Stories/journalists taking an advocacy stanceâ Stories that are inaccurate, imbalanced,
incomplete
âAll screening programs do harm; some do good as well.â - Dr. Muir Gray
All screening stories would be better if they simply stated:
Anecdotes of those who claim their life was saved by screening
Anecdotes of those who make a rational choice
not to be screened â or
those who were screened and
regret it
The tyranny of the anecdotein screening news stories: Whom do we model?
Prizes for Prostates
Dr. Otis Brawley, chief medical officer, American Cancer Society
âIâm very concerned. Thereâs a lot of publicity out there â some of it by people who want to make money by recruiting patients â that oversimplifies this â that says that âprostate cancer screening clearly saves lives.â That is a lie. We donât know that for sureâŚâŚWeâre very concerned about a number of clinics that are offering mass screening where informed decision making is not happening. Many of these free screening things are designed more to get patients for hospitals and clinics and doctors than they are to benefit the patients. Thatâs a huge ethical issue that needs to be addressed. Weâre not against prostate cancer screening. Weâre against a man being duped and deceived into getting prostate cancer screening.â
âDuped & deceivedâ
Tim Glynn, lawyer, age 47 when his doctor âdecided Iâd have a PSA test without consulting meâŚ..Men should be aware of the truly terrible consequences. As a screening tool, you could do as well by throwing dice on a table.â
Sharon Begley in Newsweek:
âDr. Stephen Smith, Professor emeritus of family medicine at Brown University School of Medicine, tells his physician not to order a PSA blood test for prostate cancer or an annual electrocardiogram to screen for heart irregularities, since neither test has been shown to save lives. Rather, both tests frequently find innocuous quirks that can lead to a dangerous odyssey of tests and procedures.â
âI can only hope my story will âŚinspire every woman who hears it to get a mammogram, to take a self-exam. No excuses. It is the difference between life and death.â
1. âNo excusesâ promotes theme of mandate, not a choice. 2. No, it is not the difference between life and death for every
woman.3. There are potential harms from mammography. They are
clear, well-known, supported by data.
Mammography screening also finds slowly growing or less aggressive tumors which would never be life-threatening. But because that cannot be predicted, about 10 of 2,000 women in the screening group had their breast completely or partially removed, though this would not have been necessary.
For years Iâve been writing about why itâs unethical for network television journalists to use their celebrity status to talk about their own health care decisions and to suggest
what other people should do.
⢠NBCâs Mike Taibbi had lung CT and endorsed the scans in his piece.
⢠NBCâs George Lewis reported on his own prostate cancer diagnosis/treatment and ended recommended all men have PSA tests
⢠NBCâs Andrea Mitchell reported on her own breast cancer diagnosis and treatment by giving advice on the air to other women
⢠NBCâs Matt Lauer & Al Roker had digital rectal exams on the air to promote prostate cancer screening and failed to discuss guidelines which donât recommend such screening.
⢠CBSâs Harry Smith followed colleague Katie Couricâs example from when she was at NBC and had a colonoscopy on the air, endorsing the practice while not mentioning other colon cancer screening options which many people might prefer and which the evidence supports.
Andrea Mitchell: âFor you women out there and for the men who love you, screening matters. Do it. This disease can be
completely curable if you find it at the right time.â⢠Katherine OâBrien â who published the ihatebreastcancer blog â wrote:
âEarly detection is not a cure. âŚâCompletely curableâ is a like a fat man wearing a hockey jersey. It covers a lot of ground. You have access to the top medical experts in the world-ask them what âcuredâ means in the setting of breast cancer.â
⢠Breast cancer patient Katie Ford Hall of the UneasyPink blog wrote to Ms. Mitchell:âWishing you the best Ms Mitchell. When you feel settled, Iâd love to talk to you about the realities of breast cancer âcaught earlyâ ⌠What you said about breast cancer is harmful and untrue. You should correct it immediately.â
⢠One expert pointed out the misuse of the overused â1 in 8â statistic, as when she said:
âI am now among the 1 in 8 women in this country â incredibly 1 in 8 â who have had breast cancer.â ⢠As the National Cancer Institute explains:
Women born now have an average risk of 12.2 percent (often expressed as â1 in 8âł) of being diagnosed with breast cancer at some time in their lives. On the other hand, the chance that they will never have breast cancer is 87.8 percent (expressed as â7 in 8âł).⢠But 1 in 8 is a lifetime risk. Risk increases with age, so the NCI provides a more helpful way of looking at
it â for all of those women watching who are of different ages: ⢠A womanâs chance of being diagnosed with breast cancer is:
from age 30 through age 39 . . . . . . 0.43 percent (often expressed as â1 in 233âł)from age 40 through age 49 . . . . . . 1.45 percent (often expressed as â1 in 69âł)from age 50 through age 59 . . . . . . 2.38 percent (often expressed as â1 in 42âł)from age 60 through age 69 . . . . . . 3.45 percent (often expressed as â1 in 29âł)
What is the quality of the shared decision-making encounterâŚ
âŚas mammograms are promoted with wine, cheese, fondue, chocolate fountains, pink treats, pink lemonade, massages, facials, manicures/pedicures, paraffin hand treatments, beauty consultations, etc. ?
Sharon Begley reported for Newsweekmagazine:
âDr. Rita Redberg, professor of medicine at the University of California, San Francisco, and editor of the prestigious Archives of Internal Medicine, has no intention of having a screening mammogram even though her 50th birthday has come and gone. Thatâs the age at which women are advised to get one. But, says Redberg, they detect too many false positives (suspicious spots that turn out, upon biopsy, to be nothing) and tumors that might regress on their own, and there is little if any evidence that they save lives.â
⢠Reported on woman with DCIS â ductal carcinoma in situ⢠Reported on her choice to pursue active surveillance rather
than immediate aggressive intervention such as bilateral prophylactic mastectomy
⢠Reported on the shared decision-making program at UCSF that helped her understand the tradeoffs.
THAT IS GOOD JOURNALISM
⢠Began with virtual colonoscopy â after routine physical.
⢠But scan also showed something on kidney, liver, and lungs.
⢠Kidney and liver issues benign after liver biopsy, PET scan and more CT scans. But lung questions led to major lung surgery
âI awoke in the recovery room after 5 hours, with a chest tube, a Foley catheter, a subclavian central venous catheter, a nasal oxygen catheter, an epidural catheter, an arterial catheter, subcutaneously administered heparin, a constant infusion of prophylactic antibiotics, and patient-controlled analgesia with intravenously administered narcoticsâŚ..Excruciating pain.â
⢠No malignant disease â all âincidentalomasâ ⢠Total cost > $50,000
⢠All precipitated by a screening test
Repeat: itâs not anti-screeningâŚ.
⢠âŚwhen you cite the evidence.⢠âŚwhen you educate people about tradeoffs.⢠âŚwhen you help people avoid âavoidable
ignorance.â
⢠We shouldnât be telling anyone not to be screened just as we shouldnât be telling anyone to be screened. We should inform about the tradeoffs so people can make informed decisions.
If we donât improve discussion on screening, we may never improve public dialogue on expensive new technologies â âGizmo Idolatryâ
Proton beam or IMRT
JOSH BILLINGS (PEN NAME OF HUMORIST HENRY WHEELER SHAW, 1818 â 1885)
âI honestly believe it is better to know nothing than to know what ainât so.â
Why?
Because there are harms in making people more hopeful or more frightened than evidence and rational thinking would warrant.
@Thank you