Data - The Next Quality Disaster - ASQ Baltimoreasqbaltimore.org/dt/present/Present200212_Next... · and Utah and 3.7 percent in New York. • 8 8 percent of adverse events led to

Data - The Next Quality DisasterAmerican Society for QualityAmerican Society for Quality,

Baltimore SectionThomas C Redman Ph DThomas C. Redman, Ph.D.Navesink Consulting Group

732-933-4669www.dataqualitysolutions.com

December 10, 2002

© Navesink Consulting GroupASQ-DQ Disaster

My messages

Bad data are at the root of the events of national and international importance Theynational and international importance. They capture our interest and won’t let go.

Fortunately, most data quality issues don’t make the newspaper Unfortunately theymake the newspaper. Unfortunately they are insidiously hidden in everyday work, increasing cost causing customer dis-increasing cost, causing customer dissatisfaction, and making it harder to make decisions.

© Navesink Consulting Group

Outline

• Data Quality DefinedQ li i i h• Data Quality Disasters in the News

• Everyday DQ Disasters• COPDQ• We Know What To Do About It• Final Remarks: A Clarion Call


What Makes Quality Data?Data are of high quality if they are fit for their

intended uses (by customers) in operations, decision-making and planning (after Juran)making, and planning (after Juran).

Data that’s fit for use

free of defects:- accessible

possess desired features:- relevant

- accurate- timely- complete

- comprehensive- proper level of detail- easy-to-readp

- consistent with other sources- etc.

y- easy-to-interpret- etc.

C t th lti t bit f lit !!


Customers are the ultimate arbiters of quality!!

Poor Data Quality in the News

“Wall Bursts and Water Pours In, Trapping 9 Miners 240 F D ”240 Feet Down”

Though Pennsylvania state regulations require a 200 foot safety wall between adjoining mines, outdated maps of area mines failed to show the miners’ proximity toof area mines failed to show the miners proximity to an existing mine. After 77 difficult hours, all nine men were freed.

The New York Times, July 26, 2002, Francis X. Clines


Poor Data Quality in the News“Right Answer, Wrong Score: Test Flaws Take Toll”

The testing industry is coming off its three most problem-plagued years. Its missteps have affected millions of students who took standardized proficiency tests in at least 20 states. An examination of recent mistakes and interviews with more than 120of recent mistakes and interviews with more than 120 people involved in the testing process suggest that the industry cannot guarantee the kind of error-free, hi h d t ti th t t d t dhigh-speed testing that parents, educators and politicians seem to take for granted.


New York Times, May 20, 2001, Diana B. Henriques

Poor Data Quality in the News“Indecision 2000: The Ballad of the Ballots”

The 2000 Presidential Election in the United States held our attention for weeks as we tried to determine:

• the accuracy of various counts and recounts• how the end-to-end "election chain" is supposed to

workwork• the final result and consequences of the above.

The Associated Press, November 19, 2000, Ted Anthony and Jerry Schwartz


Poor Data Quality in the News“Voting: What is, What Could Be”

Four to six million votes, a number which is double the population of Chicago, were lost in the 2000

id ti l l ti d t bl ith b ll tpresidential election due to problems with ballots, equipment, registration or at the polling place, according to a joint analysis by experts at the California Institute of Technology and the Massachusetts Institute of Technology.

MIT News, July 16, 2001



“Primary Day in Florida Goes On, and On, and On”

The primary election results for the Democratic nomination for governor of Florida were held upnomination for governor of Florida were held up for 3 days while Miami-Dade election officials tried to sort out problems at polling locations –despite the new $32 million voting system whichdespite the new $32 million voting system which replaced the one blamed in the 2000 Presidential Election.

The New York Times, September 14, 2002, Adam Nagourney



"Holes in System Hid Links in Sniper Attacks".

According to the authors, "In the month before the sniper attacks that left 10 dead and terrorized in the W hi t J h M h d d L M lWashington area, John Muhammad and Lee Malvo are believed to have killed or wounded seven people in four states. But because they kept on the move and the police gave only routine attention to what seemed like run-of-the-mill crimes, investigators say, Mr. Muhammad and Mr Malvo were able to slip throughMuhammad and Mr. Malvo were able to slip through the cracks, gaps and blind spots where jurisdictions meet and crime-fighting databases end."


New York Times, November 29, 2002


“CIA Fires Officer Blamed in Bombing of Chinese E b ”Embassy”

Using out-dated information, the CIA selected the address of an armory for a bombing target. At the time of the bombing however the building housedtime of the bombing, however, the building housed the Chinese Embassy.

The New York Times, April 9, 2000



“Mail from INS Stuns Flight School”

As the result of mismanaged documents, a Florida fli h h l i d i f h INS fflight school received notice form the INS of approval of student visas for two of the September 11th terrorists – six months to the day after the11 terrorists six months to the day after the attack on America.

USA Today, March 13, 2002, Kevin Johnson



“FBI Director Details Blunder on McVeigh R d ”Records”

Th FBI di l d th t it itt d " i "The FBI disclosed that it committed a "serious error" when it failed to turn over thousands of pages of interview reports in the 1995 Oklahoma City p ybombing case to lawyers for Timothy J. McVeigh, who was sentenced to death for the crime.

The New York Times, May 17, 2001, David Johnston


Data Quality In the News

• United Nations Resolution 1441 mandated that Iraq declare any current holdings of weapons of massdeclare any current holdings of weapons of mass destruction

• Does anyone seriously think that a 12,000 page y y , p gdocument, supported by 4 CD-ROMS will be error-free?



“The Wrong Foot, and Other Tales of Surgical Error”Error

At least 500 times since 1996 surgeons in hospitalsAt least 500 times since 1996, surgeons in hospitals in this country have operated on the wrong arm, leg, eye, kidney or other body part, or even on the wrong patient. The figure does not include near-misses (when surgeons started to operate on the wrong site or patient because no one collects thiswrong site or patient because no one collects this information.


The New York Times, December 11, 2001, Lawrence K. Altman, MD

Poor Data Quality in the News“Right Drug, Wrong Patient”

In 1997, a nationwide survey conducted by the trade publication Drug Topics, 53 percent of pharmacists d i d h i d i h diadmitted having made errors in the preceding two

months. A June 1996 survey of 3361 pharmacists in California and Oregon revealed that errors goccurred at an annual rate of 324 per pharmacy –nearly one a day!

Reader’s Digest, September 1998


Poor Data Quality in the News“To Err Is Human: Building a Safer Health System”

This report initiated b the Instit te of Medicine fo nd that:This report, initiated by the Institute of Medicine, found that:• Adverse events occurred in 2.9 percent of hospitalizations in Colorado

and Utah and 3.7 percent in New York.• 8 8 percent of adverse events led to death in Colorado and Utah• 8.8 percent of adverse events led to death in Colorado and Utah,

compared with 13.6 percent in New York hospitals.• Medication errors alone, occurring either in or out of the hospital, are

estimated to account for over 7,000 deaths annually., y• More people die in a given year (between 44,000 and 98,000) as a

result of medical errors than from motor vehicle accidents, breast cancer or AIDS.

• The national costs of preventable adverse events are estimated to be between $17 billion and $29 billion.


“To Err is Human,” Committee on Quality of Health Care in America, Institute of Medicine, November 1999.


“Accounting Wars”

Between 1997 and 1999, long before the current financial reform outcry at least 364 companiesfinancial reform outcry, at least 364 companies restated financial statements.

Business Week, September 25, 2000, p.158



“The Great Divide”

The Enron scandal, on the other hand, clearly was about us It told us things about ourselves that weabout us. It told us things about ourselves that we probably should have known, but had managed not to see. I predict that in the years ahead Enron, not Sept. 11, will come to be seen as the greater turning point in U.S. society.”

The New York Times, January 1, 2002, Paul Krugman


Everyday Data Quality Disasters

• Pricing errors on the internet– United Airlines sold about 1,000 tickets for all flights , g

from and through Chicago and Denver for $5– W Hotels – New Years Eve in NYC for $25/night

i t d f $259instead of $259.– Hilton Hotels listed rooms for $0 at least 3 times in 6

months.– Amazon.com sold a memory board valued at $329.99

for $9.99 and another worth $999.99 for $10.77• Hyundai over-reported horsepower ratings by as much as

9.6% on some vehicles sold since 1992.• Bear Sterns ordered $4 billion stocks sold – order should


• Bear, Sterns ordered $4 billion stocks sold – order should have been for $4 million.


Cost-of-Poor-Data-Quality

COPDQ = Cost to Prevent Errors+ Cost to Find and Fix Errors Can (usually)

be estimated*+ Cost to Make Good for Customers+ Cost of Bad/Delayed Decisions+ Cost of Lost Opportunities

be estimated*

(Usually) morepp+ Cost of Damage to Reputation+ + +

(Usually) more important

*Doing so can be tricky. If the organization will respond to COPDQ, we recommend:

D i t th “ j t l l” fi t– Doing so at the “project level” first,– Then expanding to a supplier or information chain– Working up to the entire company.


COPDQ – The Evidence• Proprietary studies: “easy-to-measure” (finding and fixing

errors) COPDQ are 10-20% of revenue.• The Data Warehousing Institute: estimates that, in the US,

$611 billion a year is lost as a result of poor customer data (name and address data). ( )

• As a result of the embassy bombing, the US paid the Chinese government $27 million. But the real cost is much higherhigher.

• As a result of the 2000 Presidential Election, Florida spent $32 million on a new system. The federal government will spend $3 9 billion in the ne t 3 ears to help states replacespend $3.9 billion in the next 3 years to help states replace outdated voting machines and train personnel.

• The error on W Hotel’s website cost $175,266 in bookings


to one hotel. The web error was posted for only 7 days.

Data Underlie Everything We Do

Perhaps they are “hidden in plain sight”


A Database is Like a Lake

To Clean Up The Lake, One Must FirstEliminate The Sources Of Pollutant


It is so easy for accountability to shift downstream!!!downstream!!!

Here’s how youdo number 3do number 3,

soncos2(x) + sin2(x) = 1


Lead, follow, or risk being acquired

Leading organizations have improved by two orders of magnitude and reduced COPDQ by two-thirdsmagnitude and reduced COPDQ by two thirds.

Best-practice:p• Adopt the philosophy of prevention • Leadership• Management Accountability at the points of data creation• Focus on customer

M t t l i t• Measurement, control, improvement• Data Culture


A Clarion CallPoor quality data are the norm and no one is immune to their

effects. Most issues are mundane, but their impact is , pstunning. They:

• Increase cost – approximately 20% of revenue• Anger customers• Make decision-making more difficult• Make it harder to implement new technologies• Make it harder to implement new technologies• Put company image at risk


Documents

Data - The Next Quality Disaster - ASQ Baltimoreasqbaltimore.org/dt/present/Present200212_Next... · and Utah and 3.7 percent in New York. • 8 8 percent of adverse events led to