Managing Data Quality in Dynamic Decision … Data Quality in Dynamic Decision Environments: An Information Product Approach Abstract Large data volumes, widely distributed data sources

Managing Data Quality in Dynamic Decision Environments: An Information Product Approach

Working paper #2003-14

G. ShankaranarayananInformation Systems Department

Boston University School of Management595 Commonwealth Avenue, Boston, MA 02215

Phone: (617) 353 2523Fax: (617) 353 5003

Email: [email protected]

P. R. BalasubramanianInformation Systems Department


Phone: (617) 353 2523Fax: (617) 353 5003


Richard Y. WangInformation Systems Department


Phone: (617) 353 2523Fax: (617) 353 5003


Mostapha ZiadDepartment of Computer Information Systems

Suffolk University Sawyer School of Management8 Ashburton Place, Boston, MA 02108


1

Managing Data Quality in Dynamic Decision Environments: An Information Product Approach

Abstract

Large data volumes, widely distributed data sources and multiple stakeholders (data providers

and data consumers) characterize a typical e-business setting. Mobile and wireless technologies

has further increased the volume, further distributed the data sources while permitting access to

data anywhere, anytime. Such environments empower and necessitate decision-makers to

act/react quicker to all decision-tasks including mission-critical ones. Decision-support in such

environments demands efficient data quality management. This paper presents a framework for

managing data quality in such environments using the information product approach. It includes

a modeling technique to explicitly represent the manufacture of an information product, quality

dimensions and methods to compute data quality of the product at any stage in the manufacture,

and a set of capabilities to comprehensively manage data quality and implement total data quality

management. The paper also posits the notion of a virtual business environment to support

dynamic decision-making and describes the role of the data quality framework in this

environment.

2

in such environments the framework permits decision-makers to gauge quality using their own

assessment of data sources and processes. It enhances their ability to better assess quality

implications by allowing them to understand the meta-details about the data being used. In the

framework, we first define a representation scheme, the IPMAP, to systematically represent the

manufacture of an information product. We then define a set of quality dimensions and describe

methods to evaluate data quality using these. When used with the IPMAP, data quality can be

evaluated at each stage of the manufacture.

We further define a set of capabilities on the IPMAP for total data quality management.

These capabilities are defined by adapting proven techniques (CPM/PERT) from operations

management and by using graph-based operations. The graph-operations are shown to be correct.

The framework consisting of the representation technique, metadata including data quality

dimensions, and the capabilities for managing data quality, guarantees total data quality

management for IPs in organizations. We have further described the architecture for a data

quality management system that incorporates this framework. Finally, we have posited the notion

of virtual business environments as a way of supporting dynamic decision-making and illustrated

the role of data quality management in these. We believe this approach to data quality

management is necessary in dynamic decision environments. Coupled with virtual business

environments it provides comprehensive support for managing data and its quality.

References

1) --------------- , Gottlieb, R. and ------- . – Virtual Business Environments for Real-time Enterprises –Boston University School of Management Working Paper, 2002.

2) Ballou, D. P, Wang, R. Y., Pazer, H., and Tayi, G. K. – Modeling Information Manufacturing Systems to Determine Information Product Quality, Management Sciences, vol. 44, 1998, pp. 462 – 484.

3) Chase, R. B., Aquilano, N. J., and Jacobs, R. F. – Production and Operations Management: Manufacturing and Services, 8th Edition, Irwin McGraw Hill, 1998.

4) English, L. P. – Improving Data Warehouse and Business Information Quality: Methods for Reducing Costs and Increasing Profits, 1999, John Wiley and Sons Inc, New York, NY.

24

5) Even, S. - Graph Algorithms, Computer Software Engineering Series, Computer Science Press, Potomac, Maryland, 1979

6) Hernadez, M. A. and Stolfo, S. J. – Real World Data is Dirty: Data Cleansing and the Merge/Purge Problem, Journal of Data Mining and Knowledge Discovery, vol.1, no. 2, 1998.

7) Jablonski, S. and Bussler, C. – Workflow Management: Modeling Concepts, Architecture, and Implementation, International Thomson Computer Press, London, UK, 1996

8) Kahn, B. K., Strong, D. M., and Wang, R. Y. – Information Quality Benchmarks: Product and Service Performance, forthcoming in the Communications of ACM, Spring 2002

9) Lee T., Bressen, S, and Madnick, S. – Source Attribution for Querying Against Semi-structured Documents, presented at the Workshop on Web Information and Data Management, ACM Conference on Information and Knowledge Management, 1998.

10) Lee, Y. W., Strong, D. M, Kahn, B. K., & Wang, R. Y. – AIMQ: A Methodology for Information Quality Assessment, forthcoming in Information and Management, 2002

11) Litvak, E. – The Program for the Management of Variability in Health Care Delivery, Boston University Health Care Program Note, 2001

12) Manber, U. - Introduction to Algorithms - A Creative Approach, Addison Wesley Publications, Reading, MA, 1989

13) Mango, P. D and Shapiro, L. D. – Hospitals Get Serious About Operations – McKinsey Quarterly, 2001, pp. 74-85

14) Morey, R. C. – Estimating and Improving the Quality of Information in the MIS, Communications of ACM, vol. 25, no. 5, 1982, pp. 337-342

15) Parssian, A, Sarkar, S., & Jacob, V.S.- Assessing Data Quality for Information Products, in the Proceedings of the International Conference on Information Systems (ICIS’99), 1999

16) Redman, T.C. (Ed.) – Data Quality for the Information Age, Artech House, Boston, MA, 1996.17) ------------------------------. – IPMAP: Representing the Manufacture of an Information Product – in the

proceedings of the MIT Information Quality Conference (IQ 2000), Boston, MA, 2000.18) Wand, Y. and Wang, R. Y. – Anchoring Data Quality Dimensions in Ontological Foundations,

Communications of ACM, November 1996.19) Wang, R. Y., Lee, Y. W., Pipino, L. and Strong, D. M. – Manage your Information as a Product,

Sloan Management Review, vol. 39, no. 4, 1998, pp. 95-105.20) Wang, R. Y., Kon, H. B., and Madnick, S. E. – Data Quality Requirements Analysis and Modeling, in

the proceedings of the 9th International Conference on Data Engineering, Vienna Austria, 1993, pp. 670-677.

21) Yourdan, E. (ed.) – Modern Structured Analysis, Y.P.C. Series, Prentice Hall, Englewood Cliffs, NJ, 1989

APPENDIX A-1

Lemma 1: Every IPMAP generates a unique IP-graph and each IP-graph converts back to one and only one IPMAP. Stated differently, no IP-graph can represent two different IPMAPs and no IPMAP can generate two different IP-graphs.

Proof: The set P consists of ordered pairs that associate each node in an IPMAP with its corresponding IP-graph. This set is unique for each ordered triple defining the mapping associated with an IPMAP and its IP-graph. Hence by construction, every IPMAP will generate a unique IP-graph and using the reverse –mapping, the IPMAP can be obtained from the IP-graph.

APPENDIX A-2

25

References

1. Department of Statistics 1994

2. B.C. Ghosh: “Management Accounting in Singapore”, Management Accounting (UK),

December 1987

3. B.C Ghosh: “Management Accounting Practices in Singapore”, DBA Thesis and also a

report for ICPAS, Singapore, 1984/85

4. B.C Ghosh: C.Y. Wan and L.H. Chung, “Management Accounting Practices by

Manufacturing and Non-manufacturing Companies in Singapore”, Singapore Accountant,

Dec 1987.

5. J.S Chiu and D.L Chang: “Management accounting in Taiwan”, Management Accounting

(US), June 1979

6. Liu Kin-Cheung: “Management accounting in Hong Kong”, a paper presented at the S.E.

Asian University Accounting Teachers Conference, held in Singapore in April 1986

7. E.A Imhoff: “Management accounting techniques: a survey”, Management Accounting

(US), November 1978

8. G Lander : Profile of Management Accounting. NAA, NY, 1983

9. C. Jones: Financial Planning and Control: A Survey Of Financial Control Practices By UK

Companies. CIMA, London, 1980

10. C. Drury and D. Dungale: “Surveys of management accounting practice”, Management

Accounting Handbook, 1992