Performance Assessment

Min Song, Ph.D.Is 465

LEARNING OUTCOMES

4.1 Compare efficiency IT metrics and effectiveness IT metrics

4.2 List and describe five common types of efficiency IT metrics

4.3 List and describe four types of effectiveness IT metrics

4.4 Explain customer metrics and their importance to an organization

Key Terms

• Efficiency: doing things the right way• Effectiveness: doing the right things• Equality: doing things fairly

CHAPTER FOUR OVERVIEW

• Efficiency and effectiveness IT metrics are two ways to measure the success of IT strategic initiatives– Efficiency IT metrics – measure the performance of

the IT system itself including throughput, speed, availability, etc.

– Effectiveness IT metrics – measure the impact IT has on business processes and activities including customer satisfaction, conversion rates, sell-through increases, etc.

BENCHMARKING – BASELINING METRICS

• Benchmarks – baseline values the system seeks to attain

• Benchmarking – a process of continuously measuring system results, comparing those results to optimal system performance (benchmark values), and identifying steps and procedures to improve system performance

THE INTERRELATIONSHIPS OF EFFICIENCY AND EFFECTIVENESS IT METRICS

• Efficiency IT metrics focus on technology and include:– Throughput – amount of information that can travel through a

system at any point in time– Speed – amount of time to perform a transaction– Availability – number of hours a system is available– Accuracy – extent to which a system generates correct results– Web traffic – includes number of pageviews, number of unique

visitors, and time spent on a Web page– Response time – time to respond to user interactions

THE INTERRELATIONSHIPS OF EFFICIENCY AND EFFECTIVENESS IT METRICS

• Effectiveness IT metrics focus on an organization’s goals, strategies, and objectives and include:– Usability – the ease with which people perform

transactions and/or find information– Customer satisfaction – such as the percentage of

existing customers retained– Conversion rates – number of customers an

organization “touches” for the first time and convinces to purchase products or services

– Financial – such as return on investment, cost-benefit analysis, etc.

THE INTERRELATIONSHIPS OF EFFICIENCY AND EFFECTIVENESS IT METRICS

• Security is an issue for any organization offering products or services over the Internet

• It is inefficient for an organization to implement Internet security, since it slows down processing time. However, to be effective it must implement Internet security. – Secure Internet connections must offer encryption and

Secure Sockets Layers (SSL denoted by the lock symbol in the lower right corner of a browser)

THE INTERRELATIONSHIPS OF EFFICIENCY AND EFFECTIVENESS IT METRICS

• The interrelationships between efficiency and effectiveness

Evaluating Efficiency-Effectiveness-Equality Trade-Offs: A Data Envelopment Analysis Approach Boaz Golany, Eran TamirManagement Science, Vol. 41, No. 7 (Jul., 1995), pp. 1172-1184

BENCHMARKING – BASELINING METRICS

• E-governement benchmarks for IT efficiency and effectiveness

DETERMINING IT EFFICIENCY AND EFFECTIVENESS

• Customer metrics – assess the management of customer relationships by the organization and include:– Market share– Customer acquisition– Customer satisfaction– Customer profitability

Web Traffic Analysis

• Most companies measure the traffic on a Web site as the primary determinant of the Web site’s success

• However, a large amount of Web site traffic does not necessarily equate to large sales

• Many organizations with high Web site traffic have low sales volumes

Web Traffic Analysis

• Web site traffic analysis can include:– Cookie – a small file deposited on a hard drive by a

Web site containing information about customers and their Web activities

– Click-through – a count of the number of people who visit one site and click on an advertisement that takes them to the site of the advertiser

– Banner ad – a small ad on one Web site that advertises the products and services of another business, usually another dot-com business

– Interactivity – visitor interactions with the target ad

Behavioral Metrics

• Click-stream data tracks the exact pattern of a consumer’s navigation through a Web site (or other information space)

• Click-stream data can reveal:– Number of pageviews– Pattern of Web sites visited– Length of stay on a Web site– Date and time visited– Number of customers with shopping carts– Number of abandoned shopping carts

Behavioral Metrics

• Visitor Web site metrics

Behavioral Metrics

• Exposure, visit, and hit Web site metrics