CIO Executive Board - IT Balanced Scorecards

IT Balanced ScorecardsEnd-to-End Performance Measurement for the Corporate IT Function

Working Council for Chief Information Offi cers

� Drivers of Balanced Scorecard Adoption in IT

� Best-in-Class IT Balanced Scorecard Metrics

� Principles of Balanced Scorecard Design and Metrics Selection

� Scorecard Development and Life-Cycle Management

© 2003 Corporate Executive Board

Working Council StaffExecutive Director

Chris Miller

Managing DirectorJaime M. Capellá

Practice ManagerKris van Riper

Project ManagersAndrew Horne • Matt McWha

ConsultantsJames Bilodeau • Carsten Schmidt

Senior AnalystEric Tinson

AnalystsKiran Mishra • Michael Scutari

Rich Flanagan • Brett Neely

Senior DirectorBrian Foster

DirectorsSheldon Himelfarb • Matt Kelly

Associate DirectorsStuart Roberts • Ken Rona • Audrey Taylor

Note to Members

This project was researched and written to fulfi ll the research requests of several members of the Corporate Executive Board and as a result may not satisfy the information needs of all member companies. The Corporate Executive Board encourages members who have additional questions about this topic to contact the Board staff for further discussion. Descriptions or viewpoints contained herein regarding organizations profi led in this report do not necessarily refl ect the policies or viewpoints of those organizations.

Confi dentiality of Findings

This project has been prepared by the Corporate Executive Board for the exclusive use of its members. It contains valuable proprietary information belonging to the Corporate Executive Board and each member should make it available only to those employees and agents who require such access in order to learn from the material provided herein, and who undertake not to disclose it to third parties. In the event that you are unwilling to assume this confi den-tiality obligation, please return this document and all copies in your possession promptly to the Corporate Executive Board.

Creative Solutions GroupLead Graphic Design Specialists

Jon Prinsky • Elizabeth Sugerman

Publications EditorDave Engle

Catalog No.: CIO1L9VDH

Legal Caveat

The Working Council for Chief Information Offi cers has worked to ensure the accuracy of the information it provides to its members. This report relies upon data obtained from many sources, however, and the Working Council for Chief Information Offi cers cannot guarantee the accuracy of the information or its analysis in all cases. Further, the Working Council for Chief Information Offi cers is not engaged in rendering legal, accounting, or other professional services. Its reports should not be construed as professional advice on any particular set of facts or circumstances. Members requiring such services are advised to consult an appropri-ate professional. Neither the Corporate Executive Board nor its programs is responsible for any claims or losses that may arise from (a) any errors or omissions in their reports, whether caused by the Corporate Executive Board or its sources, or (b) reliance upon any recom-mendation made by the Working Council for Chief Information Offi cers.


2000 Pennsylvania Avenue NW

Washington, DC 20006

Telephone: 202-777-5000

Facsimile : 202-777-5100

166 PiccadillyLondon, W1J 9EFUnited KingdomTelephone: +44-(0)20-7499-8700Facsimile: +44-(0)20-7499-9700

www.cio.executiveboard.com

ii

Executive Summary • iv

Member Self-Diagnostic • vii

An Exhaustive Compendium of IT Balanced Scorecard Metrics • 1

Introduction: Principles of Balanced Scorecard Design and Metrics Selection • 9

Best-in-Class IT Balanced Scorecard Metrics • 21

Financial Performance • 22

Project Performance • 26

Operational Performance • 30

Talent Management • 34

User Satisfaction • 40

Information Security • 46

Enterprise Initiatives • 50

Scorecard Development and Life-Cycle Management • 55

Scorecard Rollout • 56

Data Collection and Quality Assurance • 58

Scorecard Review and Revision • 60

Facilitating Scorecard Adoption • 62

Appendix I: Collected IT Balanced Scorecards • 67

Appendix II: IT Balanced Scorecard Tools and Vendors • 91

Order Form • 97

In-Person Research Presentations • 98

Working Council Project Support Desk • 99

Table of Contents

iii© 2003 Corporate Executive Board

Institutionalizing IT Balanced Scorecards

Executive Summary

Multiple Drivers of Greater IT Performance Scrutiny

IT organizations are increasingly adopting the balanced scorecard as a management tool incorporating fi nancial, operational, talent management, project management, and user satisfaction perspectives into assessments of the performance of the function. This responds to two major categories of challenges:

1. Business-Centric Challenges—These include the CIO’s need to link IT strategy with business strategy, to monitor service levels while at the same time reducing expenses, and to better demonstrate the business value of IT.

2. IT-Centric Challenges—These include the CIO’s need to better manage IT’s human capital and customer satisfaction, to baseline IT’s performance with respect to external providers, to better manage historically politicized resource allocation decisions, and to correct chronic project underperformance.

Design Principles of Exemplar IT Balanced Scorecards

Based on an extensive review of IT balanced scorecards collected from corporate exemplars, the Working Council fi nds that the most advanced scorecards share six key structural attributes:

1. Simplicity of Presentation—Exemplar enterprise IT balanced scorecards are limited to a single page of 10 to 20 metrics that communicate performance in concise, non-technical language for consumption beyond the IT function.

2. Explicit Links to IT Strategy—Best-in-class IT balanced scorecards are tightly linked with the outputs of the annual IT strategic planning process and help track progress against IT’s key goals and objectives.

3. Broad Senior Executive Commitment—Exemplars involve a mix of senior IT and business leadership in balanced scorecard design, metrics selection, and regular review.

4. Enterprise-Standard Metrics Defi nitions—Progressive practitioners achieve consensus on and clearly document scorecard metrics defi nitions, allowing review meetings to focus on decisions rather than debate over the composition or relevance of individual metrics.

5. Drill-Down Capability and Available Context—To supplement a high-level view of IT performance and allow for the detailed review of trends or variance, exemplar balanced scorecards provide visibility into the component elements, source, and context of scorecard metrics.

6. Individual Compensation Linked to Scorecard Performance—To facilitate adoption of the IT balanced scorecard and ensure stakeholder accountability, leading companies link achievement of scorecard targets to individual compensation.

Seeking a More Balanced View of IT Performance

Progressive IT balanced scorecard practitioners track metrics in fi ve key categories, seeking to “balance” traditional supply-side operational metrics with demand-side measures such as customer satisfaction. The fi ve key metrics categories are:

Financial Performance—A set of granular fi nancial metrics allows CIOs to understand IT spending in the context of service levels, strategy implementation, and project progress.

Project Performance—Spurred by a legacy of failed projects and chastened by the reality that large-scale initiatives are most at risk, exemplars utilize the balanced scorecard as a vehicle for tracking of project progress and status, with a focus on enterprise projects.

iv


Executive Summary (continued)

Institutionalizing IT Balanced Scorecards

Operational Performance—Instead of concentrating measurement efforts on system-specifi c, day-to-day IT operational metrics, best-in-class balanced scorecard practitioners seek to provide an aggregate, customer-focused view of IT’s operations.

Talent Management—Seeking to better manage the IT organization’s human capital, progressive scorecard practitioners track IT staff satisfaction and retention, as well as the attractiveness of the IT department on the external IT skills market.

User Satisfaction—By regularly assessing end-user satisfaction, CIOs can more easily identify service delivery problems and more accurately articulate cost and service quality trade-offs for consideration by business decision makers.

Additional Categories in the Ascent

Exemplars include supplemental metric categories to better understand IT’s performance in two critical areas:

Information Security—The recent surge in hacks and viruses, coupled with increased geopolitical tensions, has led advanced IT balanced scorecard practitioners to elevate information security to the category level, using the scorecard to monitor remediation efforts for known vulnerabilities and track proactive policy and certifi cation efforts.

Enterprise Initiatives—Progressive IT departments are also using their balanced scorecards to highlight IT’s contributions to initiatives of corporate strategic importance (for example, IT integration work in service to post-merger synergy targets).

Scorecard Life-Cycle Management

After designing the IT balanced scorecard and selecting the appropriate metrics, advanced practitioners develop a handful of critical competencies to manage the IT balanced scorecard across its “life cycle”:

Performance Transparency—To facilitate achievement of targets and increase visibility into IT performance, best-in-class scorecard users stage regular executive-level reviews of metric status and gap to goal.

Formalized Data Collection Roles and Responsibilities—Exemplars are creating clearly delineated collection processes and roles to ensure data freshness and accuracy.

Closed-Loop Scorecard Revision—Progressive scorecard practitioners create a closed-loop process for updating categories and metrics as business and IT strategies change.

An IT Balanced Scorecard Deployment Toolkit

The Working Council has included three appendices in this brief to assist IT organizations in developing and deploying IT balanced scorecards. The fi rst reproduces the IT balanced scorecards used by the companies profi led in this brief to provide members with an illustrative sample of scorecard presentation formats. In addition, we have included a compendium of metrics, organized by scorecard category, to allow members to qualitatively benchmark themselves against other scorecard practitioners or compress the cycle time required to generate candidate metrics for newly created scorecards. The third appendix contains an overview of the major vendors of balanced scorecard software solutions, designed to help IT executives exploring the purchase of one of these tools better understand potential solutions.

v© 2003 Corporate Executive Board v

vi


vii© 2003 Corporate Executive Board

Diagnostic Evaluation

If four or more “No” answers, then adoption of an IT balanced scorecard may facilitate IT performance management at your organization.

Member Self-Diagnostic

Assessing the Need for an IT Balanced ScorecardThe following questions are intended to assist CIOs in diagnosing whether an IT balanced scorecard would be a useful addition to their current IT performance management framework.

1. Can I clearly articulate the link between IT operational and project activities and the organization’s stated strategic business goals?

2. Is there a process or mechanism in place to track the impact on service levels and satisfaction of ongoing cost-effi ciency efforts?

3. Can I describe the performance of the IT function in a concise, non-technical, business-friendly fashion?

4. Can I effectively communicate the value that IT creates for the business?

5. Can I communicate a holistic perspective of IT performance consistently across various geographies and business units?

6. Can I easily compare the performance of my IT function to that of industry competitors or companies with similar geographic dispersion or scale?

7. Do IT performance management meetings focus almost solely on discussions of metric comparability and validity rather than on making resource allocation decisions?

8. Do I have a suffi cient understanding of the progress and status of ongoing IT project work to allow for corrective action if major projects are at risk for scope creep, budget overruns, or schedule delays?

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Subtotal “No” _______ Subtotal “No” _______

Total “No” _______

viii


Member Self-Diagnostic (continued)

Structuring the Scorecard

1. Does the IT balanced scorecard fi t onto a single page (or screen)?

2. Are the IT balanced scorecard’s metrics devolved from the goals articulated in the IT strategic plan?

3. Are these metrics expressed in non-technical language, allowing business decision makers to easily understand IT performance?

Selecting Scorecard Categories

4. Does the IT balanced scorecard supplement fi nancial and operational metrics with categories that track project performance, user satisfaction, and talent management?

5. Does the IT balanced scorecard also include categories for information security?

6. Does the scorecard outline target levels for each metric that have been agreed upon by both senior IT and business leadership?

7. Are scorecard categories, metrics, and weightings revisited on an annual cycle, to ensure continued relevance to changing business needs?

Selecting Scorecard Metrics

8. Do fi nancial metrics move beyond simple reporting of total IT spend to help decision makers reallocate IT funding between functional areas, business units, and portfolio categories?

9. Are operational metrics aggregated to provide decision makers with a “user’s perspective” of IT performance?

10. Does the balanced scorecard’s project performance category include an assessment of compliance with enterprise architecture goals and contribution to corporate business strategies?

11. Do measures of customer satisfaction incorporate both end-user and executive perspectives on IT performance?

12. Do talent management metrics focus on gauging staff satisfaction, external reputation of the IT organization, and other organizational attributes likely to make the company a destination for high-potential IT talent?

13. Does the balanced scorecard build awareness of information security issues by providing senior decision makers with an assessment of the organization’s vulnerability?

14. Are metrics designed to track IT’s contribution to major enterprise initiatives aggregated in a single scorecard category, allowing business sponsors to quickly assess IT’s level of support?

Designing and Maintaining a World-Class IT Balanced ScorecardTo assist member companies in the initial design and implementation of IT balanced scorecards, as well as the reevaluation of existing scorecards, the Working Council has created the following diagnostic questionnaire.

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Subtotal “Yes” _______ Subtotal “Yes” _______

ix© 2003 Corporate Executive Board

Diagnostic Evaluation

Total Number of “Yes” Answers Assessment

1–8 IT Balanced Scorecard Novice

9–16 Developing IT Balanced Scorecard Competency

17–22 Balanced Scorecard Exemplar

Member Self-Diagnostic (continued)

Ensuring Data Accuracy and Relevance

15. Do all metrics have clear, well-documented defi nitions, agreed upon by senior IT and business leadership?

16. Does each IT balanced scorecard metric have a defi ned collection frequency (e.g., monthly, quarterly, annually) based on the volatility of the business strategy it helps enable?

17. Have all scorecard metrics been assigned to a metric owner whose compensation is based on his or her timely delivery of required data?

18. Is data accuracy verifi ed by local metrics experts before data are published to the scorecard?

19. Does the IT balanced scorecard provide readers with the ability to drill down into the data underlying scorecard metrics?

20. Does the IT balanced scorecard provide readers with context for changes in performance (for example, historical reference data, external benchmarks, or metric owner comments)?

Facilitating Management Decision Making

21. Have business decision makers, metrics owners, and IT leaders received training on the basic concepts and uses of balanced scorecards?

22. Is the IT balanced scorecard reviewed on a regular basis by IT and business executives senior enough to make decisions based on scorecard information?

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Subtotal “Yes” _______ Subtotal “Yes” _______

Total “Yes” _______

Designing and Maintaining a World-Class IT Balanced Scorecard

Yes No

Yes No

Process Problems Hinder Successful Scorecard Deployment

Working Council research into the IT balanced scorecard deployments at dozens of large organizations reveals that those efforts often fall prey to a set of seven “deadly sins” of balanced scorecard design and use:

1. An IT-Centric View of IT Performance—Failure to involve senior business decision makers in metrics design and selection leads to an operations-biased perspective of IT performance.

2. Measures That Don’t Matter—Poor linkage between IT balanced scorecard metrics and articulated IT strategies results in strategic “drift” between day-to-day IT work and the needs of the business.

3. No Common Ground—The lack of a standard set of metrics defi nitions leads to divisional, regional, business unit, and local variation, complicating scorecard data aggregation.

4. Overreliance on Tools—Focus on the rollout of data collection tools at the expense of creating a clearly articulated data collection process results in inaccurate, outdated scorecard information, hindering effective decision making.

5. Lack of Drill-Down Capability and Metric Context—Limited access to detailed data and contextual information underlying scorecard metrics stymies trend or variance analysis and complicates scorecard interpretation.

6. Too Many Metrics—A surplus of metrics overwhelms the scorecard reader and leads to suboptimal use of senior decision-makers’ limited time.

7. No Individual Impact—The absence of incentives linking individual behavior to IT balanced scorecard use hampers scorecard adoption and achievement of targets.

x


Roadblocks to Successful Scorecard Use

xi© 2003 Corporate Executive Board

Seven Deadly SinsCommon Pitfalls Encountered Along the Balanced Scorecard Life Cycle

Source: Working Council research.

Nature of Problem

Strategic

Tactical

Stage of IT Performance Measurement Process

Selection Collection Reporting Usage

An IT-Centric View of IT Performance: Lack of senior business executive involvement in metrics selection and refi nement

No Common Ground: Lack of standard metrics defi nitions complicates aggregation

Overreliance on Tools: Lack of focus on data collection process leads to inaccurate, outdated data

No Individual Impact: Individuals lack incentive to infl ect scorecard performance

Measures That Don’t Matter: No explicit link between metrics and IT strategy

Too Many Metrics: Lack of aggregation and screening of low-level metrics, resulting in cumbersome reports

Solution:

p. 60

Solution:

p. 56

p. 24

p. 58

p. 44

p. 32

p. 62

Solution:

Solution:

Solution:

Solution:

Solution:

Lack of Drill-Down Capability and Metric Context: Unavailable context for scorecard-level metrics hinders interpretation

1

2

3

4

5

6

7

xii


An Exhaustive Compendium of IT Balanced Scorecard Metrics

Financial Performance Metrics

Project Performance Metrics

Operational Performance Metrics

Talent Management Metrics

User Satisfaction Metrics

Information Security Metrics

1© 2003 Corporate Executive Board


IT Balanced Scorecards 2

Balanced Scorecard Metrics

Financial PerformanceMost Common Metrics IT Departmental Cost

Total IT expenditures IT cost per employee

Percentage of IT expenditures delivering new functionality Total IT spending by geography

Percentage of “lights on” operating costs (including break/fi x, depreciation) versus total IT spend Total IT spending by business unit

Project and Investment Cost Performance Expenses compared to revenue per quarter

Percentage of R&D investment resulting in operational applications Year-to-date net book expense

Total value creation from IT enabled projects Spend per portfolio category

Percentage differential in business case estimate and actual benefi ts of projects Performance against IT spending performance

Percentage of key strategic projects initiated with cost/benefi t analysis Central IT spend as percentage of total IT spend

Systems and Services Cost Net present value delivered during payback period

Dollar value of technology assets still in use beyond depreciation schedule

Share of discretionary spending shared by IT

Percentage reduction in maintenance cost of all systems

Average network circuit cost reduction per quarter

PC/laptop software maintenance cost per month per user

Workstation software maintenance cost per month per workstation

E-mail service: cost per month per user

Infrastructure spending

Total maintenance cost

Individual systems cost

Dollars saved through vendor reviews and negotiations

Percentage of year-over-year cost reduction per service

Total cost of ownership of IT services versus external benchmarks

Service unit cost

© 2003 Corporate Executive Board 3


Project PerformanceMost Common Metrics Project Alignment with IT Strategy

Percentage of projects on time, on budget Percentage of project requirement fulfi lled via reuse

Percentage of projects compliant with architectural standards Percentage of applications deployed on a global basis

Project Spending and Costs Percentage of infrastructure standardization projects of total project pool

Total spent on non-compliant projects as a percentage of total project spending Percentage of projects using common project methodology

Actual versus planned ROI for implementation of key initiatives Percentage of application failures within fi rst 90 days of deployment

Percentage of projects with completed business case Percentage of “at-risk” projects that adopt quality, security, and compliance standards

Percentage of budget allocated to unplanned projects Increase in project management maturity

Share of technology planning projects evolving into full projects Project quality index

Percentage of projects managed with detailed budget data Percentage of projects with completed requirements and architecture document

Project Timeliness and Delivery

Average project duration

Percentage of projects with detailed project plan

Dollars saved through productivity improvement and reusable code

Percentage of projects started on time

Percentage of project milestones delivered




Operational Performance LargerMost Common Metrics IT Inventory and Contracts

Key applications and systems availability Complete review of global licenses

Help-desk fi rst-call resolution rate Quarterly partner/supplier ratings and in-person review sessions established with all major suppliers and partners

User-Centric Operational Performance Assets of terminated users placed in global IT asset pool—asset reuse measured

Global desktop availability (aggregate e-mail/servers/LAN/WAN) Percentage of applications purchased versus built

Average number of incidents per user per month (average number of times end user experiences global desktop availability outages per month)

Inventory accuracy

Consistently available and reliable IT services to users Help-Desk Performance

Rate of failure incidents impacting business Mean time to repair for all network outages less than 100 minutes

Network and Systems Performance Mean time to repair for all application systems outages less than four hours

Print server availability Percentage of infrastructure service requests closed within service level agreements

All critical systems and infrastructure have viable business continuity plans by end 2002 Operational Strategy Adoption

System/application database maintained with more than 95 percent accuracy Completion of service transformation with minimum business disruption

Activity increase on the hub All announced changes completed within advertised downtime window

E-mail transmit less than 20 seconds (all regions) All IT provisioning decommissioned for terminated users performed in less than 72 hours after notifi cation by HR

Monthly average of network availability consistently more than 99.5 percent Network access terminated for terminated users in less than 72 hours after notifi cation by HR

Monthly average of critical systems availability consistently above 99.5 percent Percentage of IT architectural plans approved, reviewed, and accepted by business

Mean time to repair for all client outages less than two hours Number of applications used by more than one line of business

Network uptime Percentage of desktop PC standardized

CRM availability End-to-end availability for customer service

PC/laptop hardware fi x or replacement within 48 hours IT effectiveness in resource allocation supporting business objectives

Total cost of ownership of identifi ed products and services compared to industry standards IT solutions meet business needs

Identify and manage strategic alliances with IT partners

Performance of shared services

Decrease average development cost by 10 percent

Percentage of consumer orders processed online



Talent Management

Most Common Metrics Training and Personal Development

Employee morale/satisfaction (multiple point scale, low to high) Percentage of employees who have met with direct manager at least once within the past month

Percentage of individual annual career plan goals met Percentage of performance assessment and development plans delivered to employees

Overall IT staff retention and attrition rate Percentage of employees with mentors

Staffi ng Percentage of employees with individual development plans

Percentage of non-entry-level position fi lled internally Percentage of individual training objectives met

Average tenure of solid performers (in years) Employee “business knowledge” survey performance

Percentage of projects led by non-project managers Percentage of managers trained in employee motivation

Percentage of projects assignments that are cross-functional Percentage of staff with appropriate measures for their personal goals

Ratio of skills sets needed to skills set represented Percentage of IT employees who attended one location- and/or function-specifi c town meeting per quarter

Performance against staff diversity goals Share of IT training spent in business units

Number of candidates interviewed per open position Number of IT person-hours spent at industry events

Job changes (counting moves transferring out) national versus international Number of training hours per employee per quarter

IT headcount (number of full-time IT staff) Corporate Strategy–Related Metrics

Contractor headcount Number of awards won by company for use of IT

Number of candidates interviewed per open position Competitiveness of current employment offer versus industry

Percentage of planned staffi ng levels Citation of IT organization in press

Average years of IT experience




User SatisfactionMost Common Metrics Survey Questions

Overall end-user satisfaction survey Perceived versus actual price competitiveness of IT services (1 to 10 scale, from user survey)

Help-desk fi rst-call resolution rate Perceived ability to deliver technical/business solutions and services (multiple point scale)

Surveys Quality of communication about available services and new technologies (multiple point scale)

Overall business executive satisfaction rating Help-desk client satisfaction—percentage dissatisfi ed

Non-Survey Metrics Satisfaction with individual operational services, e.g. voice services, network infrastructure

Percentage of hardware service requests closed within 48 hours Contribution to business process improvement

Percentage of software service requests closed within 24 hours Contribution to business value creation

Degree of alignment of IT services with articulated business priorities Contribution to business competitive advantage

Help-desk tickets per user per month Contribution to corporate business strategy

Percentage of service level agreements not met


Information SecurityMost Common Metrics

Percentage of systems compliant with IT security standards

Number of security incidents in operational systems or infrastructure which lead to material loss

Time to respond to incidents

Percentage of network access points covered by intrusion detection systems

Percentage of external partners in compliance with security standards

Percentage of security patches for client and server applications applied within deadline

Percentage of high-level security breaches dealt with in set time

Percentage of new initiatives that receive security and compliance sign-off

Number of security manager training programs

Percentage of systems compliant with IT security standards

Virus containment 100 percent (zero intrusion)

Percentage of hardware updated with latest virus patch in less than 24 hours after update release


8


IntroductionPrinciples of Balanced Scorecard Design and Metrics Selection




A Balanced Model for Corporate Performance Measurement

The Balanced Scorecard Fills a Need For More Holistic Corporate Performance Management

In their January 1992 Harvard Business Review articles, “The Balanced Scorecard: Measures That Drive Performance,” Robert S. Kaplan and David P. Norton introduced the concept of the Balanced Scorecard to businesses as an alternative to the conventional fi nancial-only view of corporate performance. Organizations have embraced the balanced scorecard concept, which supplements fi nancial performance with metrics tracking customer satisfaction, process performance, and learning and growth to create a more holistic approach to corporate performance management. According to Bain & Company, 50 percent of Fortune 1,000 companies currently use balanced scorecards to manage organizational performance.

IT-Specifi c Balanced Scorecards Becoming More Prevalent

With the current economic downturn, CIOs report more pressure to cut costs and demonstrate how the existing IT budget is being spent. These twin drivers have led CIOs to reexamine ways to more effectively measure the cost and service performance of the IT function. In response, many are deploying balanced scorecards to capture a more business-focused view of performance. A 2002 Survey by CIO Insight magazine of 357 senior IT executives found that 24 percent were using an IT balanced scorecard and that another 30 percent were considering deploying one in the future. The 2003 CIO Insight survey of 345 senior IT executives found that one year later this prediction had come to pass, with 30 percent of IT departments using an IT balanced scorecard. This growing adoption of the balanced scorecard as an IT performance management tool is underscored by the Working Council’s 2003 Research Agenda Poll, which reveals that 39 percent of member IT organizations have deployed an IT balanced scorecard.

© 2003 Corporate Executive Board Introduction: Principles of Balanced Scorecard Design and Metrics Selection 11

Growing Scorecard Adoption in IT

Harvard Business Review

1992

…large companies have adopted thebalanced scorecard at the corporate level…

Percentage of Fortune 1,000 Companies Using Balanced Scorecards1999 Bain & Company’s CEO Management Tools and Techniques Survey

50%50%

…and are now increasingly adopting scorecards in IT in response to performance pressure

Kaplan and Norton introduce the Balanced Scorecard concept in an Harvard Business Review article

24%

30%

2002 2003

Source: Bain & Company.

50 percent of Fortune 1,000 companies have adopted a corporate balanced scorecard

Source: CIO Insight .

Since its creation in 1992 as a more holisticcorporate performance management tool…

Balanced Scorecard Use in IT DepartmentsSurvey of More Than 345 Senior IT Executives

IT Balanced Scorecard Adoption Among Working Council MembersSurvey of 124 CIOs

Source: Working Council 2003 Agenda Poll.

39%61%

Percentage of Working Council members not using an IT balanced scorecard

Percentage of Working Council members using an IT balanced scorecard



Many Roads Lead to Balanced Scorecards

Balanced Scorecards as a Multifaceted IT Management Tool

During the course of its research, the Working Council has observed seven key drivers of IT balanced scorecard adoption, including both pressures from IT’s business “clients” and challenges endemic to IT:

Business-Related Drivers

1. Linking IT Strategy with Business Strategy—IT departments frequently lack an understanding of how their efforts enable corporate strategy. The IT balanced scorecard can help CIOs be more responsive to corporate strategy on an ongoing basis by highlighting IT metrics that are directly linked to corporate strategic goals.

2. Monitoring Service Levels While Cutting Expenses—Across the past several years, most CIOs have seen slowing growth or reductions in IT budgets, while at the same time meeting service levels becomes more critical as more business processes are enabled by IT. IT balanced scorecards provide a vehicle to help CIOs balance cost cutting and service delivery needs.

3. Demonstrating IT Value to the Business—Reviews of IT operational measures are not always effective in communicating IT’s impact on the business to project sponsors or business executives. As a result, articulating why a signifi cant portion of the company’s spending is consumed by IT, or why IT might need to reduce its budget at a slower rate than other functional areas or business units, is a challenge for most CIOs. By tracking granular, business-focused metrics, the IT balanced scorecard can help CIOs communicate IT’s performance to the rest of the organization.

IT-Specifi c Drivers

4. Providing a Holistic View of IT Operations—CIOs need to ensure they run the IT function with the same business discipline that characterizes other corporate functions. While operational dashboards provide a snapshot of the performance of specifi c systems, IT balanced scorecards provide CIOs with a more holistic presentation of the performance of the IT organization’s resources and assets, including staff and end-user satisfaction.

5. Baselining IT’s Performance with Respect to External Providers—As companies increasingly bid work competitively with outsourcers, internal IT organizations are being asked to improve their IT operational and project performance to compete with third parties for available work. Balanced scorecards provide IT with a tool for assessing baseline IT performance against vendor cost and quality benchmarks, and for tracking corporate IT progress against gap-closing goals.

6. Depoliticizing Resource Allocation Decisions—IT resource allocation decisions at the majority of organizations are often made lacking complete, accurate information about project costs, benefi ts, and risks. The IT balanced scorecard provides a set of agreed-upon, up-to-date, standard metrics for measurement of IT service quality and performance, facilitating less politicized funding trade-offs.

7. Correcting Chronic Project Underperformance—Most IT organizations are plagued by projects that fail to deliver—the Standish Group’s 2002 CHAOS survey reveals that 90 percent of projects larger than $3 million fail. The IT balanced scorecard provides a vehicle for tracking project progress and status, as well as progress on IT initiatives, such as systems retirement, allowing CIOs to make mid-course corrective decisions or cancel projects when necessary.


Providing a Holistic View of IT Operations• Problem: IT function traditionally

neglects staff management and satisfaction performance

• Balanced scorecard tracks all key organizational resources and assets

Q: Does Your IT Department Regularly Measure Customer Satisfaction with IT Services?

Survey of 539 IT Executives

Baselining IT’s Performancewith Respect to External Providers• Problem: IT organizations lack data

to compare their performance against that of outside service providers

• Balanced scorecard provides baseline performance, which can be used to assess feasibility of outsourcing

Share of IT Budget Allocatedto External Service Providers

Depoliticizing Resource Allocation Decisions• Problem: Resource allocation decisions

made lacking project cost, benefi t, risk information

• Balanced scorecard provides standard metrics to inform resource/quality trade-off discussions

Politicization of Prioritization Process

Survey of 1,077 CIOs, CTOs, and VPs of IT, July 2001

Correcting Chronic Project Underperformance• Problem: Little traction of IT projects• Balanced scorecard provides a single

view of overall project status and enables proactive measures to keep projects on track

IT Project Failure Rates

Survey of 35,000 IT Projects from 1994 to 2002

Linking IT Strategywith Business Strategy• Problem: Unclear

relationship betweenIT efforts and corporatestrategy

• Balanced scorecard tracksmetrics that are linkedto corporate strategy

Q: How Is Your JobPerformance Primarily Evaluated?

Survey of 388 CIOs, CTOs, and VPs of IT, November 2002

Monitoring Service Levels While Cutting Expenses• Problem: IT budgets cut while

growing share of business processes enabled by IT

• Balanced scorecard provides ability to balance cost and service quality in an informed manner

Decline in IT Capital Expenditures

Reported Capital Expenditures, 752 Corporations

Demonstrating IT Valueto the Business• Problem: Diffi culty of tracing

IT’s impact on business performance

• Balanced scorecard measures broad range of granular, business-focused metrics

Q: Has Pressure to Demonstrate ROIIncreased or Decreased in Past 12 Months?

Survey of 365 CIOs, CTOs,and VPs of IT, December 2002

Drivers of IT Balanced Scorecard Adoption

Contribution to Achieving Business Strategy

Operational Performance

Financial PerformanceInteractions with Peers,

Superiors, and Subordinates

Project Competition

42.3%

23.4%

11.9%

11.4%

5.7%

Increased

Decreased

Stayed the Same

1

2

3

Source: Goldman Sachs Global Equity Research.

2002 2003 (E)

(15%)

(10%)

60.0%36.7%

3.3%

Source: CIO Insight.

Scorecard adoption in IT is driven by both business pressures… …and IT’s own internal needs

Believe Prioritization

Process Is Depoliticized Believe

Prioritization Process Is Politicized

42% 36%

17%

4%Yes; External

Business Partners/Customers

No

Source: 2003 State of the CIO Survey, CIO Magazine.

Yes; Both Internally and Externally

Yes; Internal Employees

10%19%

1994 2003 (E)

Source: Gartner.

62%38%


Projects Larger Than $3M

All Projects

90%

23%

Source: 2002 CHAOS Survey, The Standish Group.

4

5

6


7



Clearing Up the Confusion

Moving Beyond the Aggregation of Operational Data

While the balanced scorecard is a relatively new performance measurement tool for IT groups, most IT organizations have already created some form of dashboard to track operational performance. While both tools are used to measure the performance of the IT function, dashboards and scorecards differ fundamentally across the following fi ve dimensions:

1. Purpose—The IT dashboard is a tool that provides a snapshot of current and past IT performance, and is aimed at facilitating in-the-moment corrections of emerging service problems. Consequently, dashboards frequently include automated alerting functionality. The IT balanced scorecard is optimized to track the implementation of a given set of strategies over time.

2. Audience—IT dashboards focus almost exclusively on tracking operational IT performance, catering primarily to an audience of IT operational managers. The IT balanced scorecard focuses on providing business decision makers, both within and outside of IT, with information required to make key funding and staffi ng decisions.

3. Metrics Tracked—While IT dashboards focus on tracking the performance and availability of specifi c applications or infrastructure (e.g., Web site availability or server uptime), IT balanced scorecards supplement these operational measures with aggregate metrics for tracking IT’s performance against organizational goals (for example, in the areas of talent management and customer satisfaction).

4. Data Collection Process—Dashboards are frequently bundled with the systems and services they are designed to track, with hardware vendors like Hewlett-Packard offering built-in server dashboards, and a range of third-party vendors providing similar offerings. Despite the availability of automated balanced scorecard solutions from vendors such as SAP and PeopleSoft which extract and display information from ERP and CRM systems, most companies utilize spreadsheets or homegrown reporting tools to aggregate IT balanced scorecard data. In many cases, companies also manually collect data for categories such as user satisfaction.

5. Frequency of Data Update—Dashboards track data continuously in order to create near-real-time performance visibility. In contrast, IT balanced scorecards are typically reviewed on a calendar-based cycle, and individual metrics are updated at different collection frequencies, such as quarterly or annually.


IT Operational Dashboards IT Balanced Scorecards

Purpose Operational—Real-time performance tracking and alerting

Analytical—Trend analysis and tracking of strategy execution

Audience Access generally limited to IT management

Executive-level audience, both within IT and the business

Metrics Tracked Operational performance data abouta particular system or process Performance against organizational goals

Data Collection Process

Automated data collection, in many cases integrated into monitored systems

Portions of required data collected, aggregated manually

Frequency of Data Update Continuous At set intervals (quarterly, annually, etc.), with

individual metrics updated at differing frequencies

Differentiating Dashboards and ScorecardsMigrating from “in-the-moment” operational oversight to infl uencing strategic management decisions


1

2

3

4

5

Sample Metrics

• Web server uptime• SAP availability• Help-desk fi rst-call

resolution rate

Sample Metrics

• Percentage of projects delivering new business functionality

• Global desktop availability• Percentage of applications

meeting security standards



Building a Better Scorecard

The Structure of the Ideal IT Balanced Scorecard

Based on the review of IT balanced scorecards collected from corporate exemplars, the Working Council fi nds that the most advanced IT balanced scorecards share six key attributes:

1. Simplicity of Presentation—The best enterprise-wide IT balanced scorecards are limited to a single page of metrics that communicate top-line performance in concise, non-technical language for consumption beyond the IT function. These scorecards typically showcase between 10 and 20 key business-impacting metrics to convey a high-level perspective of corporate IT’s performance.

2. Informed by the Goals of the Annual IT Strategic Plan—The most successful IT balanced scorecards are the end result of the annual IT strategic planning process, and track progress against the key goals and objectives articulated in a written IT strategic plan. To help track progress against the execution of these strategies, each scorecard metric is reported as “actual” and “target” for the current period, and is often contrasted with historical performance for the previous period.

3. Broad Senior-Level Ownership—Exemplar IT balanced scorecards are the product of cross-functional collaboration, with a mix of senior IT and business leadership involved in designing the scorecard and selecting and reviewing metrics.

4. Clearly Defi ned Metrics—Since the scorecard is used by both IT and business decision makers, establishing a common, well-documented set of metrics defi nitions is essential. By creating a shared understanding of scorecard metrics, companies can better focus review meetings on actual decisions rather than debate over the composition or relevance of individual measures.

5. Drill-Down Capability and Metric Context—While a high-level view of IT performance is the most relevant to senior decision makers, understanding and developing solutions to any problems highlighted by the IT balanced scorecard requires a more granular view of information. To allow for this, the IT balanced scorecard must provide the reader with visibility into the component elements and source of scorecard metrics as well as context around metric variance or trends in scorecard metrics.

6. Links to Individual Compensation—To facilitate the adoption of the balanced scorecard within the IT organization and provide incentive for staff to achieve scorecard targets, leading companies link scorecard performance to individual compensation.


Structural Attributes of the IT Balanced Scorecard IdealSix design principles of world-class IT balanced scorecards

Simplicity of Presentation

• Single page of key performance categories and metrics

• Non-technical language for easy consumption by business sponsors

• Limited number of metrics (10 to 20)

Informed by Goals of Annual Plan

• Categories and metrics directly linked to strategies articulated in annual IT strategic plan

• Provides insight into ongoing progress of strategy execution by tracking performance against goals

Broad Senior-Level Ownership

• Representative cross-section of senior IT and business leaders involved in scorecard creation and metrics selection

• Scorecard results are regularly reviewed by CIO and IT and business management

Clearly Defi ned Metrics

• Each metric has a clear defi nition, agreed on by IT and the business

• Companion scorecard document outlines metric defi nitions, assumptions, and collection methods

Drill-Down Capability and Metric Context

• Scorecard allows for drill down into more granular data underlying metrics

• Metrics annotated with source information and contextual explanation of variance or trends


Links to Individual Compensation

• Achievement of balanced scorecard targets linked to individual compensation of IT leadership team $

1

2

3

4

5

6



A Progression in IT Balanced Scorecard Metrics Sophistication

Cardinal Balanced Scorecard Categories

Like corporate balanced scorecards, IT balanced scorecards will differ from company to company based on corporate strategic direction. However, despite the fact that no single scorecard template can provide the optimal set of performance metrics for every IT organization, there is a set of key categories IT balanced scorecards should incorporate to create a truly balanced view of a company’s IT performance. The fi ve most common IT balanced scorecard categories include:

1. Financial Performance—Particularly in times of cost cutting, fi nance is the foundational scorecard category, allowing CIOs to understand IT costs and spending in the context of service levels, strategy, implementation, and project progress.

2. Project Performance—New application development projects, infrastructure upgrades, and systems consolidation efforts are often at risk for budget and schedule overruns. To avoid this, exemplar companies use the balanced scorecard as a mechanism for periodic tracking of the progress and business impact of their largest or most important projects.

3. Operational Performance—In many cases, companies already track day-to-day IT operational metrics using operational dashboards; exemplars use IT balanced scorecards to provide a more holistic, customer-focused view of IT’s operations (for example, system availability to end users during peak business hours).

4. Talent Management—Exemplar companies use this category to track job satisfaction and retention of key IT staff, as well as the attractiveness of the IT department on the external IT skills market.

5. User Satisfaction—By regularly assessing end-user and executive sponsor satisfaction with IT services, CIOs can more easily identify service delivery problems and more accurately articulate cost and service quality trade-offs for consideration by business decision makers.

Categories in the Ascent

In addition to the previous fi ve metrics categories, Working Council research reveals that IT balanced scorecard exemplars frequently add two categories to their scorecards:

6. Information Security—Exemplar scorecard practitioners use this category of metrics to track security breaches and calibrate the organization’s response in terms of spending on preventative measures such as security architecture and training. This is especially important given that the number of reported corporate information security breaches is steadily increasing—U.K.-based MessageLabs reports that e-mail viruses, which represent only a small fraction of all security incidents, doubled in 2002 versus 2001.

7. Enterprise Initiatives—Although the project performance category tracks large IT projects, progressive IT departments are also using their balanced scorecards to track IT’s specifi c contributions to initiatives of corporate strategic importance (for example, IT integration work in service to post-merger synergy targets). This category is often a temporary one, added to and removed from the scorecard with the ebb and fl ow of enterprise-critical initiatives.


Actual Target Status

Financial Performance• Cost of data communications per seat• Relative spending per portfolio category

$497N.A.

$450N.A.

Project Performance• Percentage of new development investment resulting in

new revenue streams• Percentage of IT R&D investment leading to IT service

improvements

65%

80%

70%

90%

Operational Performance• Peak time availability• Critical process uptime 90% 98%

Talent Management• Retention of high-potential staff• External citations of IT achievement

2%3

4%5

User Satisfaction• Entire user population• Focused executive feedback• Comprehensive perspective

3.7/596%3.3/5

3.8/598%3.5/5

Information Security• Percentage of staff receiving security training• Percentage of external partners in compliance with

security standards

40%25%

70%50%

Enterprise Initiatives• Percentage of acquired company systems integrated in

M&A category• Number of business process steps enabled by technology

in Process Reengineering category

55%

2

40%

3

Covering All the BasesThe balanced scorecard provides a holistic view of IT performance

Categories of IT Balanced Scorecard Metrics Illustrative

Financial Performance

Connecting service cost with strategy implementation and project progress to facilitate principled trade-offs


Providing a customer-focused view of IT operations

Enterprise Initiatives

Monitoring IT’s contribution to initiatives of corporate strategic importance

User Satisfaction

Identifying service delivery problems by assessing end-user satisfaction

Project Performance

Concentrating senior executive attention on the largest and most important projects

Explore Market Need

Project

Quat 1 Quat 1Quat 1Quat 1

Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan

Final Quality Assurance Testing

Explore Market Need

Explore Market Need

Final Quality Assurance TestingExplore Market Need



Explore Market Need

Explore Market NeedFinal Quality Assurance Testing

Explore Market Need


Explore Market Need

Talent Management

Assessing staff job satisfaction, retention, and the attractiveness of the IT workplace

Information Security

Focusing IT’s efforts on security spending and training


1

2

3

4

5

6

7

20



Best-in-Class IT Balanced Scorecard MetricsThe following case profi les present more detailed examinations of metrics sophistication in each of the seven IT balanced scorecard categories. A comprehensive list of scorecard metrics, organized by performance category, is included in the introduction to this brief.

Financial Performance p. 24

Project Performance p. 28

Operational Performance p. 32

Talent Management p. 36, 38

User Satisfaction p. 42, 44

Information Security p. 48

Enterprise Initiatives p. 52



Financial Discipline at the Heart of the IT Balanced Scorecard

Although the concept of the balanced scorecard was developed in order to overcome the traditional reliance on fi nancial-only metrics for performance measurement, a core set of fi nancial metrics is a key component of exemplar IT balanced scorecards. These metrics are especially critical in the current climate of stagnating or shrinking IT budgets, as accurately tracking and reporting cost and budget performance is a prerequisite for corporate IT organizations. The fi nancial IT balanced scorecard category typically exhibits three progressive levels of metrics sophistication:

An Aggregate Spending Overview—The baseline fi nancial metric for most IT balanced scorecards is some absolute measure of company total IT expenditure (for example, total IT spending or IT spending as a percentage of revenue). While important to track on an ongoing basis and an essential measure of IT’s fi nancial health, this type of metric fails to provide decision makers with actionable information and context about IT spending’s link to the business’s strategic imperatives.

Directional Granularity—Advanced IT balanced scorecard practitioners adopt a more nuanced view of IT’s fi nancial performance, delineating IT expenditures by geography, by business unit, by technology category, or by technology life-cycle stage. This provides decision makers with at least the directional ability to target the most signifi cant cost areas. Typical metrics include total infrastructure spending, operating cost, or total IT spending by business unit.

Measuring by Portfolio Mix—As a strategic performance management tool, the IT balanced scorecard needs to present the strategic impact of IT investments. Exemplar companies supplement a foundation of granular fi nancial metrics, such as total IT cost and cost of data communication per seat, with an understanding of how these costs compare to industry peers, usually using external benchmarking data. In addition, exemplars also track portfolio mix—spending per portfolio category—to anchor IT’s fi nancial performance in a broader strategic context and help decision makers assess spending in each category relative to that in other categories, as well as past spending levels.

Institutionalizing Financial Rigor in ITFinancial Performance

Best-in-Class IT Balanced Scorecard Metrics 23© 2003 Corporate Executive Board

Financial Performance Metric Maturity TrajectoryRelative Maturity of Financial Performance Metrics

Illustrative

Strength of Metric Link to Business Outcomes

Granularity of Measure

High

LowLow High


Baseline

An Aggregate Spending Overview

Sample Metrics:

• Total annual IT spending• IT spending as a percentage

of company revenue

Progressive Practitioner

Directional Granularity

Sample Metrics:

• Total IT spending by geography• Total IT spending by business unit• Total infrastructure spending

Exemplar

Measuring by Portfolio Mix

Sample Metrics:

• Cost of data communications per seat• Relative spending per portfolio

category




Company Background

Schlumberger Limited is a $13.5 billion global technology services company with 76,000 employees of more than 140 nationalities. The company has operations in 140 countries and includes three primary business segments: Schlumberger Oilfi eld Services, the world’s largest oilfi eld services company and the leading supplier of technology services and solutions to the international oil and gas industry; WesternGeco, jointly owned with Baker-Hughes, the world’s largest surface seismic company; and SchlumbergerSema, an IT services company providing consulting, systems integration, and managed services to the energy, public sector, telecommunications, and fi nance markets. Other Schlumberger businesses include Smart Cards & Terminals, the NPTest semiconductor testing solutions unit, Verifi cation Systems, and Water Services.

Scorecard Background

Schlumberger introduced its IT balanced scorecard in 2002 as part of a larger IT initiative to better understand the comparative IT spending levels and align that spending more closely with business needs.

Providing Granular Cost Transparency and Comparative Context

As a foundation for the fi nancial performance category of its IT balanced scorecard, Schlumberger tracks granular fi nancial metrics such as the cost of data communication per seat. In addition, the scorecard provides readers with an understanding of how these costs compare to those of industry peers using benchmarking data from various sources, including third-party vendors and its own IT services arm. To supplement these granular fi nancial measures, Schlumberger also tracks the percentage of IT spending dedicated to a set of eight portfolio categories, which include business productivity–oriented categories such as fi nance and

human resources, as well as enterprise-centric categories such as competitive positioning, employee effi ciency, and IT effi ciency and support. This metric of current portfolio “mix,” along with historical mix data, allows decision makers at Schlumberger to ensure that each of the portfolio categories receives suffi cient funding and to reallocate funding from one category to another to better align with shifting business strategy.

More Effective Identifi cation of Standardization and Portfolio Optimization Opportunities

One benefi t Schlumberger reports from its use of the IT balanced scorecard is that it has been able to more programmatically identify missed standardization opportunities. In one instance, a business unit manager looking at the detailed fi nancial metrics on the scorecard realized that the unit’s IT cost per seat was higher than the company average. An in-depth review of the unit’s IT cost structure revealed that it had not kept pace with the corporate infrastructure standardization initiative, resulting in higher operational costs. A second benefi t is that Schlumberger is able to make more informed portfolio decisions. For example, by reviewing the scorecard, the IT department observed that the amount of IT investment directed to customer care applications was unusually low compared to spending in other portfolio categories. This triggered a closer look at customer care spending, which revealed that the spending level appeared low for two reasons. First, the company had just completed the rollout of a major customer care application, and second, some customer care spending was being handled by the company’s business units outside of the IT budget. As a result, Schlumberger decided not to adjust spending on customer care upward, despite its initial inclination to do so. Following this early success Schlumberger’s next step in its IT balanced scorecard development is to include business metrics taken from the corporate scorecard.

Establishing a Financial Frame of ReferenceFinancial Performance


IT Spending by Portfolio CategoryIllustrative

Objective Portfolio Category Current Percentageof IT Spending

Previous Year Benchmark

Track Spending by Portfolio Category

Competitive Positioning 1% 0.5% 2%

Customer Care and Services 0.2% 1% 2%

Field Operations 10% 11% 7%

Finance 14% 13.5% 10%

HR 2% 4% 4%

Employee Effi ciency 1% 1% 5%

IT Effi ciency and Support 43% 40% 40%

Management, IT Operations, and Other 28.8% 29% 30%

Schlumberger’s IT Balanced ScorecardSample Metrics

Financial Performance• Projected cost for IT services• Year-to-date cost IT services• IT cost per seat• Cost of data communication per seat• IT spending by portfolio category

Project Performance• Average application project delay

Operational Performance• Quarterly activity increase on the corporate

knowledge portal• Percentage of standardized PCs

Resource Management• IT headcount (excluding contractors)

User Satisfaction• Tickets per registered user per month• Help-desk calls• Help-desk fi rst-call resolution rate

Information Security• Number of security breaches• Incident rate• Percentage of infrastructure protected• Percentage of sites with valid information

security audit

Baseline of granular unit cost metrics

Performance against industry benchmarks puts portfolio mix into comparative perspective

Aligning IT Spending with Business NeedSchlumberger supplements a foundation of unit cost measures with measures of IT portfolio mix

Spending tracked by portfolio category to inform resource allocation decisions

Source: Schlumberger; Working Council research.


Historical data allow decision makers to track improvements or spotlight areas of potential underspending



Monitoring Ongoing Project Performance to Avoid Surprises

A 2002 study by the Standish Group showed that out of 35,000 IT projects conducted between 1994 and 2002, only 28 percent were successfully completed, 23 percent failed, and the majority—49 percent—either ran over budget or past schedule or ended up with fewer features and functions than planned. In addition, larger projects exhibited a higher rate of failure—68 percent of projects smaller than $500,000 were fi nished successfully, while only 10 percent of projects larger than $3 million were deemed a success at completion. In addition to consuming IT resources that could be used more effectively elsewhere, these project failures—especially those of larger projects—can have serious repercussions outside of the IT organization. One example of the potential corporate impact of large IT projects that fail to deliver is NIKE’s deployment of i2’s supply chain management software. The well-publicized failure led to a one-quarter sales decrease of $100 million as a result of problems with inventory levels. Another is Hershey, whose $112 million SAP R3 implementation resulted in shipment disruptions just ahead of the 1999 holiday season and was blamed for a 19 percent reduction in third-quarter profi ts. One of the key failure points cited was a lack of disciplined project planning and management. The examples of NIKE and Hershey are extreme, but by using the IT balanced scorecard, CIOs can potentially prevent project disasters by tracking metrics at three levels:

Project Progress and Status—At a minimum, companies measuring project performance on their IT balanced scorecards include three basic metrics: the percentage of projects delivered on time, the percentage of projects delivered on budget, and the percentage of projects delivered within scope.

Post-Project Review—In addition to tracking project budget, schedule, and scope adherence, more progressive IT balanced scorecard practitioners also add metrics that highlight the performance of completed projects. These companies gather feedback from project staff on the project execution process and from the business sponsor or unit using the fi nal product on the solution’s usefulness and the service received from the project team. In addition, some IT organizations also track the failure rate of deployed technologies as a proxy for the quality of project work.

Project Contribution to Business Goals—Exemplar companies use the IT balanced scorecard to track project alignment with key business goals, supplementing in-progress and postmortem project execution information with an articulation of how specifi c IT projects enable one or more IT corporate strategic imperatives (for example, systems simplifi cation efforts or business process standardization).

Avoiding Project DisasterProject Performance


Project Performance Metric Maturity TrajectoryRelative Maturity of Project Performance Metrics

Illustrative

Metric Time Horizon

Metric Altitude Baseline

Project Progress and Status

Sample Metrics:

• Percentage of projects deliveredon time

• Percentage of projects deliveredon budget

• Percentage of projects delivered within scope


Post-Project Review

Sample Metrics:

• Sponsor satisfaction score• Early-life failure rate

Exemplar

Project Contribution to Business Goals

Sample Metrics:

• Percentage of new development investment resulting in new revenue streams

• Percentage of IT R&D investment leading to IT service improvements

Linked to Corporate

Strategy

Self-Referential

Point-in-Time Life Cycle


Project Performance



Highlighting IT’s Role as a Corporate Enabler

Company Background

Bowne & Co., a New York-based document, information management, and printing solutions provider, has $1 billion in revenues and 8,400 employees


In 2000, Bowne’s CEO engaged the services of a strategic planning consultant who advocated the creation of corporate and functional balanced scorecards. The IT balanced scorecard, which is directly mapped to Bowne’s IT strategy, has fi ve main categories: fi nancial performance, project performance, operational performance, talent management, and user satisfaction. The Excel-based scorecard includes a list of metrics and owners for each category, a clear defi nition for each metric, a description of how it will be measured, and most importantly, a list of initiatives that the IT organization must undertake in order to be able to measure performance in each category.

Spotlighting Project Impact on the Business

The project category of Bowne’s IT balanced scorecard includes several metrics that seek to spotlight the direct impact of IT projects on the business. These include measures of:

Competitive Value Creation—Gauges IT’s collaboration with each individual business unit to develop differentiating technology-enabled solutions, as measured by the percentage of new development investment that results in new corporate products or services.

External Partnership Building—Tracks IT’s collaboration with technology vendors and industry experts to identify business uses for specifi c technologies, as measured by the percentage of vendors on Bowne’s partner “ecosystem” map that the IT organization has partnered with. This map documents existing vendor relationships, current spending levels, and contacts at other organizations that Bowne wishes to partner with.

IT Value Generation—Assesses IT’s performance in realizing the potential business impact of new technologies, as measured by the percentage of technology R&D investment that leads to new IT operational services.

A Higher Corporate Profi le Pays Off

By highlighting IT’s enablement of business strategies and proactive approach to more effi cient operations, Bowne’s CIO has been able to protect the IT budget during a time of spending cuts across the company. During the 2002 budgeting process, IT was spared any cuts, while IT capital expenditures actually increased by 75 percent.

Project Performance


Bowne & Co.’s IT Balanced ScorecardSample Metrics

Financial Performance• Percentage year-over-year operational savings for specifi c IT

services• TCO of IT products and services versus industry benchmarks• Percentage variance in actual return versus business case estimate

Project Performance

Operational Performance• System availability• Number of help-desk calls per application• Percentage of existing systems conforming to architectural

standards

Talent Management• Percentage of individual training objectives met• Product and customer knowledge survey performance• Percentage of employees with individual development plans

User Satisfaction• Business sponsor satisfaction score• End-user satisfaction score

Project PerformanceIllustrative

Objective Metric Current Performance Initiatives

Competitive Value Creation—Partner with business to seize market opportunities by deploying timely, cost-effective, and integrated new solutions

Percentage of investment in IT new product development resulting in new production products or services(Jim Smith*)

30%

Develop a new product development life-cycle process (Amy Johnson*)

Incorporate integration standards(Amy Johnson)

External Partnership Building—Create alliances and partnerships to capitalize on new technologies

Percentage “strategic” partnership coverage (Sue Miller*)

40%

Develop “supplier interdependency” map(Sue Miller)

Incorporate supplier contribution on metric reporting process(Amy Johnson)

IT Value Generation—Recognize the potential business value in new technologies

Percentage of IT R&D investment resulting in new IT operational services(Jim Smith)

30%

Develop fi ve-year technology road map (including document life cycle and infrastructure) (Jim Smith)

Ensuring Project Contribution of Business Benefi tsBowne’s IT organization links IT project work with corporate benefi ts

Explore Market Need

Project

Quat 1 Quat 1Quat 1Quat 1

Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan


Explore Market Need

Explore Market Need

Final Quality Assurance TestingExplore Market Need



Explore Market Need

Explore Market NeedFinal Quality Assurance Testing

Explore Market Need


Explore Market Need

Scorecard provides insight into IT value creation for business

Scorecard describes initiatives that must be taken to enable reporting of designated metrics

Source: Bowne; Working Council research.* Names are for illustrative purposes only and do not represent actual employees.

Objectives and metrics clearly described in nontechnical terms

Project Performance

Both metrics and initiatives have designated owners to ensure accountability for data collection and execution



Optimizing Operational Performance

Most IT organizations track the performance of individual operational systems, including servers, applications, and desktops using dashboards and alerting functionality to inform operational managers of service quality problems and system failures. However, the IT department’s customers, the end users, are most likely to judge IT’s operational performance based on their personal experience, asking “Does my PC work?” or “Do I have Internet access?” As a result, the typical IT balanced scorecard provides little intelligence to CIOs and business decision makers as to how changes in IT’s performance impact users. Metrics in the operational category of the IT balanced scorecard range in maturity across the following three levels:

Measuring the Performance of Individual Systems—The majority of companies include specifi c operational data such as network uptime or application availability in the operational category of their IT balanced scorecard. This information is critical for the ongoing management of performance levels, but of limited value as a scorecard metric for two reasons. First, a report produced quarterly will provide operational managers with data only when it is too late to act; and second, granular operational metrics provide little insight into where decision makers can cut costs without adversely affecting service levels that matter to end users.

An Aggregated View of Operational Performance—More advanced balanced scorecard practitioners are aggregating subsets of IT dashboard metrics to present an overview of the performance for particular units of IT service (for example, desktop availability or Web front-end uptime). This approach takes an initial step toward denominating IT operational performance in terms relevant to end users, providing the CIO and senior management with a more informed basis for cost and service-level trade-offs. In addition, IT organizations are also using the operational category to track system compliance with architecture standards.

Operational Performance from the End-User’s Perspective—Exemplar companies take this customer-centric approach a step further, focusing their operational scorecard metrics on how the end user actually experiences IT service and managing IT operations with the end user in mind. While a network outage might severely impact IT’s uptime performance when viewed through the lens of an operational dashboard, it has little impact on end users when it happens in the middle of the night, or only impacts backup systems. On the other hand, if a critical application is not available during peak work hours, the impact on end users can be comparable to a total system crash. Consequently, to help improve operational performance, exemplars utilize scorecard metrics that track an aggregate measure of uptime for all systems that impact end users and sensitize those metrics according to the impact of performance problems on critical business processes.

Drilling Down into the Operations PerformanceOperational Performance


Operational Performance Metric Maturity Trajectory

Relative Maturity of Operational Performance MetricsIllustrative

Metric Relevance to End User

Level of Metric Detail

Aggregated Services

Individual Systems

Low High

Baseline

Measuring the Performanceof Individual Systems

Sample Metrics:

• CRM availability• LAN uptime


An Aggregated View of Operational Performance

Sample Metrics:

• Web front-end uptime• Desktop availability• Percentage of systems compliant

with architecture standards

Exemplar

Operational Performance from the End-User’s Perspective

Sample Metrics:

• Peak time availability• Critical process uptime

Source: “Formula for ROI,” PC Week (28 September 1998); Working Council research.

Putting the Business First“In the past, we’ve tended to take a more internally focused view of IT with our metrics to determine things like data center or network availability. However, customers don’t necessarily care about what is causing a network outage, but more about the fact that the system is down. …Now we’re attempting to measure ourselves on things business people think are important.”

Mike Hyzy Manager of Global IT Metrics, NCR Corp.




Company Background

T. Rowe Price Group, Inc., the Baltimore-based asset management fi rm, has $925 million in revenues and 3,700 employees. The company administers 80 funds and manages assets worth more than $155 billion.


IT management reporting practices at T. Rowe Price historically suffered from two fl aws. First, reports focused on operational performance in minute detail while neglecting non-technical performance metrics. Second, even for operational metrics, the reports lacked established targets for adequate performance. To provide a holistic view of corporate IT’s performance with respect to fi nancial, project, operational, talent management, and user satisfaction measures, T. Rowe Price created an IT balanced scorecard in early 2001. T. Rowe Price updates the scorecard quarterly and uses it to highlight a continued commitment to streamlining total IT expenditures and to showcase IT’s partnership with the business.

Managing Performance from the End-User’s Perspective

Recognizing that end-user productivity should be the focus of IT operational metrics, T. Rowe Price tracks the number of system failures that impact the business user’s ability to do work.

The Scorecard as a Compliance Lever

While most companies track some form of system reliability as part of their scorecard efforts, fewer organizations use their scorecards to track compliance with architectural standards and project management processes or gauge the effectiveness of vendor management efforts. The operational category of T. Rowe Price’s IT balanced scorecard tracks compliance of new project work with existing architecture standards and the IT organization’s standard system development methodology. By highlighting areas where business sponsors or project managers have deviated from T. Rowe Price’s established standards, the scorecard provides a measure of “name and shame” reporting, forcing discussion of non-standard work in the presence of senior decision makers. In addition, T. Rowe Price also includes a “competitive bids” metric as part of the operations category. This metric is aimed at driving down overall IT spending by encouraging sponsors to solicit external bids for new projects.

Balancing Business and IT Perspectives on OperationsOperational Performance


T. Rowe Price’s IT Balanced ScorecardSample Metrics

Financial Performance• Percentage change in EBITDA• Weighted IT expense ratio versus industry• Total IT expenditures• Total expenditure on new functionality• NPV delivered during payback period

Project Performance• Percentage of projects delivered on time, on budget, with

acceptable customer satisfaction• Percentage of active projects with approved project plan


Talent Management• Percentage on non-entry-level positions fi lled internally• Average years’ tenure of solid performers• Number of candidates interviewed per open position• Percentage of task forces/projects led by non-project managers

User Satisfaction• End-user perception of system stability• Performance versus user expectation of delivery speed• Price competitiveness of IT services versus external benchmark

Operational PerformanceIllustrative

Objective Metric Current Quarter

Performance

Current Quarter Target

Status Next Quarter Target

Comments

Integrate Solutions Using Defi ned Architectures

Percentage of projects complying with existing architectural standards and complying with standard development methodology

10% 30% 35%

Includes projects in progress before new standards created

Provide Reliable and Functional Systems

Number of failure incidents with business impact per quarter 3.5 3.0 2.5

Severe weather damaged storage capability

Effectively Select and Manage Sourcing Relationships

Percentage of strategic projects bid competitively 66% 65% 70%

On track

Metrics goals for current and upcoming quarter allow more informed assessment of current performance and timeline for closing gap to goal

Metrics focus on systems failures with negative business impact

Monitoring User Impact and Standards ComplianceT. Rowe Price utilizes the IT balanced scorecard to track business-focused

operational metrics and monitor architectural compliance

Source: T. Rowe Price; Working Council research.


Scorecard supplements traditional availability metrics with measure of architectural and methodological compliance of proposed and ongoing project work

Scorecard supplements availability metrics with measure of architectural and methodological compliance of proposed and ongoing project work

Spotlighting poor standards compliance or methodology adherence at executive level acts as “name and shame” for business sponsors and project managers

High-Level Scorecard fi ts onto one page



Keeping and Attracting Top Performers

Talent at the Heart of the IT Enterprise

The size of many IT departments has been decreasing across the past several years in response to cost pressures, and as a result, CIOs have been forced to focus on talent management in an effort to maximize the productivity of existing staff. This involves both upskilling the entire IT workforce and focusing on the retention of staff with critical skills. IT balanced scorecard practitioners use their scorecards to better manage IT talent in several ways:

Keeping Track of IT Staff—As one of the four categories outlined in the original Kaplan and Norton scorecard, talent management is a category included almost universally in IT balanced scorecards. The most common scorecard talent metrics include total IT staffi ng, individual and average staff tenure, employee satisfaction, and overall staff retention rate.

Measuring the Skills Bench—More progressive scorecard practitioners supplement basic staff demographics with metrics to track the organization’s existing skill sets and the career goals of IT staff (for example, the number of staff trained in skills identifi ed as critical and the percentage of IT staff with individual development plans). These metrics allow CIOs to better spotlight emerging gaps or opportunities for staff development.

Becoming a Talent Magnet—Exemplar IT organizations use the IT balanced scorecard to focus senior executive attention on IT’s success in creating a “destination” for available talent, tracking metrics like the retention of high-potential staff, the number of press citations of the IT organization and company-developed technologies, or the number of industry awards the company has won for its use of technology.

Talent Management


Talent Management Metric Maturity TrajectoryRelative Maturity of Talent Management Metrics

Illustrative

Focus of IT Brand Building

Targeted Employee Population

Baseline

Keeping Track of IT Staff

Sample Metrics:

• Total number of staff• Staff tenure (individual and

average)• Employee satisfaction• Overall retention rate


Measuring the Skills Bench

Sample Metrics:

• Current skills inventory• Percentage of staff with

individual development plans

Exemplar

Becoming a Talent Magnet

Sample Metrics:

• Retention of high-potential staff

• External citations of IT achievement

Individual HIPOs

All Staff

Internal External


Talent Management



Investing in the Future

Company Background

AstraZeneca PLC, the global pharmaceuticals company, is one of the top fi ve pharmaceuticals companies in the world with more than $17.8 billion in revenues and has 58,000 employees.


AstraZeneca introduced the balanced scorecard in 1998 as a tool to help communicate corporate strategy and priorities, ensure alignment of functional initiatives with those strategies and priorities, and monitor execution of specifi c strategies. While adoption of the balanced scorecard is widespread at AstraZeneca, there are no corporate templates that individual functions must follow. The content of the IS balanced scorecard is dictated by two principal drivers, the strategic priorities of the company and the vision for the IS organization, which are refl ected in three key priorities for the IS function:

1. Deliver business value

2. Transform IS effi ciency and effectiveness

3. Develop IS leadership and capabilities

The PowerPoint-based scorecard is updated on a quarterly basis. Each scorecard objective is assigned to an owner on the CIO’s global IS management team who updates the metrics that underpin that objective and submits the metrics and commentary to the assembled global IS management team for review and discussion. The fi nal scorecard comprises the strategic agenda for the IS senior management team and is used to track ongoing execution of IS strategy. Once fi nalized, the scorecard is published on the company intranet. IS line mangers are strongly encouraged to discuss the balanced scorecard with their staff and defi ne their team goals in a fashion that is consistent with the corporate IS vision that is encapsulated in the scorecard.

Leadership Development as a Foundation for Value Creation

AstraZeneca’s stated goal is to develop the leadership performance and skills base of its existing IS staff at all levels of the organization. To monitor its progress toward that goal, AstraZeneca’s IS organization tracks the progress of two key objectives in the talent category of its IS balanced scorecard:

Developing and Demonstrating Leadership—The fi rst set of talent management metrics on AstraZeneca’s IS balanced scorecard is designed to measure the development of leadership capabilities within the technology organization. These metrics include the percentage of IS staff, IS leaders, and identifi ed high-potential staff with established development plans, and the percentage of key IS roles fi lled.

Developing and Demonstrating IT Skills and Disciplines—AstraZeneca’s IS balanced scorecard also tracks performance in creating skills and development opportunities for existing IS staff by including metrics such as training attendance versus capacity and the percentage reduction in skills shortages fi lled by external consultants. More specifi cally, AstraZeneca’s IS balanced scorecard refl ects the IS organization’s emphasis on cultivating project management skills and developing global service delivery capabilities, tracking metrics that measure compliance with project management methodologies and project manager skills development, as well as those measuring collaborative tool usage, virtual team productivity, and the number of rotational assignments undertaken by IS staff.

Talent Management

A Multipurpose Performance Management Tool“The IS balanced scorecard acts as an extremely powerful vehicle for communicating AstraZeneca’s IS strategy and priorities to the company, helps ensure that IS initiatives are clearly aligned to business priorities and strategy, and allows effective tracking of IS organization success in strategy execution.”

Balvinder Dhillon AstraZeneca


Developing IS LeadersAstraZeneca employs its IS balanced scorecard to help execute

on its strategy of identifying and developing IS leaders

AstraZeneca’s IS Balanced Scorecard

Sample Metrics

Project Performance• Ongoing benefi ts captured versus plan• Project delivery progress versus budget,

schedule, and quality milestones

Operational Performance• Number of business interruptions caused

by security intrusion• Percentage of internal portals adopting

architecture guidelines• Number of material adverse regulatory

compliance observations by FDA or other IS audits

Talent Management

Develop IS Leadership and CapabilitiesIllustrative

Objective Metric Current Performance

Previous Performance Status Comments

Develop and Demonstrate Leadership Capability

Percentage of IS staff development plan reviewed and actions implemented 60% 75%

Percentage of incumbent IS leaders with development plan in place 90% 92%

Percentage of global high-potential staff with development plan in place 80% 75%

Change management and business acumen programs in place No No

Number of annual IS rotational assignments 10 11

Diversity performance (scale of 1 to 5) 4 4

Percentage of key roles fi lled 99% 92%

Develop and Demonstrate IS Professional Skills and Disciplines

Percentage of projects following project management framework 70% 70%

Percentage of projects with KPIs for visible value demonstration 80% 50%

Percentage of project managers with development plan and career path 60% 40%

Percentage increase in project manager capability 20% 10%

Attendance versus capacity in existing learning programs 60% 72%

Percentage increase in collaborative tool usage 40% 30%

Improvement of “virtual” team productivity +5 -1

IS capabilities versus benchmarks (scale of 1 to 5) 3 2

Percentage reduction in skills shortages fi lled by consultants 20% 15%

Scorecard provides reader with commentary from metric owner and an outline of required performance improvement steps

Source: AstraZeneca; Working Council research.

Scorecard categories mirror AstraZeneca’s global IS strategic priorities

Talent Management

Metrics concentrate on career planning and capability development of existing staff, with a focus on high performers

Metrics also focus on desire to strengthen global IS service delivery capabilities

Multiple metrics underscore IS organization’s emphasis on developing project management skills



Tracking IT’s Brand Equity

Company Background

Schneider National, Inc., the Green Bay, Wisconsin–based transportation and logistics provider, has $2.6 billion in revenues and 21,000 employees. The company operates a fl eet of 14,000 trucks and more than 40,000 trailers.


Schneider National’s IT balanced scorecard was implemented in 1998 and is composed of four major types of metrics: fi nancial performance, project performance, operational performance, and talent management. The IT organization views the scorecard as a tool to help align IT efforts with the needs of the business and reduce the time senior IT management needs to identify and fi nd solutions to service problems. The scorecard is updated on a monthly basis and reviewed by the CIO, the IT vice presidents, and IT directors. The scorecard includes the target, current month, and year-to-date performance for each metric.

Developing an IT Brand

Schneider’s talent management metrics include typical internally focused measures like staff turnover rates and employee tenure with the company and in IT. However, in service to a corporate initiative to boost brand awareness of Schneider and its products and services, the IT organization also tracks two externally facing metrics: the number of press citations of Schneider’s IT organization and Schneider-developed technologies, and the number of industry awards Schneider has won for its use of technology.

By using the balanced scorecard to track the improvement of the internal employment brand for IT, companies can reduce staff turnover, especially for high-potential staff who would be diffi cult to replace on the internal talent market and costly to replace on the external market. In fact, Schneider credits the ability to regularly track the metrics in the talent management scorecard category with helping it achieve a signifi cant reduction in annual staff turnover, which has declined from 26 percent before the scorecard was implemented to 7 percent, across a period of fi ve years. Some of this turnover has likely been stemmed as a result of the waning fortunes of the overall economy across the past several years, but Schneider claims that by providing decision makers with a consolidated view of talent metrics, the IT organization is able to more effectively deploy its resources toward improving the internal IT employment offer, with reduced turnover as the result. In addition, by using the scorecard to track press citations and awards, Schneider’s IT organization has also begun developing an external IT brand, becoming a destination for available talent in the industry.

Talent Management


Baseline of internally-focused metrics supplemented with metrics of external IT “brand”

Tracking of external citations of Schneider National National’s IT organization boosts brand awareness, positively impacting retention rate and increasing the visibility of IT, to in play talent

Measuring IT’s Reputation Inside and Outside the OrganizationSchneider National utilizes its balanced scorecard to build IT’s brand as an employer of choice

Schneider National’s IT Balanced ScorecardSample Metrics

Financial Performance• Central IT budget as a percentage of revenue• IT maintenance cost per load, per movement• Desktop total cost of ownership• Data communications cost as a percentage of revenue

Project Performance• Percentage of projects on time, on budget, with

acceptable satisfaction score• Percentage of budget allocated to unplanned projects

Operational Performance• Number of server outages per month• Reduction of desktop applications • Number of UNIX servers

Talent Management

Talent ManagementIllustrative

Objective MetricCurrent Month

Performance*

Current Month

Target*

Year-to-Date Performance*

Create an Attractive IT Environment

1. Annual turnover rate 7% 7% 8%

2. Average years of employee experience at Schneider National

8 8 7.9

3. Percentage of planned staffi ng levels 97% 95% 94%

Build a World-Class IT

Organization

4. Citations of Schneider National technology organization in the press

6 10 14

5. Schneider National technology and corporate awards

1 2 1

6. Average years of employee experience in IT 4.8 5 5.4

Baseline of internally focused metrics supplemented with metrics of external IT “brand”

Tracking of external citations of Schneider National’s IT organization boosts brand awareness, positively impacting retention rate and increasing the visibility of IT to available talent

Source: Schneider National; Working Council research.* Numbers are for illustrative purposes only, and do not represent actual performance or targets.

Talent Management



User Satisfaction as an Early-Warning Indicator

To combat the historical fi nancial bias of corporate reporting and create a truly “balanced” perspective of corporate performance, Kaplan and Norton advocated the inclusion of customer satisfaction metrics on the balanced scorecard. For IT organizations, these “customers” are, in most cases, end users and business sponsors. Faced with limited performance measurement resources, CIOs have integrated satisfaction-surveying efforts into their IT performance measurement toolkit to track the perception of IT service delivery with three varying degrees of rigor:

Ad Hoc Assessment—In many cases, companies choose to measure end-user satisfaction anecdotally, either because they see no need or are unwilling to incur the potential costs involved in regularly collecting the information. As a result, these scorecards do not truly represent a “balanced” view of IT performance, relying almost exclusively on fi nancial metrics as a proxy for IT value.

Reactive Information Gathering—The majority of companies that do track user satisfaction as part of their IT balanced scorecard process opt for brief, random questionnaires in direct response to user requests or process milestones, such as help-desk calls or project completion. Although this approach provides IT departments with an inexpensive way to collect a sample of user perspectives on IT performance, it may not offer an accurate picture of user satisfaction. Since these surveys tend not to be anonymous, users may respond less candidly, and if the survey sample population is generated based on help-desk tickets, the sample may be biased toward users whose most recent interaction with IT has been extreme, either unusually positive or unusually negative.

A Census-Level View of IT Performance—Best-in-class IT balanced scorecards incorporate the results of formal satisfaction surveys covering the entire user base. Exemplars conduct surveys on at least an annual basis, either surveying the entire user population at once or by surveying a smaller but representative sample of the population each quarter. Leading companies also take steps to document executive-level perspectives of IT performance rather than aggregating all user feedback into an undifferentiated average. The results of these two types of satisfaction surveys allow CIOs to more quickly and effectively identify changes in user perception of IT which may be connected to a decline in service levels or problems with vendor contracts.

Measuring IT’s Public PerceptionUser Satisfaction


User Satisfaction Metric Maturity Trajectory

Relative User Satisfaction Metric SophisticationIllustrative

Seniority of Respondents

Penetration of User Base

Baseline

Ad Hoc Assessment

Sample Metrics:

• None; scorecard includes user testimonials (anecdotal evidence)


Reactive Information Gathering

Sample Metrics:

• “Customer satisfaction” drawn from a random selection of users (with potential sample bias)

Exemplar

A Census-Level View of IT Performance

Sample Metrics:

• “Customer satisfaction” drawn from entire user population (comprehensive perspective)

• “Senior executive satisfaction” drawn from focused executive feedback

Census

Self-Selected Users

Undifferentiated End Users

Business Sponsor/Executive


User Satisfaction



Company Background

J.D. Edwards & Company, the Denver-based ERP and supply chain software maker, has $904 million in revenues and 4,954 employees. The company serves 6,500 customers in more than 100 countries, and 75 percent of revenues come from professional services.


Although not formally called a balanced scorecard, J.D. Edwards’ six corporate initiatives, which include “revenue and growth,” “customer satisfaction,” and the creation of a “knowledgeable and committed workforce,” approximate several of the major categories of an IT balanced scorecard.

An Uncoordinated Approach to Customer Satisfaction

While growing from $300 million in revenues in 1998 to $1 billion in 2000, J.D. Edwards’ IT organization struggled to capture actionable end-user feedback that would allow it to better enable the company’s growth strategies. The IT organization faced two hurdles to these ambitions. First, the company often relied on external benchmarking data to choose IT products and services without input from internal customers. Second, when the IT organization did administer end-user surveys, it did so on an ad hoc basis, with users often receiving multiple surveys from multiple groups within IT. Bombarding end users with too many surveys generated sub-optimal response rates, leading to a non-representative view of IT’s performance.

Combating Survey Fatigue

To solve this problem of survey “fatigue,” J.D. Edwards adopts a quarterly satisfaction survey, with each survey covering 25 percent

of the company’s approximately 5,000 users. In place since 1999, the survey is composed of 26 questions that gauge user satisfaction with existing IT products and services and requires only 15 minutes to complete. The survey instrument is an internally developed Web-based tool. The tool delivers the survey to IT customers and compiles survey responses and results at a cost of less than $1,000 per year. Survey results are presented as a series of “dials” on the IT intranet. These dials include a scale from one to seven and show average scores for satisfaction with specifi c IT services. Viewers can double-click and view individual comments. A single IT account manager administers the survey, manages the survey process, and analyses the results in addition to his/her regular account management duties. In addition, the account manager collaborates with IT services managers to ensure that action plans are created to address the problems highlighted by survey results, and also works with the other account managers and IT service managers to select additional IT services to measure from a customer satisfaction perspective.

Giving Customers What They Want

The IT department uses the feedback captured to iteratively test service improvements. For example, unfavorable survey feedback about the help desk led to the termination of a contractor’s services and the creation of an internal help desk that was able to meet the established service-level goal. In another instance, quarterly user survey feedback revealed dissatisfaction with the company’s remote access service provider based on signifi cant downtime and unsatisfactory cost. In response to continued user dissatisfaction, the IT organization contracted with an alternative service provider and upgraded its connectivity architecture, resulting in an improved user satisfaction score.

Listening to the Voice of the CustomerUser Satisfaction


A Call to Action

“To put on a survey and collect all the feedback and then not act on the results would be a mortal sin for everybody.”

Frank Catalfamo, Manager IT Metrics and M&As, J.D. Edwards

Using End-User Feedback to Improve Service Quality

Annual survey generates detailed insight into drivers of user satisfaction…

…and provides a vehicle for iterative improvement of service delivery

Iterative Service Improvement ProcessIllustrative

J.D. Edwards IT User Survey • Selected Questions

IT Usage Assessment (Yes/No Response) Yes No

Off-site network access usage þ ¨

Offi ce move or expansion involvement þ ¨

Utilize or coordinate voice services ¨ þ

Request application services ¨ þ

Conduct training classes in training classroom þ ¨

IT’s Overall Performance

Overall performance of IT Services

Ability to deliver technical/business solutions and services

Communication about available services and new technologies

User Satisfaction with IT Services

Remote access

Internal voice services and support

PC services and support

IT problem resolution

Network infrastructure

User Satisfaction with IT Intranet

Accessibility

Value-Add (Write-in Response)

Additional Functionality (Write-in response)

User Satisfaction with Voice Services

Voice services information availability

Value-Add (Write-in Response)

Additional Functionality (Write-in response)

Q1 Survey

Remote Access Service

Feedback: Low user satisfaction with remote access service

Corrective Step: Notify access provider

Q2 Survey


Feedback: Dissatisfaction remains

Corrective Step: Change access provider

Q3 Survey


Feedback: Partial improvement

Corrective Step:Roll out new connectivity application

Q4 Survey


Feedback: Service quality goal is met

Corrective Step: None required

Source: J.D. Edwards; Working Council research.

User Satisfaction

Options

1. Extremely Unsatisfi ed2. Very Unsatisfi ed3. Unsatisfi ed4. Neutral5. Satisfi ed6. Very Satisfi ed7. Extremely Satisfi ed

1

2

34

5

6

7 1

2

34

5

6

7

1

2

34

5

6

7 1

2

34

5

6

7



Company Background

Eli Lilly and Company, the Indianapolis-based pharmaceutical manufacturer, has revenues of $11 billion and 43,700 employees.


In order to better track IT’s contributions at the enterprise level, the IT organization at Eli Lilly created an IT balanced scorecard in 1999.

Enfranchising Senior Business Decision Makers

Recognizing that internal customer satisfaction surveys often fail to make the important distinction between the feedback of end users and truly empowered business decision makers, Eli Lilly’s CIO supplements the IT organization’s end-user satisfaction survey by conducting an annual feedback exercise with 12 senior business executives. Feedback is gathered from each executive by the CIO in a 30 to 60 minute interview, using six qualitative questions designed to gauge senior-level perceptions of IT value creation. The questions ask business leaders to assess:

• IT’s contribution to business value creation

• IT contribution to business process improvement

• IT contribution to company competitive advantage

• IT contribution to corporate business strategy

• IT operational performance

• Overall satisfaction with IT service

Each respondent is asked to rate the corporate IT function on a scale of 1 to 5, with 5 being very satisfi ed. The aggregate results of the survey are then presented as part of the IT balanced scorecard, with individual responses kept confi dential to encourage an honest appraisal of IT’s performance. The intranet-based scorecard allows decision makers to drill down into top-level scorecard metrics, including underlying data, commentary from the metric “owner,” and an overview of action steps that will be taken to improve performance.

Heeding the Business’s Advice

In addition to the numerical feedback provided by executives, Lilly also solicits qualitative feedback. This not only helps capture executive perceptions of current IT service, but is also an opportunity to gather future service requirements. So, by capitalizing on this occasion for dialogue provided by the executive satisfaction survey, Eli Lilly’s CIO gets both direction for the present and a glimpse into what business sponsors want IT to be in the future. Eli Lilly has used this formal feedback from senior business executives to guide and improve its IT efforts in several concrete ways. In response to feedback that the business divisions required more clarity on IT’s funding decisions, Eli Lilly’s IT organization redesigned its portfolio prioritization process to provide more transparency to business heads. In addition, executive survey feedback prompted Lilly’s IT organization to expand the size and scope of its “reverse mentoring” program, in which junior IT staff serve as mentors to senior business executives, helping them understand IT terms and concepts.

Collecting the Executive View of IT PerformanceUser Satisfaction


Eli Lilly’s IT Balanced ScorecardSample Metrics

Financial Performance• Economic Value Added (EVA)• Unit service cost

Project Performance• Project Portfolio

Operational Performance• Base investment• Service levels versus external benchmarks• Compliance with enterprise methodologies and

standards

Talent Management• Diversity• Progress on people• Key capabilities

User Satisfaction

Executive Management Feedback Collection FormIllustrative

Executive Name: ________________ Business Unit: _______________ Date: ______________

IT is not at all meeting the needs of the corporation

IT is exactlymeeting the needs of the corporation

Expectations Future Requirements

1. IT Contribution to Business Value Creation

1£ 2R 3 £ 4 £ 5 £ Not enough clarity around IT cost

Update portfolio prioritization process

2. IT Contribution to Business Process Improvement

1£ 2 £ 3R 4 £ 5 £ Close relationship in redesigning process

Creation of reverse mentoring program

3. IT Contribution to Company Competitive Advantage

1£ 2R 3 £ 4 £ 5 £ More proactive technology scanning

Charter advanced technology group

4. IT Contribution to Corporate Business Strategy

1£ 2 £ 3 £ 4R 5 £

5. IT Operational Performance 1£ 2R 3 £ 4 £ 5 £

6. Overall Satisfaction with IT Service

1£ 2 £ 3R 4 £ 5 £

Survey conducted annually by CIO in individual sessions with 12 senior business executives

Only aggregate results of executive surveys are published to the scorecard to encourage honest feedback

An Occasion for Dialogue

“The executive management feedback surveys are a useful tool for understanding and responding to business perceptions of IT’s performance. The results are not as important as the dialogue between IT and the business.”

Roy Dunbar, CIO, Eli Lilly

Assessing IT Value CreationEli Lilly supplements a user satisfaction survey with executive feedback to inform performance improvement efforts

Joe Smith Marketing Dir. 4/20/2002

Source: Eli Lilly; Working Council research.

User Satisfaction

Qualitative feedback gathered includes executive expectations, perception of IT’s achievement with regard to those expectations, and future service requirements



A Proactive Approach to Security

Extended Enterprise Increases Security Risks

As organizations’ reliance on Web-based transactions, remote and wireless computing, and connectivity to external partners continues to grow, the number of potential information security vulnerabilities has also increased. This trend toward greater enterprise permeability, coupled with the tragic events of September 11 and high-profi le security incidents such as 2003’s Slammer virus, has caused senior decision makers to focus their attention on the unpredictable and costly information security threats posed to the extended enterprise. IT organizations exhibit three levels of sophistication in managing that potential information security spending using a balanced scorecard.

Reactively Tracking Security Incidents—Most companies have not yet added an information security category to their IT balanced scorecard, instead tracking a handful of technical security measures in the scorecard’s operational category. These measures are aimed primarily at retroactively documenting the number of security incidents detected per period and the time required to respond to those incidents.

Proactively Assessing Potential Vulnerabilities—More advanced balanced scorecard practitioners include metrics to proactively assess potential vulnerabilities. In most cases, these metrics are still included in the operational performance category of the scorecard, not in a stand-alone information security category. They include measures like the percentage of systems compliant with security standards, the percentage of network access points covered by intrusion detection systems, and the percentage of IT security audit issues remedied by the agreed-upon deadline.

Hardwiring Security Compliance—Exemplar IT organizations create a separate information security category on their IT balanced scorecards. By elevating security metrics to the category level, exemplars seek to clearly communicate urgency of these measures to senior IT staff and business sponsors. Exemplars also supplement retroactive report of security incidents with tracking the implementation of and compliance with security policies and preventive measures. Typical metrics include the percentage of staff receiving security training and the percentage of external partners in compliance with security standards.


© 2003 Corporate Executive Board Best-in-Class IT Balanced Scorecards 47

Information Security Metric Maturity TrajectoryRelative Maturity of Information Security Metrics

Illustrative

Metric Posture

Focus of Measurement

Baseline

Reactively Tracking Security Incidents

Sample Metrics:

• Number of security incidents detected per period

• Average time to respond to incidents


Proactively Assessing Potential Vulnerabilities

Sample Metrics:

• Percentage of systems in compliance with IT security standards

• Percentage of network access points covered by intrusion detection systems

• Percentage of IT security audit issues fi xed by agreed deadline

Exemplar

Hardwiring Security Compliance

Sample Metrics:

• Percentage of staff receiving security training

• Percentage of external partners in compliance with security standards

Policy

OperationalReactive Proactive





Safeguarding the Enterprise

Company Background

Schlumberger Limited is a $13.5 billion global technology services company with 76,000 employees of more than 140 nationalities. The company has operations in 140 countries and includes three primary business segments: Schlumberger Oilfi eld Services, the world’s largest oilfi eld services company and the leading supplier of technology services and solutions to the international oil and gas industry; WesternGeco, jointly owned with Baker-Hughes, the world’s largest surface seismic company; and SchlumbergerSema, an IT services company providing consulting, systems integration, and managed services to the energy, public sector, telecommunications, and fi nance markets.


Schlumberger introduced its IT balanced scorecard in 2002 as part of a larger IT initiative to better understand the comparative IT spending levels and align that spending more closely with business needs.

Assuming a Proactive Security Posture

To better track security spending and the progress of its ongoing information security initiatives, Schlumberger has created a stand-alone information security category on its IT balanced scorecard. Schlumberger tracks a mix of metrics that gauges both past information security performance as well as proactive preventive measures, including:

Incident Rate—Documents the number of security incidents per 1,000 employees per year. Reported incidents are logged in a central database and categorized by severity as serious, major, and catastrophic to allow for year-over-year comparison and trend analysis.

Percentage of Infrastructure Protected—Focuses on the protection of PCs, servers, and network infrastructure from malicious attacks. Protection regimes include deployment of anti-virus software on PCs, maintaining servers to close all known vulnerabilities, and applying security patches, and confi guring networks according to established security guidelines..

Percentage of Sites Audited for Security—Tracks the percentage of Schlumberger sites that have had an information security audit within the past 12 months. Audits are conducted by Schlumberger’s operational security group, and the results are cataloged in the central database to allow tracking of remediation progress for any identifi ed vulnerabilities.

Focusing Attention on Information Security

Schlumberger’s focus on proactive vulnerability identifi cation and remediation provides the IT organization with the knowledge of current system vulnerabilities. The scorecard also helps ensure that established security standards are being followed, and that security audits are being conducted on an ongoing basis. In addition, by prominently including security metrics on the IT balanced scorecard, Schlumberger increases security awareness among its IT staff and business sponsors.



Information SecurityIllustrative

Objective Metric Defi nition Current Performance

Benchmark

Safeguard the Enterprise

Incident Rate per 1,000 Employees per Year

Rate defi ned as the number of catastrophic/major/serious Information Security incidents per 1,000 employees per year within the company

8.6 7.7

Percentage of Infrastructure Protected

Indicator of the degree of protection from malicious attack of PCs, servers, and network infrastructure; protection for PCs means the device has updated antivirus and security patches; protection for servers means no high vulnerabilities; for network infrastructure protection means that they are confi gured according to our published standards

40% 29%

Percentage of Sites with Valid Information Security Audit

Percentage of sites that have conducted an information security audit within 12 months; audits are conducted by Operations Security and management personnel

10% 40%

Schlumberger’s IT Balanced ScorecardSample Metrics

Financial Performance• Projected cost for IT services• Year-to-date cost IT services• IT cost per seat• Cost of data communication per seat• Spending by portfolio category

Project Performance• Average application project delay

Operational Performance• Quarterly activity increase on the corporate

knowledge portal• Percentage of standardized PCs

Resource Management• IT headcount (excluding contractors)

User Satisfaction• Tickets per registered user per month• Help-desk calls• Help-desk fi rst-call resolution rate


Increasing Information Security Awareness and PerformanceSchlumberger includes both retroactive and proactive metrics in its security category



Scorecard includes both retroactive and proactive security measures

Clear defi nition of metrics

Information security elevated to category level



Giving Corporate Business Initiatives Their Due

Detailing IT’s Contribution to Enterprise Initiatives

In addition to its day-to-day operational role, IT often plays a critical role in enabling discontinuous enterprise-level business initiatives, such as post-merger integration or business process reengineering. By setting aside one category of the IT balanced scorecard to track IT’s contribution to the execution of these major business initiatives, CIOs are able to better manage the function’s enablement efforts and communicate those efforts to senior business decision makers and initiative sponsors. These communications efforts range from the nonexistent to the programmatic:

An IT-Centric View of Performance—In most cases, IT balanced scorecards focus solely on IT’s efforts to better manage its own activities. The link between corporate strategic initiatives and the metrics tracked on the IT balanced scorecard is tenuous at best, as the scorecard has not been developed in conjunction with the corporate strategy, but rather as a stand-alone IT performance measurement tool.

An Uncoordinated View of IT’s Contribution—More advanced practitioners use the IT balanced scorecards to manage the effective execution of IT work related to enterprise initiatives. However, relevant metrics are typically dispersed across the scorecard’s operational and project categories, making it diffi cult for the CIO to articulate and business owners of enterprise initiatives to understand IT’s full contributions to the business’s efforts.

Spotlighting Enterprise Business Initiatives—Exemplar companies create a separate category on the IT balanced scorecard that tracks metrics of IT’s contribution to major corporate projects or initiatives, often over a multiyear period, for initiatives such as merger integration. These metrics act as a vehicle for communicating IT’s value creation and are usually removed from the scorecard once the initiative has been completed.



Enterprise Initiatives Metric Maturity TrajectoryRelative Maturity of Enterprise Initiatives Metrics

Illustrative

Metric Location on Scorecard

Altitude of Link to Strategy

Baseline

An IT-Centric View of Performance

Sample Metrics:

• No metrics linked to execution of enterprise business initiatives


Uncoordinated View of IT’s Contribution

Sample Metrics:

• Percentage of acquired systems retired in Operational Performance category

• Number of business process enablement projects completed in Project Performance category

Exemplar

Spotlighting Enterprise Business Initiatives

Sample Metrics:

• Percentage of acquired company systems integrated in M&A category

• Number of business process steps enabled by technology in Process Reengineering category

Corporate Strategy

IT StrategyDispersed Consolidated





Putting IT on the Enterprise Stage

Organization Background

The Federal Aviation Administration has an annual budget of $14 billion and is responsible for oversight of air traffi c control, airport regulation and safety, and air regulation and certifi cation.


In early 2001, the FAA’s IT department deployed an IT balanced scorecard that tracks strategy implementation in six key areas: fi nancial performance, operational performance, talent management, user satisfaction, security and safety, and eGovernment. The FAA CIO and the CIOs of each of the FAA’s divisions review the scorecard on a weekly basis, and the same group reviews the scorecard’s categories and metrics yearly to ensure that they are aligned with the organization’s stated strategy. The scorecard includes the current status, two-year targets, and interim milestones for each metric, and documents potential risks that could jeopardize the achievement of those targets. It is published on the FAA’s intranet site and is visible to the entire IT organization.

Regulation Pushes eGovernment onto IT the Balanced Scorecard

The 1996 Clinger-Cohen Information Technology Management Reform Act requires that all United States federal departments and agencies create and implement an enterprise IT architecture plan. More recently, the 2002 Presidential Management Agenda calls for all federal departments and

agencies to establish an enterprise architecture by 2004 and to move services to the Web as part of the eGovernment initiative. This urgency in developing and implementing an enterprise architecture and migrating to electronic service delivery has led the FAA to create a stand-alone eGovernment category on the IT balanced scorecard, distinguishing these efforts from the other project-related metrics tracked in the scorecard’s operational performance category.

Improving Customer Service and Ease of Use

The eGovernment category of the FAA’s IT balanced scorecard includes three main objectives. The fi rst is to make interaction with the FAA easier and faster through the migration to Web-based technologies. The second is to ensure the consistent delivery of services by adopting standard guidelines for Web architecture, interface design, and content presentation. Finally, the FAA aims to ensure that data shared internally and externally are timely and accurate.

Generating Cost Savings Through Higher Visibility

By elevating the IT organization’s eGovernment efforts to the category level on the IT balanced scorecard, the FAA has not only increased IT staff awareness of its importance, but has also made those efforts more visible to both the organization’s IT and non-IT leadership. This increased visibility allowed the CIOs of the individual lines of business to realize more than $2.5 million in savings per year by collectively purchasing standard infrastructure.



eGovernment (Illustrative)


Current Year Target

Status Milestones Potential Risks

Reduce the Burden on Customers by Better Leveraging Web-Based Technologies

Percentage of applications migrated to Web

90% 100% • Inventory baselined• E-authentication

service interfaces defi ned

• Tools for e-authentication services may be required

Ensure Effective Service Delivery Capabilities

Percentage of FAA Web sites that comply with Web architecture, requirements, style, and content guidelines

7% 10% • Milestones to be collected and updated by CIO offi ce

• Possible change in due date

Ensure that Data and Information That Are Used to Conduct Critical Agency Business or Publicly Disseminated Are Timely, Accurate, Accessible, Understandable, and Secure

Percentage of FAA information systems critical to agency business or used to disseminate information publicly that use approved data elements

60% New15% Legacy

50% New10% Legacy

• Systems using approved data elements baselined

• Additional 250 data elements approved

• Full governance process developed and approved

• Ability to monitor compliance will be challenging

Increasing Awareness of IT’s Contribution to Enterprise InitiativesThe FAA’s IT organization elevates eGovernment efforts to the category level to underscore IT’s enterprise contribution

FAA’s IT Balanced ScorecardSample Metrics

Financial Performance• Percentage of IT investment dollars managed using

best practices as determined by appraisal• Percentage reduction in adaptation cost per system

Operational Performance• Percentage of program components compliant with

the “to be” enterprise architecture• Percentage of shared components, standards, and

infrastructures across multiple applications

Talent Management• Percentage of program managers with project

management certifi cation

User Satisfaction• Number of information quality complaints received

Security and Safety*

eGovernment

Potential risks to success are listed on scorecard in advance to allow creation of mitigation strategies

Source: FAA; Working Council research.


Key agency-level “eGovernment” project receives own scorecard category to allow IT organization to monitor “Webifi cation” and enterprise architecture ambitions

Current and next year targets as well as project milestones listed to inform “gap to goal” discussions

* Metrics not released by FAA.

54


Scorecard Development and Life-Cycle Management


Scorecard Rollout p. 56

Data Collection and Quality Assurance p. 58

Scorecard Review and RevisionFacilitating Scorecard Adoption

p. 60p. 62

Balanced scorecard exemplars view the use of IT balanced scorecards as on ongoing process rather than a discrete event. To ensure that the IT balanced scorecard is a useful decision-making tool, exemplars regularly refresh the scorecard to maintain its alignment with the IT strategy, adjusting the categories and metrics accordingly. In addition to regular revision of scorecard categories and metrics, exemplars also take steps to help ensure the timeliness and accuracy of scorecard data and facilitate scorecard adoption, codifying data collection roles and responsibilities and creating templates and incentives to encourage scorecard use.



A Process, Not a Document

Company Background

Bowne & Co., a New York-based document, information management, and printing solutions provider, has $1 billion in revenues and 8,000 employees.

The Balanced Scorecard as a Closed-Loop Process

Bowne’s IT organization implemented its balanced scorecard in 2000 as part of a corporate-wide scorecard initiative, spurred by the arrival of a new CEO. This scorecard implementation and life-cycle process involves seven key steps:

1. Kick Off Training for IT Staff—Bowne kicks off its balanced scorecard initiative with a three-day, off-site training session for all of the company’s divisional and functional senior managers, including those from IT. The training includes a basic overview of balanced scorecard concepts and terminology, but the session focuses on an exercise to help participants initially align their unit’s strategy with the corporate strategy and then develop a set of metrics to measure contribution to each strategy. This process also allows each unit leader to communicate the unit’s own substrategies to his or her peers.

2. Ongoing Strategy Mapping—Each year, Bowne creates a top-down corporate strategy which becomes the basis for a corporate balanced scorecard. Each business unit and function, including IT, then devolves its strategies from the corporate strategic plan.

3. Metrics Selection—Building off of the IT strategy, a team including the corporate CTO and his six direct reports—senior directors for global operations, client integration, enterprise architecture, and corporate systems, and vice presidents of IT, HR, and Finance—creates a set of recommended metrics to track IT’s progress against each strategic pillar. These recommendations are then presented to the CIO for fi nal approval.

4. Metrics Defi nition—The same CTO-led team then creates a defi nition for each of the metrics on the IT balanced scorecard. This defi nition includes a one-sentence description, an overview of the measurement technique used to track the metric—for example, a user survey or a dashboard—and a list of initiatives that must be undertaken to enable the tracking of the metric using the outlined technique (for instance, the development or purchase of new reporting software).

5. Assigning Metric Ownership—Each of the metrics on the scorecard is assigned an owner who reports directly to the CTO. The metric owners are directly involved in the scorecard creation and update process, and a percentage of their bonus is contingent on their scorecard-related duties.

6. Data Collection and Quality Assurance—The frequency of data collection varies by metric according to several factors, including the cost of data collection, the corporate fi nancial reporting cycle, and the volatility of the business climate.

7. Scorecard Review and Revision—The CIO and the other 11 Bowne corporate offi cers meet every six months to review the scorecards for every area, including IT. The applicability of all metrics is reviewed annually by the CIO, the CTO, and their direct reports, and approximately 20 percent of metrics turn over each year.

Beyond the initial consulting fees for the balanced scorecard training, Bowne’s IT organization invested approximately 120 person-days in managing the balanced scorecard process in 2002. This effort was distributed to the largest extent across the 20-person IT management team.

Scorecard Rollout

© 2003 Corporate Executive Board Scorecard Development and Life-Cycle Management 57

Scorecard Rollout

The IT Balanced Scorecard Life CycleBowne creates a “closed-loop” IT balanced scorecard process

Bowne’s IT Balanced Scorecard Adoption ProcessIllustrative

2. Ongoing Strategy Mapping

• Annual IT strategy devolved from corporate strategy

5. Assigning Metric Ownership

• Owners assigned to each metric are responsible for scorecard completion

• Owners report to CTO and their bonuses are linked to their scorecard-related duties

4. Metrics Defi nition

• CTO-led team creates standard defi nitions for all metrics, defi nes measurement technique and data collection processes, and outlines initiatives that must be completed to allow tracking of metrics

Source: Bowne; Working Council research.

3. Metrics Selection

• Team of CTO and direct reports creates list of metrics

• List refi ned by using analysis of each potential metric’s strengths and weaknesses

• Final approval by CIO

Metric

Brainstorming

Metric Brainstorming

ü

ü

6. Data Collection and Quality Assurance

• Data collection frequency varies by metric based on cost of collection, the corporate fi nancial reporting cycle, and volatility of the business climate

7. Scorecard Review and Revision

• CIO, CTO, and corporate offi cers review scorecard every six months

• Metrics revisited annually by CTO-led group

1. Kick Off Training for IT Staff

• Balanced Scorecard 101 for divisional and functional senior managers

• Initial strategy alignment and metrics selection exercise

Process Costs

Initial consulting fees

$10,000 license fee for software and $3,500/year maintenance and support costs

120 person-days/year for ongoing process management



Making Data Collection a Part of the Job

Assigning Data Collection Responsibility

While establishing and continually updating categories and metrics is critical for the success of an IT balanced scorecard, those metrics are useless if the underlying data are outdated or inaccurate. Most IT organizations have loosely defi ned responsibility for metrics collection, adding names of “metrics owners” to their scorecards. Exemplars defi ne a clear collection process, designating an owner for each metric, and in some cases, basing a portion of the owner’s compensation on timely, accurate delivery of data. By creating greater accountability for reporting, CIOs maximize the scorecard’s decision-making effectiveness, ensuring the completeness and accuracy of data, as well as its comparability over time.

Company Background

Cemex, S.A. de C.V., the Mexico-based manufacturer of aggregates, concrete, and cement, has $6.7 billion in revenues and 26,000 employees. The company has operations in more than 30 countries.


Cemex’s IT balanced scorecard was created as part of a corporate balanced scorecard initiative, launched in 2001 by the corporate VP of planning and fi nance. IT was among the fi rst functional areas to adopt the balanced scorecard methodology, and since then the scorecard has undergone some changes, including a reduction in the number of objectives and metrics measured. A subset of the IT balanced scorecard metrics feeds directly into the enterprise-level balanced scorecard.

Building a Data Collection ‘Infrastructure’

Cemex IT initially struggled with the collection of balanced scorecard data based on varying levels of metric standardization and a lack of explicit collection responsibility. To help ensure scorecard data is aggregated in a timely, accurate fashion from across its distributed operations, Cemex’s IT organization has laid out an eight-step process for scorecard data collection:

1. Data Request—To populate the scorecard in advance of its quarterly review, a data collection coordinator (a staff member of a corporate CIO direct report) requests information from a network of regional collection agents.

2. Data Collection—The four regional collection agents (Asia, North America, South & Central America, and Europe) coordinate with a network of local

metric experts to collect the individual data elements required to create scorecard metrics.

3. Data Proofi ng—Once the required data have been extracted from source systems and aggregated by the regional collection agents, they are passed along to the collection coordinator, who conducts an initial check of data completeness and accuracy.

4. Data Verifi cation—If the required data are not present or a particular data point seems out of line with past performance, the collection coordinator then directly taps the relevant local metric expert for verifi cation.

5. Data Finalization—Once the local metric expert verifi es the accuracy of the data they are passed by the collection coordinator to the owners of the objectives on Cemex’s IT balanced scorecard. There are 16 objective owners, drawn from the ranks of Cemex’s IT directors and managers.

6. Data Analysis—The objective owners are charged with analysis of the data related to their assigned objectives. Cemex’s IT organization deliberately absolves objective owners of the mechanics of data collection, believing their time is better spent on trend and variance analysis and metric target setting.

7. Scorecard Creation—Post-analysis, the objective owners work with the collection coordinator to create the balanced scorecard, aggregating metrics for each of their objectives and providing comments with respect to performance against pre-defi ned targets.

8. Quarterly Presentation—Finally, the objective owners present the balanced scorecard to the CIO and IT management quarterly. This group identifi es areas for performance improvement and develops initiatives to address those problems. The scorecard is then made available to all IT staff via the IT intranet, while only the metrics that fl ow upward to the corporate balanced scorecard are shared outside of IT.

The collection and quality assurance process, from initial request by the collection coordinator to delivery of the requested information to objective owners, takes approximately two weeks and involves 30 IT staff, who each spend a small portion of their time on balanced scorecard activities. Time spent by individuals varies slightly by the number of metrics collected in each region, and Cemex’s IT organization claims that the total time investment of all staff involved is equivalent to 1 to 2 FTEs for two weeks.

Data Collection and Quality Assurance


Routinizing Scorecard Data AggregationCemex’s IT organization clearly defi nes a data collection process and roles to ensure timely, accurate scorecard data

Cemex’s IT Balanced Scorecard Data Collection ProcessIllustrative

Data Request Data Collection Data Proofi ng

• Collection coordinator (a direct report to director level) requests data from regional collection agents

• Four regional collection agents coordinate with local metric experts to collect requested data

• Collection coordinator receives data from regional collection agent and conducts initial accuracy check

• In case of data inaccuracies, data are sent back to local metric experts for verifi cation

Data Verifi cationQuarterly Scorecard Presentation

• IT’s balanced scorecard is presented to CIO and IT management quarterly by objective owners

• After presentation, scorecard is made available to IT department

• Local metric experts verify data and communicate them back to collection coordinator

Scorecard Creation Data Analysis Data Finalization

• Collection coordinator aggregates data and populates balanced scorecard template

• Objective owner receives and analyzes data for assigned objective

• Collection coordinator receives fi nal data and passes them to objective owner

1 2 3

4

567

8Where the Going Gets Tough

“It is easy to defi ne a great model on paper for a balanced scorecard. However, a consistent and reliable process for data collection and analysis is key in order to make it work.”

Sergio J. Escobedo IT Planning, Cemex

Source: Cemex; Working Council research.

Data Collection and Quality Assurance



Making the Scorecard a Living Document

Cultivating a Long-Term Balanced Scorecard Perspective

While offering a snapshot of IT’s current performance, the IT balanced scorecard can also help CIOs and business decision makers track IT performance over time. This requires a sustained commitment by IT management, including review of the scorecard on an ongoing basis to align scorecard metrics with changing business goals and strategies, and the creation of a process to help drive adoption of the scorecard to all levels of the IT organization.

Company Background

Corning Incorporated, the Corning, New York–based manufacturer of fi ber-optic and telecommunication equipment, has $3.2 billion in revenues and 23,200 employees.


Although the balanced scorecard has not been adopted at the corporate level, Corning’s IT organization has been using a balanced scorecard since 2000. The scorecard is composed of four categories: value creation, customer-facing process excellence, internal process excellence, and talent management.

Ensuring Scorecard Evolution

To maintain the continued viability of its IT balanced scorecard, Corning adapts it to keep pace with shifting business and IT strategies. The company is currently using the fourth generation of its IT balanced scorecard, and while the four main categories have remained the same on a year-over-year basis, the individual metrics within those categories have changed over time. For example, in response to the sharp decline of its telecommunications-related businesses in 2001, Corning modifi ed the objective of its fi nancial performance category from enabling business profi tability to cash

conservation and profi tability within IT to refl ect the “new reality” of required cost cutting in 2002. In addition, the weightings of metrics carried over from year to year have been changed and new metrics have been added to underscore the need for cost discipline within IT. Corning’s aim is to change the set of scorecard metrics only as much as is absolutely required in order to preserve a comparable multi-year view of IT performance.

Building a Repeatable Process

To calibrate the scorecard’s metrics with the IT strategy, senior IT leaders engage in an annual series of three meetings. During the initial meeting in June, the CIO and senior IT managers review the current scorecard metrics to evaluate if each can be mapped to some aspect of the coming year’s IT strategy, identifying opportunities for removing old metrics or areas where new metrics are needed. The second meeting occurs in September, and the group focuses on evaluating a list of potential new metrics for use in the coming year. A third meeting in December results in the selection of new metrics and their addition to the scorecard.

Moving from Lagging to Leading Indicators of IT Performance

Across the past four years, this reevaluation process has also shifted the focus of Corning’s IT balanced scorecard metrics from lagging indicators of IT performance toward more forward-looking measures, which Corning calls “windshield” metrics. For example, the operational metrics on Corning’s scorecard have shifted from tracking post-deployment service interruptions in 2000 to improving the pre-deployment planning process in 2003, helping the company to avoid potential problems before they occur. In addition, Corning has used the review to reduce the number of individual scorecard metrics from 15 in 2002 to 7 in 2003.

Scorecard Review and Revision


Scorecard Review and Revision

Ensuring Continued Scorecard Comparability and RelevanceCorning adjusts its IT strategy and metrics to respond to the changing business environment…

Objectives and Metrics for Financial Performance Category of Corning’s IT Balanced Scorecard2000–2003

…while migrating from lagging to leading measures at the level of individual metrics

Operational Performance Metrics of Corning’s IT Balanced Scorecard2000–2003

2000 2001 2002 2003Objective Maximize the value of dollars invested

in ITComplete, commit to, and communicate a comprehensive IT strategy for identifi ed organization

Today’s reality—conserve cash, return to profi tability, protect the future

Support the company’s strategy at a signifi cantly lower cost

Metrics • Contribution to business cost and revenue objectives (100)

• Percentage core service offerings at benchmark (50)

• Percentage IT spend in application development (50)

• Variance to global IT budget (50)

• Identifi ed business/functional units have an IT strategy (125)

• Contributions to business profi t objectives (75)

• Percentage IT planned spending in application development (50)

• Performance to spending targets (90)

• Strengthen IT strategies (160)

• Meet commitments for worldwide IT cost (150)

• Updated and integrate unit IT strategies (100)

As corporate business strategies change in response to external economic factors…

…and new metrics are added to refl ect new demands on IT

2000 2001 2002 2003Operational Performance Metrics

• Application availability (100)• Percentage core infrastructure

offerings achieving commitments (100)

• Application availability (100)• Percentage core infrastructure offerings achieving commitments

(75)• IT expectations established for Corning employees (75)

• Percentage of service commitments met (150)

• Implement customer feedback process and system (100)

• Strengthen and expand processes to plan and coordinate work (250)

Metrics focus on performance of systems that have already been deployed

Forward-looking operational expectations added to existing metrics

Measures of customer feedback added to proactively improve service offering

Metrics focus on pre-deployment systems planning to avoid potential future problems

Source: Corning; Working Council research.

…metrics that are retained in scorecard change weighting based on changing business priorities…



Encouraging Scorecard Use at All Levels of the IT Organization

Driving Scorecard Adoption

Corning’s second step in creating a sustainable IT balanced scorecard process was driving the adoption of the scorecard throughout all levels of the IT organization. To achieve this, Corning employs two strategies:

Creating a Customizable Template—Corning’s enterprise-level scorecard is fed by IT scorecards for each IT functional area and business unit–based IT organization. The corporate scorecard’s four categories are evenly weighted for purposes of incentive compensation, and while each functional or business unit–based IT organization must use the corporate-standard balanced scorecard template, they are free to alter the category weightings as they see fi t as those weightings serve to determine their functional or business unit–level incentive payouts. For example, in the case of the head of infrastructure services, a portion of his incentive payout would be linked to the performance of the enterprise-level scorecard, while another portion would be contingent upon functional scorecard performance. That said, business unit scorecards must include metrics in each of the four categories, but individual categories can be weighted as low as 5 percent. By using a system of tiered scorecards, Corning aims to encourage a baseline level of scorecard standardization while allowing each individual IT group the freedom to focus its scorecard efforts on specifi c categories or metrics.

Linking Scorecard Use to Individual Compensation—To encourage the initial adoption of the IT balanced scorecard during its fi rst two years of use, Corning linked the compensation of its 10 most senior IT staff to scorecard performance. To reinforce this link between overall IT and individual performance, Corning appends a document that it calls an “odometer” to the balanced scorecard.

This document lists more detailed targets than those refl ected on the balanced scorecard, as well as actual performance for each scorecard metric. These automotive metaphors are derived from a larger IT improvement initiative called “Journey to Excellence.” The CIO chose to continue the metaphor in the IT balanced scorecard, referring to it as the “road map” for IT improvement. The “odometer” is the device that Corning uses to assess its progress along the “journey” and includes a maximum of 1000 “miles,” with 250 miles allocated to each of the scorecard’s equally weighted categories. Within a scorecard category, each metric target has an assigned “mileage,” with each incremental mile representing an improvement in performance. The readings on the odometer are linked to the performance bonuses for individual IT staff and can result in payouts ranging from zero percent to 140 percent of potential bonus payments. The rule of thumb for bonus calculation is that if total mileage is below 600 miles out of a possible 1,000, no bonus is paid out. Based on the early use of this process with the 10 senior IT executives, Corning has expanded the program, and by 2002 Corning had linked scorecard performance with the bonus structure of approximately 200 corporate IT staff.

Balancing Individualand Enterprise Interests

“If you don’t link the balanced scorecard to individual performance, all you have is a nice-looking document.”

Richard Fishburn CIO, Corning Incorporated

Facilitating Scorecard Adoption


Facilitating Scorecard Adoption

Making the Scorecard Relevant to the Whole IT OrganizationTo facilitate scorecard adoption, Corning creates

a tiered system of customizable scorecards for each functional and business unit IT group…

…and makes a substantial portion of bonus compensation contingent on achievement of scorecard goals

2003 Scorecard Performance “Odometer*” for Talent ManagementIllustrative

Enterprise-Level ITBalanced Scorecard

Financial Performance: 25%

Project Performance: 25%

Operational Performance: 25%

Talent Management: 25%

Display Technologies Scorecard





Infrastructure ServicesScorecard





Talent ManagementObjective: Effectively transition the workforce to new model Max. = 100 miles

By the end of February 2002, each unit CIO and the IT shared services leader will have a transition plan for their employees.

All units meet their plan, some late ¨ 60 miles

All unit meet their plan þ 80 miles

All units meet their plan, some early ¨ 100 miles

These measures will be confi rmed by a random survey of employees taken Q2, Q3, and Q4

Objective: Meet employees’ learning plans Max. = 150 miles

Each unit CIO will count the total number of learning plan items on 2003 Learning Plans and measure achievement of those plans. All will use the following defi nitions of success:

a. 88 percent of learning plan items met

b. 94 percent of learning plan items met

c. 100 percent of learning plan items met

¨ 90 miles

¨ 120 miles

þ 150 miles

Total Actual/Possible 230/250 miles

Clearly defi ned performance targets linked with value to individual

Enterprise-level IT scorecard fed by functional and business unit IT scorecards

Bonus compensation of 200 IT employees dependent to varying degrees on “odometer mileage” of corporate balanced scorecard


Corporate scorecard weights all categories equally

Business units and functions use corporate template but can weight each category as they see fi t

Set of core metrics required at business unit/functional level, but units/functions can add supplemental metrics

* For a more detailed version of Corning’s odometer please see p. 84.



Designing and Implementing an IT Balanced Scorecard

Key Takeaways

1. Basic Principles of Scorecard Design—Exemplar IT balanced scorecards exhibit six key characteristics: a concise set of top-level metrics expressed in nontechnical terms; metrics devolved from the annual IT strategic plan; senior-level scorecard ownership; enterprise-standard metrics defi nitions; scorecard drill-down capability and metric content; and clear links between individual compensation and scorecard performance.

2. Metrics in the Ascent—In addition to the cardinal IT balanced scorecard categories of fi nancial performance, project performance, operational performance, customer satisfaction, and talent management, progressive companies are elevating metrics tracking information security and enterprise initiatives to the category level to ensure they receive the required IT and business sponsor visibility.

3. Ensuring Continued Scorecard Relevance—Exemplar IT balanced scorecard practitioners create closed-loop processes for updating scorecard categories and metrics as business and IT strategies change.


Designing and Implementing an IT Balanced Scorecard (continued)

Key Takeaways

4. Formalizing Data Collection—Scorecards are only useful decision-making tools if decision makers have confi dence in the freshness and accuracy of the information they contain. Consequently, exemplar IT balanced scorecard practitioners are creating clearly defi ned collection processes and task-specialized roles to ensure data quality.

5. Facilitating Scorecard Adoption—In addition to linking individual compensation with IT balanced scorecard performance, exemplars are also deploying tiered scorecard structures to help drive adoption across the IT organization.

Questions for Discussion

1. What are the major hurdles to the adoption of an IT balanced scorecard?

2. What key “lessons learned” do experienced IT balanced scorecard practitioners have for IT organizations that are considering implementing an IT balanced scorecard?

3. What metrics are companies tracking on their IT balanced scorecards that lead to more effective decision making?

4. What specifi c decisions are companies using their IT balanced scorecard to make? What positive outcomes have they seen as a result?

5. What strategies have companies employed to reduce the administrative cost of maintaining an IT balanced scorecard?

66


Appendix ICollected IT Balanced Scorecards


This appendix includes complete versions of the corporate IT balanced scorecards used by the companies profi led in this brief. In some cases the scorecard format and language have been slightly modifi ed to allow for greater comparability and to remove company-specifi c terminology.

Schlumberger p. 68

Bowne p. 69

T. Rowe Price p. 72

AstraZeneca p. 74

Eli Lilly p. 77

Federal Aviation Administration p. 78

Cemex p. 80

Corning p. 82

J.D. Edwards (End-User Survey) p. 85



Schlumberger’s Information Technology Balanced Scorecard

Category Metric Current Performance Benchmark Variation

Financial Performance Projected cost for IT services

Year-to-date cost for IT services

IT cost per seat

Cost of data communication per seat

Quality of Service and User Satisfaction Tickets per registered user per month

Help-desk fi rst-call resolution rate

Project Timeliness Average application project delay

Core Services Activity increase on corporate knowledge portal

Percentage of standardized PCs

Portfolio StrategyPercentage of spending by category

Competitive positioning

Customer care and services

Field operations

Finance

HR

Employee effi ciency

IT effi ciency and support

Management, IT operations, and other

People IT headcount

Projected Cost of Outsourcing To SchlumbergerSema

To SchlumbergerSema-Network and Infrastructure Solutions

To external providers

Security Incident rate

Percentage of infrastructure protected

Percentage of sites with valid IS audit


© 2003 Corporate Executive Board Appendix I: Collected IT Balanced Scorecards 69

Bowne’s Information Technology Balanced Scorecard

Financial

Objective Metric Current Performance Owner Initiatives Status Owner

Maximize earnings contribution

Standard cost for service compared to industry • Develop cost models for products and services

Enable revenue growth

Percentage utilization of Bowne technology versus business demand plan

• Implement process for time reporting, to enable utilization tracking

Return on technology investment • Develop a reporting process to monitor the return on the technology investment

Optimize IT investment Percentage of the projected ROI achievement • Audit business cases for validation of ROI percentages

Reduce cost base Year-over-year percentage of operating savings of like services

• Centralize contract and license negotiations and purchasing efforts where effi ciencies would result

Client


Understand Bowne strategies and processes

General client survey results • Develop client survey

Provide undisputed IT expertise

General client survey results • Partner with Marketing to showcase IT skills (e.g., create technical competitive analysis sheets for products, tech papers)

• Develop client survey

Deliver superior service at a competitive price

Total cost of ownership of identifi ed products and services compared to industry standards

• Conduct benchmarking study comparing IT costs versus market (companies with similar services and levels)

Percentage technology availability (includes: prod-uct delivery, unplanned business interruptions)

• Pilot SLAs• Pilot technology availability measurement process

Enable productivity improvements

Percentage variance of actual ROI vs. ROI proposed in business case

• Audit business cases• Implement estimating system

Actual vs. planned business productivity for imple-mentation of key initiatives

• Develop project plan for composition workfl ow• Develop technology road map• Implement NetFax• Integrate BowneLink for eDistribution

Provide Bowne with competitive advantages

General client survey results • Create a technical competitive analysis sheet for products


Client


Understand Bowne strategies and processes

General client survey results • Develop client survey

Provide undisputed IT expertise

General client survey results • Partner with Marketing to showcase IT skills (e.g., create technical competitive analysis sheets for products, tech papers)

• Develop client survey

Deliver superior service at a competitive price

Total cost of ownership of identifi ed products and services compared to industry standards

• Conduct benchmarking study comparing IT costs versus market (companies with similar services and levels)

Percentage technology availability (includes: prod-uct delivery, unplanned business interruptions)

• Pilot SLAs• Pilot technology availability measurement process

Enable productivity improvements

Percentage variance of actual ROI versus ROI proposed in business case

• Audit business cases• Implement estimating system

Actual versus planned business productivity for implementation of key initiatives

• Develop project plan for composition work fl ow• Develop technology road map• Implement NetFax• Integrate BowneLink for eDistribution

Provide Bowne with competitive advantages

General client survey results • Create a technical competitive analysis sheet for products




Secure the Base


Manage product and service quality

Percentage of technology availability (includes product delivery, unplanned business interruptions)

• Implement project tracking and monitoring process• Pilot SLAs• Pilot technology availability measurement process

Internal audit (including standards, policies, procedures, percentage on project central)

• Develop standard process for products and services implementation (including system development, testing, deployment)

Protect information assets

Security audit results • Establish security infrastructure (including security offi ce, strategy, policies)

• Develop asset library• Data center consolidation and redundancy

Optimize IT processes

Number of tickets per application • Develop root cause analysis process

Percentage of systems adhering to enterprise systems standards

• Defi ne, publish, and enforce enterprise systems standards based on best practices

• Develop enterprise technology inventory


Partner with Clients


Improve and expand products and solutions

Customer (business sponsor) satisfaction grade • Implement standards and process to IT products and solutions

Supply cost-effi cient capacity that can be fl exed with business cycles

Percentage of utilization versus plan (labor and equipment)

• Develop a process for time reporting to enable utilization tracking

Supply and enable global Client Relationship Management capabilities

Percentage of technology availability of CRM system

• Complete CRM implementation

Align with the business units and processes

Percentage completion of identifi ed IT-business alignments among business units and between the business units and processesAdherence to timelines for delivery of IT-busi-ness alignments

• Complete IT-business alignments with three business units, operations platforms, and one corporate function



Create Breakthrough Value


Partner with Business to seize market opportunities by deploying timely, cost-effective and integrated new solutions

Percentage of investment in IT new product development resulting in new production products or services

Develop a new product development life-cycle process

Create alliances and partnerships to capitalize on new technologies

Percentage “strategic” partnership coverage Develop “supplier ecosystem” map

Recognize the potential business value in new technologies

Percentage of IT R&D investment resulting in operational application

Develop fi ve-year technology road map (including document life cycle and infrastructure)

People


Recruit, train, and develop a diverse high-performance workforce

Regretted turnover • Assess management needs for new-hire selection skills

Percentage of successful targeted selection • Provide targeted new-hire selection training as required

Percentage of individual training objectives metemployee survey results

• Implement plans for pilot group

Develop employee product knowledge alignment with the business

Results technology employee “business knowledge” survey (product knowledge and customer alignment tools)

• Establish focus groups with other business units to provide product knowledge exchange

• Customer-supplier mapping conducted with business segments• Develop technology “Business Knowledge Survey”

Align compensation and incen-tives to the business strategy

Employee survey results • Align and enhance existing compensation and incentives system

Percentage of objectives met (relative percentage of objective—e.g., threshold, target, stretch)

• Implement performance management system• Implement the S.M.A.R.T. goal-setting program

Provide opportunities for career growth

Percentage of employees with individual development plans

• Implement training

Percentage of goals achieved as documented in the S.M.A.R.T. system

• Refi ne and implement description and policies for technical and managerial career tracks (the career ladder)

Employee survey results • Develop fi ve-star skills matrix assessment development program for the support center




T. Rowe Price’s Information Technology Balanced Scorecard


Objective Metric Current Quarter Performance

Current Quarter Target Status Next Quarter

Target Comments

Increase EBITDA Percentage change in EBITDA

Maximize fund performance

Weighted expense ratio versus industry

Percentage of funds rated by Morningstar at four stars or above

Manage IT expendituresTotal IT expenditures (millions)

Total expenditures on delivering new functionality

Maximize business units’ ability to add value

Percentage of strategic projects initiated with a cost-benefi t analysis

NPV delivered during payback period

Partner User Satisfaction



Target Comments

“Keep my systems running” Systems stability (partner survey)

“Quickly implement solutions”

IT’s sense of urgency (partner survey)

“Demonstrate a competitive price”

IT’s ability to demonstrate a competitive price (partner survey)

Price competitiveness—external benchmark

“Offer me innovations that create business value”

Innovation to enhance business value (partner survey)



T. Rowe Price’s Information Technology Balanced Scorecard

Internal Process (Project and Operational Performance)



Target Comments

Operational Excellence

Integrate solutions using defi ned architectures, platforms, and processes

Percentage of projects complying with architectural standards

Provide reliable and functional systems Rate of failure incidents impacting business

Effectively select and manage sourcing relationships

Percentage of strategic solutions competitively bid

Business Unit Alliance

Deliver according to plan Percentage of active projects delivered on time, on budget, with customer satisfaction

Percentage of active projects with approved project plan

Provide world-class customer service External customer rating

World-class service (partner survey)

First-call resolution rate

Understand business unit strategies and operations

Percentage of IT training spent in business units

Solutions Leadership

Propose compelling business cases for IT solutions

Percentage of discretionary spending sponsored by IT

Anticipate applications of technology in the Financial Services industry

Number of IT person-hours spent at industry event

Percentage of pilot projects evolving into full projects

Learning and Growth (Talent Management)



Target Comments

Hire, develop, and retain solid performers Percentage of non-entry-level positions fi lled internally

Average tenure of solid performers (years)

Number of candidates interviewed per open position

Foster an environment that encourages and recognizes contribution

Contribution recognition rating (associate survey)

Communicate and lead at all levels Communication and leadership rating (associate survey)

Percentage of task forces and projects led by those other than project managers




AstraZeneca’s Information Systems Vision and Balanced Scorecard


Deliver Business Value

Objective MetricIllustrative

Current Status

Previous Status Comments Actions

Taken

Be a creative, fast, effective organization

Number of business units incorporated into shared service model

Access management project implementation progress versus milestones

Percentage of staff incorporated into corporate directory

Growth through key products Percentage of Priority 1 requirements delivered for key brands

Number of brands supported globally by IS

Number of IS activities identifi ed to support individual brands

Win in the United States U.S. ERP project implementation progress versus milestones

Campaign management tools implementation versus milestones

Secure the fl ow of new products Number of clinical studies utilizing IS tools

R&D supply chain project implementation progress versus milestones

Be the fi rst choice for customers Percentage of consumer data migrated to new database

Percentage of help-desk tools updated

AstraZeneca’s IS Balanced Scorecard

AstraZeneca’s Vision for IS Globally


AstraZeneca’s Information Systems Balanced Scorecard


Transform IS Effi ciency and Effectiveness

Objective Metric Illustrative

Current Status


Taken

Improve quality and regulatory compliance process and practices

Self assessments for IS quality policy in place

Percentage of IS units with trained quality manager

Annual avoided IS costs

Percentage of targeted savings delivered

KPIs and measures of benefi ts developed

Vendor audit database rolled out

Percentage reduction in re-validation work

Corrective actions from audits implemented

Rating versus goals for quality workshops

Global quality training content developed

Step-up project portfolio management

IS portfolio management process in place

IS “value model” established

Two-year IS portfolio model ready

Agreement on IS portfolio by all stakeholders

Deliver information management basics to support informed decision making

Percentage of intranet policies and standards implemented

Number of intranet sites killed versus reduction target

Percentage of portal projects with robust business case

Annual avoided costs from global licenses

Project progress versus schedule estimates

Percentage of projects using standards business caseIdentify and implement application support best practices

Application support best practices document available

Percentage of business units operating with new application best practice guideline

Manage the global IT infrastructure

Actual IT infrastructure cost savings versus plan

End-user satisfaction survey versus target

Number of material adverse regulatory compliances observed by FDA or other IS audits

Number of business interruptions caused by security intrusion



AstraZeneca’s Information Systems Balanced Scorecard


Develop IS Leadership and Capabilities

Objective MetricIllustrative

Current Status


Taken

Develop and demonstrate leadership capability

Percentage of IT staff development plan reviewed and actions implemented

Percentage of incumbent IS leaders with development plan in place

Percentage of global high-potential staff development plan in place

Change management and business acumen programs in place

Number of annual IS rotational assignments

Diversity performance (scale of 1 to 5)

Percentage of key roles fi lled

Develop and demonstrate IT professional skills and disciplines

Percentage of projects following project management framework

Percentage of projects with KPIs for visible value demonstration

Percentage of project managers with development plan and career path

Percentage increase in project manager capability

Attendance versus capacity of existing learning programs

Percentage increase in collaborative tool usage

Improvement of “virtual” team productivity

IS capabilities versus benchmarks (scale of 1 to 5)

Percentage reduction in skills shortages fi lled by consultants

Transform IS Effi ciency and Effectiveness (continued)

Objective Metric Illustrative

Current Status


Taken

Develop KPIs Initiative KPIs developed

IT infrastructure services KPIs

Support and maintenance KPIs

Portfolio and project KPIs

Financial and resource management KPIs

Supplier KPIs

Leadership KPIs

Security KPIs


Eli Lilly’s Information Technology Balanced Scorecard

Source: Eli Lilly; Working Council research.



FAA’s Information Technology Balanced Scorecard


eGovernment


Current Year Target Status Milestones Risks

Ensure effective service delivery capabilities

Web site usability as determined by formal external appraisal • First report due to Department of Transportation CIO in Q2 FY03Percentage of agency Web sites that are compliant with agency and Federal

policies, rules, and regulations

Achieving “green” on eGovernment scorecard

Percentage of automated transactions using e-authentication services

Reduce the burden on agency customers by bet-ter leveraging Web-based technologies

Percentage of transactions fully automated • Inventory baselined• Agreement reached with Department

of Transportation on inventory and defi nition of “complete”

• e-authentication strategy baselined• e-authentication service interfaces

defi ned• e-authentication services available• Department of Transportation

acknowledge completion

Reduction in customer time to perform automated services versus manual baseline

Reduction in agency time and cost to deliver automated services versus manual baseline

Increase in scores on customer satisfaction survey for service quality and availability

Number of Federal agencies with which information, applications, or enterprise license agreements are shared

Ensure that data and information that are used to conduct critical agency business or publicly disseminated are timely, accurate, accessible, under-standable, and secure

Percentage of critical data and information that meet quality standards • Systems using approved data elements baselined

• Additional 250 data elements approved• Full governance process developed and

approved• Governance in place to ensure new

systems use approved data elements

Percentage of critical data and information for which there is adequate “infrastruc-ture” to maintain quality, including registration, stewardship, and a quality process

Number of information quality complaints received and percentage resolved

Increase in score on customer satisfaction survey for satisfaction with agency data and information

Percentage of information systems critical to agency business or used to disseminate information publicly that use approved data elements


Business Value


Current Year

TargetStatus Milestones Risks

Provide the right mix of qualifi ed IT professionals and IT tools for each business need of the agency

Number of professional IT certifi cates held by the workforce in areas deemed critical to the agency, such as security, architecture, Web management, IT program management

• Agreement reached on criteria used to decide if a certifi ed program manager is needed to run a program, including different levels of certifi cation

• Agreement reached on set of recognized credentials• Policy decided on how to address existing program

managers• Policy decided on how to phase in certifi cation

requirement for new program managers• Agreement reached on who specifi cally will be certifi ed

and when• Agreements reached with certifying organizations

Standardize and simplify the enterprise architecture to ensure IT investments are aligned with FAA business processes

Percentage of components, standards, and infrastructures shared across multiple applications

• Agreement on governance process for architecture

Extent to which architectures are documented in accordance with Offi ce of Management and Budget and Department of Transportation requirements

Percentage of local business unit fi nancial and human resources systems retired because of functionality provided by Enterprise Resource Planning system

FAA’s Information Technology Balanced Scorecard




Cemex’s Information Technology Balanced Scorecard


Financial

Objective Metric Current Performance Status

Optimize cost of IT services Total cost of IT operations per employee

Total cost of IT operations as percentage of total sales

Customer


Ensure consistently available and reliable services to IT users Percentage compliance with service level agreements

Ensure that IT solutions meet the business needs Business alignment index (executive management survey)

Meet employee IT service needs Level of service-mindedness and professionalism (end-user survey)



Cemex’s Information Technology Balanced Scorecard

Process


Excel at managing projects Percentage of deviation from plan of projects exceeding budgeted time

Percentage deviation from plan of projects exceeding budget

Percentage of IT projects budget managed by program management offi ce

Operate data centers effi ciently System availability

Number of unplanned system/network stoppages

Percentage of servers outside of CPU utilization standards

Maintain reliable offi ce computing and telecom infrastructures Network reliability index

Percentage of obsolescence (PC, data, and voice networks)

Provide effective user support for IT services Weighted average of incident/problem resolution time by priority

Percentage of incidents resolved at fi rst level

Learning and Innovation


Develop operational, coordination, and project management capabilities in the IT staff

Percentage of IT staff certifi ed in project management

Percentage of IT staff certifi ed in coordination skills

Percentage of IT staff certifi ed in operational cost structure understanding

Partner proactively with business leaders to establish trustworthy communication

Percentage of eGroups initiatives with IT participation

Business partner relationship index (end-user and executive survey)

Develop a customer service orientation within IT Percentage of IT staff completing courses in customer service

Percentage of IT staff passing courses in SLA development



Corning’s Information Technology “Road Map”A Journey to Excellence

To facilitate its ongoing performance improvement initiative, “Journey to Excellence,” Corning’s IT organization has adopted a set of automotive metaphors. The IT balanced scorecard is the “road map” for IT improvement and includes a maximum of 1,000 “miles,” with each mile designating further progress toward established goals. Each of the scorecard’s four categories is equally weighted at 250 total miles. Corning supplements the “road map” with an “odometer”—a more detailed view of scorecard metrics that allows Corning to assess its progress along against goals, and is one of the inputs used to determine bonus compensation. For additional details about Corning’s “odometer” please see page 63.

Corning’s “Road Map” for 2003: IT Strategic Goals

Measures of Success

1. Meet commitments for worldwide IT costs

2. Update and integrate unit IT strategies

150 miles

100 miles

Total 250 miles

Measures of Success

6. Effectively transition the workforce to the new model

7. Meet employees’ learning plans

100 miles

150 miles

Total 250 miles

Measures of Success

5. Strengthen and expand processes to plan and coordinate work 250 miles

Total 250 miles

Measures of Success

3. Meet project commitments

4. Meet service commitments

125 miles

125 miles

Total 250 miles

Objective

Support the company’s strategy at a signifi cantly lower cost

Objective

Deliver the projects and services which our customers require

Objective

Standardize the processes which are critical to our success

Objective

Effectively transition to, and begin operating in, a new organizational model

Value PeopleIT ExcellenceProcess Excellence



Corning’s Information Technology “Odometer”


Value—Support the company’s strategy at a signifi cantly lower cost

Objective Metric Potential Score Performance Scale

Meet commitments for worldwide IT cost

Worldwide IT spending performance versus goal 150 Miles 100% of goal98% of goal96.5% of goal

90 Miles120 Miles150 Miles

Update and integrate unit IT strategies

Business and staff unit strategy creation maturity trajectory (Includes IT strategy refreshment, identifi cation of shared services leverage points, and creation of a program portfolio)

100 Miles Unit IT strategy refreshed to refl ect changes in business strategy Strategy refreshed, including opportunities to leverage global shared services organizationsStrategy refreshed, shared services leveraged, and 2004 programs incorporated into operating plan

60 Miles 80 Miles

100 Miles

Process Excellence—Deliver the projects and services which our customers require


Meet project commitments

Project performance index measure for 20 representative projects selected by unit CIO/program offi ce (Evaluation dimensions include business value and capabilities delivered, fi nancial and schedule performance, project management quality)

125 Miles Average score 1.8Average score 2.0Average score 2.2


Meet service commitments

SLA compliance for 10 key shared services (Shared service managers/unit CIOs select services)

100 Miles Minimal threshold of commitments met All commitments met All commitments met, plans in place to reduce costs in 2004

6 Miles/Service8 Miles/Service

10 Miles/Service

Average score across all units on general manager satisfaction survey (Five-point scale where 1 = does not meet expectations, 3 = meets expectations, 5 = exceeds expectations)

25 Miles Average score 3.0 Average score 3.5 Average score 4.0




People—Effectively transition to, and begin operating in, a new organizational model


Effectively transition the workforce to new model

Number of unit CIOs and IT shared services leaders creating a transition plan for their employees within designated time frame

100 Miles All units meet their plan, some lateAll units meet their planAll units meet their plan, some early

60 Miles80 Miles

100 Miles

Meet employees’ learning plans

Percentage of staff development plan objectives met 150 Miles 88% of development plan objectives met94% of development plan objectives met100% of development plan objectives met


1000 Miles

IT Excellence—Standardize the processes which are critical to our success


Strengthen and expand processes to plan and coordinate work

Defi nition, implementation, and measurement of the following processes:

250 Miles Process fully defi ned, pilots completeProcess fully deployed and measured 2004 improvement plans in place

30 Miles/Process40 Miles/Process50 Miles/Process• Performance excellence

• Demand management• Time tracking

• Business case analysis• Prioritization

Corning’s Information Technology “Odometer”



J.D. Edwards’ IT User Satisfaction Survey

General Information

1. Please rate your overall SATISFACTION with J.D. Edwards IT servicesa. Overall performance of J.D. Edwards IT Servicesb. Ability to deliver technical/business solutions and servicesc. Communication about available services and new technologiesd. Ease of doing business

2. Please rate the CONFIDENCE you have that the core IT services will be up and performing to your expectations-(Core services are Phone, PC, Network, Servers in the Data Center and Business Applications)

3. Please rate IT�s ability to be PROACTIVE in providing solutions to meet your business needs.

Network

4. Please rate your SATISFACTION with the J.D.Edwards network and network services (Connectivity to Internet, E-mail, Applications,Data Files and Network Printing).a. Overall performanceb. Speed and reliability of Internet access

The J.D. Edwards Information Technology department would like you to take a few minutes and complete the following satisfaction survey. The valuable feedback you provide will be used to improve the tools and services IT provides the JDE Enterprise. If any question does not seem to apply to you, please skip to the next question. When you are Þ nished, please click the Submit Survey button at the end of the survey.

Thank you for your time and help.

c. Speed and reliability of access to data on Þ le serversd. Speed and reliability of e-mail systeme. Speed and reliability of network printing

5. What could IT do to better meet your needs for the network and network services?

PC Support

6. Please rate your satisfaction with PC services and supporta. Overall performanceb. Promptness in responding to your problems/issuesc. PC equipment provided meets your needsd. Ability of staff to understand and address the issuee. Satisfaction with initial PC setup or EOL replacement

7. What could IT do to better meet your PC needs and requirements?




Help Desk

8. Please rate your satisfaction with the IT Help Deska. Overall Performance

b. Timelessness in providing repairs or servicesc. Courteousness of HD Staffd. Ability of HD staff to understand and address the issuee. Timelessness of status updates

9. How could IT improve the Help Desk to better meet your needs?

Application Support

10. Please rate your SATISFACTION with maintenance and support of corporate applications-(OnWorld, Expense Express, Ariba, Rolling Thunder, Augeo, Aprimo, On-Track, Personic)a. Overall performance of corporate applicationsb. Quality of Corporate Applicationsc. Promptness in responding to your application problems and issuesd. Ongoing Support of corporate applicationse. Ability to deliver approved strategic business systems changesf. Ease of use of corporate applications

11. How can IT improve the corporate applications to better meet your needs? Please be speciÞ c regarding what application you are providing feedback on- OneWorld, Expense Express, Ariba, Rolling Thunder, Augeo, Aprimo, On-Track, Personic).

Voice Services

12. Please rate your SATISFACTION with the IT voice services (Phones, Cellular, Pager, Calling Cards)a. Overall performance of internal voice services and supportb. Information provided to you about voice servicesc. Promptness in responding to your questions/concernsd. Timeliness in providing solutionse. Completeness of solutions

13. What could IT do to better meet your needs for voice services?

Field OfÞ ce Support

14. Please rate your satisfaction with your local Þ eld ofÞ ce support. If you work in the Denver Corporate ofÞ ce please skip to question #16.a. Overall performance of local IT staffb. Courteousness of local IT staffc. Ability of local IT staff to understand and address the problem/issued. Communication by local staff about available services and new technologiese. Follow up and closure of reported problems.

15. How could IT improve your local services to better meet your needs?





Remote Access

16. Please rate your satisfaction with IT remote access (or dial up) services. If you do not utilize remote access in your position at JDE please skip to question #18.a. Ability to use broadband services top connect to JDEb. Ability to use wireless access in any location that has it availablec. Communication about available remote access servicesd. Reliability of dial up accesse. Help Desk support of broadband services

17. How could IT improve remote access to better meet your needs?

18. We welcome any other general comments you may have about J.D. Edwards IT.


Thank you for completing the survey.

Please submit the survey by clicking on the button below.

Submit

88


Appendix IIIT Balanced Scorecard Tools and Vendors


Despite this wealth of off-the-shelf offerings, Working Council research reveals that most companies do not seek outside help or buy application packages to produce scorecard reports, but are instead using Excel or similar homegrown solutions which require manual data input.

Kaplan and Norton’s balanced scorecard concept has spawned the creation of a new complement of tools and vendors aimed at automating balanced scorecard data collection and presentation. Some of these vendors provide tools that enable the presentation of data, offering Web-based solutions that provide structured data input and graphical presentation. In addition, leading database and ERP vendors are also creating scorecard modules for their product suites, providing their customers with an easily integrated tool that aggregates and displays basic scorecard data.

The following vendor briefs are designed to provide members who are considering the purchase of balanced scorecard software tools, with an overview of the major players in the balanced scorecard space. Each brief includes vendor contact information, an outline of selected functionality, and a selection of major customers.



Vendor and Product Contact Information Selected Functionality Selected Customers

Hyperion

Hyperion Performance Scorecards

1344 Crossman Avenue Sunnyvale, CAU.S.A. 94089Tel: 408-744-9500 Web: http://www.hyperion.com

• Integrated dashboard and scorecard application • Budgeting and forecasting tools can directly feed scorecard tools• Business intelligence platform integrates Hyperion reporting tools with existing

systems (CRM, ERP, legacy systems)

AirbusBoston Beer CompanyDHLFrance Telecom

Panorama Business Views

pbviews

50 John StreetSuite 600 Toronto, Ontario M5V 3E3, CanadaTel: 800-449-3804Web: http://www.pbviews.com

• Scorecard presentation tool • Modular, Web-based system providing drill-down capability • Application training and integration services available

Bank of AmericaCanadian Dept. of National DefenceGlaxoSmithKlineVerizon Communications

ProDacapo

ProDacapo Balanced Scorecard

Barnhusgatan 4, 4 tr SE-111 23 Stockholm, SwedenTel: +46 (0)8-622-25-00Web: http://www.prodacapo.com

• Web-based balanced scorecard, dashboard, and cost management tools• Training, consulting, and tool implementation and integration services available• Pre-built with connectors to most existing IT systems

ABBICAScania

QPR

QPR Scorecard

Sörnäisten rantatie 27 A, 3. kerros 00500 Helsinki, Finland Tel: +358 (0)9 4785 411Web: http://www.qpr.com

• Web-based balanced scorecard presentation tool• Integrates with SQL data warehouse

CanonElectroluxSwisscomStoraEnso

Balanced Scorecard Software ToolsSoftware that aggregates data from source systems and presents them in a balanced scorecard format. Data are usually semiautomatically or manually updated.

© 2003 Corporate Executive Board Appendix II: IT Balanced Scorecard Tools and Vendors 91


Cognos

Enterprise Scorecard

3755 Riverside Drive Ottawa, ON K1G 4K9, CanadaTel: 613-738-1440Web: http://www.cognos.com

• Product suite for strategy planning, monitoring, and presenting scorecard data• Uses OLAP data from any source• Graphical presentation of data with historical view• Consulting and training services available

BMWDowHarrah’sKeyCorpNASASiemens

CorVu

RapidScorecard

3400 West 66th Street Suite 445 Edina, MNU.S.A. 55435Web: http://www.corvu.com

• Scorecard presentation tool that allows for drill-down into both the scorecard data and strategy maps

• Consulting and training services available

Bowne

Crystal Decisions

Crystal Performance Scorecards

895 Emerson Street Palo Alto, CAU.S.A. 94301-2413 Tel: 800-877-2340Web: http://www.crystaldecisions.com

• Web-based reporting and analysis tool• Allows for ad hoc customization of reports• Out-of-the-box connectivity for major ERP systems• Consulting and training services available

AetnaCanfor CorporationFox Filmed Entertainment

Balanced Scorecard Software Tools and ConsultingVendors that provide both balanced scorecard training and consulting and software for data collection and presentation.



Bolt-Ons to Enterprise SystemsPresentation applications that integrate with vendors’ enterprise resource planning suites to extract data and present them in a balanced scorecard format.


Oracle

Business Intelligence Applications

500 Oracle ParkwayRedwood City, CAU.S.A. 94065Tel: 650-506-7000Web: http://www.oracle.com

• Part of Oracle’s Business Intelligence suite• Presentation tool that reports defi ned PKIs and extracts data from existing

Oracle systems

First UnionMobistar

PeopleSoft

Enterprise Scorecard

4460 Hacienda Drive Pleasanton, CAU.S.A. 94588-8618Tel: 800-380-SOFT (7638)Web: http://www.peoplesoft.com

• Presentation tool automatically extracts data from relevant modules of Peoplesoft suite

Danske BankEntergy

SAP

mySAP Business Intelligence Solution

Neurottstrasse 16 69190 Walldorf, Germany Tel: +49-6227-7-47474 Web: http://www.sap.com

• Part of SAP Strategic Enterprise Management suite • Automatically collects data from relevant SAP modules

AvayaThe Coca-Cola CompanyHenkelThe Woolwich

SAS

SAS Balanced Scorecard

100 SAS Campus Drive Cary, NCU.S.A. 27513-2414 Tel: 919-677-8000Web: http://www.sas.com

• Aggregates and presents data from SAS applications in a balanced scorecard format

Generali-ProvidenciaING BankQuaker Chemical

© 2003 Corporate Executive Board With Sincere Appreciation 93

Agere Systems Inc.

Agilent Technologies, Inc.

Amersham plc

AXA Australia

Bechtel Corporation

With Sincere Appreciation

Special Thanks

The Working Council for Chief Information Offi cers would like to express its gratitude to those individuals and organizations that have been so generous with their time and expertise throughout the preparation of this study.

Mr. Balvinder DhillonIS ManagerAstraZeneca PLC

Mr. Richard FishburnVice President and Chief Information Offi cerCorning Incorporated

Mr. Jan ChodzkoIT ArchitectSchlumberger Limited

Ms. Ruth HarencharChief Information Offi cerBowne & Co.

Mr. Roy DunbarVice President and Chief Information Offi cerEli Lilly and Company

Mr. Bob GravienVice President Application DevelopmentSchneider National, Inc.

Mr. Sergio EscobedoIT PlanningCemex, S.A. de C.V.

Mr. Robert RovinskyProgram Director, Strategy & Investment Analysis DivisionFederal Aviation Administration

Mr. Michael ButlerIT FinanceT. Rowe Price Group, Inc.

Mr. Frank CatalfamoManager ITMetrics and M&AsJ. D. Edwards & Company

Partial List of Participating Companies E.ON AG

Exxon Mobil Corporation

Hallmark Cards, Inc.

KEMET Corporation

KeyCorp

L.L. Bean, Inc.

Microsoft Corporation

Philip Morris Companies Inc.

SBC Communications

Yellow Corporation

With Sincere Appreciation 93

94



95


You may order an unlimited number of copies without additional charge.

Study Requested Quantity

IT Balanced Scorecards: End-to-End Performance Measurementfor the Corporate IT Function

CATALOG NO.: CIO1L9VDH ____________

Name & Title __________________________________________

Institution __________________________________________

Address __________________________________________

__________________________________________

__________________________________________

Telephone _______________________________________________

COPY AND FAX TO: Working Council for Chief Information Offi cers202-777-5270

ORDER FORM IT Balanced Scorecards: End-to-End Performance Measurement for the Corporate IT Function is intended for broad dissemination among senior executives and management. Members are welcome to unlimited copies without charge. Online ordering is available at www.cio.executiveboard.com. Alternatively, you can call the Publications Department at 202-777-5921, e-mail your order to [email protected], or fax in the order form on this page.

Additionally, members interested in reviewing any of the Working Council’s past strategic research are encouraged to request a complete listing of our work.

Working Council for Chief Information Offi cers2000 Pennsylvania Avenue NW

Washington, DC 20006Telephone: 202-777-5000

www.cio.executiveboard.com



In-Person Research Presentations

Frequently Requested Presentations

1. Structural IT Cost Effi ciency: What tools and processes are CIOs using to achieve structural IT cost effi ciency?

2. Strategic Vendor Management: How are pioneering organizations managing IT vendor performance to maximize value and fl exibility?

3. Responsive IT Portfolio Prioritization: How are leading companies structuring and reprioritizing IT portfolios to ensure continued alignment with changing business strategies?

4. End-to-End Data Visibility: What technical and organizational innovations are exemplars using to enable end-to-end data visibility?

5. Rapid Resource Redeployment: How are CIOs enabling rapid resource redeployment in response to changing priorities?

6. Aligning IT with Corporate Strategy—Case Studies in Enterprise Architecture Migration: How are premier companies creating self-funding IT architectures that mirror and advance corporate strategy?

7. IT-Enabled Collaboration: What are the most effective uses of IT-enabled collaboration? How are exemplars deploying these tools to ensure effi ciency and maximize collaboration?

8. Deploying Enterprise Portals: What are the functional capabilities and IT requirements of “world-class” portals and enterprise information systems?

9. IT Strategic Planning Excellence: What are the leading practices for aligning corporate and IT priorities?

10. Customer Relationship Management: How are exemplars rescoping CRM initiatives for high-impact management of internal and external customers?

Formal ResearchPresentation

Facilitated ResearchDiscussion

Interactive WorkingSession

One-to-One Briefi ng

An in-person research briefi ng is an interactive session that members can use to spark internal discussion and debate around strategic issues that the Working Council has researched. A senior research director will travel to a member’s location to present and discuss research fi ndings that are most relevant to member issues and challenges. This in-person research briefi ng service allows member CIOs to share Council case studies, practices, and insights with their staff and larger audiences within their organization.

Common Formats

Photo Credits: Digital Imagery® copyright 1999 PhotoDisc, Inc.

In-Person Research Presentations 96


Project Support Desk

• Research resource for strategic IT staff, available free of charge

• Customized, member-initiated research projects

• Fast turnaround (overnight to three weeks)

• For ideation in strategic planning, data support, and qualitative benchmarking for business cases and budgets

• Not intended for technology assessment, benchmarking, or competitive research

To request a project, contact your relationship manager, send an e-mail to [email protected], or visit www.cio.executiveboard.com

Custom Research for Strategic IT Staff, Available (Free) for the Asking

As a complement to our ongoing research, members are encouraged to take advantage of the Project Support Desk. A free resource to assist senior IT staff with “work in the moment,” the Project Support Desk offers fast-turnaround research for business case preparation, strategic planning exercises, and budgeting. Members describe their project goals and time constraints and then decide what combination of literature search, fact retrieval, and networking contacts best suit their needs. In keeping with the Working Council’s charter, the Project Support Desk does not conduct competitive research, benchmarking, or vendor assessments.

What are job descriptions and career paths for “IT Centers of Excellence?”

What are the featuresof world-class, IT fi nance–reporting intranet sites?

What are the key vendor contracting trends impacting my peers?

What is the typical payback period for call center technology investments?

Project Support Desk

• Research resource for strategic IT staff, available free of charge

• Customized, member-initiated research projects

• Fast turnaround (overnight to three weeks)

• For ideation in strategic planning, data support, and qualitative benchmarking for business cases and budgets

• Not intended for technology assessment, benchmarking, or competitive research

To request a project, contact your relationship manager, send an e-mail to [email protected], or visit www.cio.executiveboard.com.

Working Council Project Support Desk

IT HR

IT Finance

IT Procurement

BU CIO

Working Council Project Support Desk 97