11/29/2012

1

1

Risk Analysis

2 Chapter 5

Assessing Risk

• The potential consequences of not assessing and managing risks can include the following: – Failure to attain expected benefits from the project,

– Inaccurate project cost estimates,

– Inaccurate project duration estimates,

– Failure to achieve adequate system performance levels, and

– Failure to adequately integrate the new system with existing hardware, software, or organizational procedures.

11/29/2012

2

3 Chapter 5

Project Risk Factors

• Project size – Team size, organizational departments, project duration,

programming effort

• Project structure – New vs. renovated system, resulting organizational changes,

management commitment, user perceptions

• Development group – Familiarity with platform, software, development method, application

area, development of similar systems

• User group – Familiarity with IS development process, application area, use of

similar systems

4 Chapter 5

Assessing Risk(Cont.)

• Risk can be managed on a project by:

– Changing the project plan to avoid risky factors.

– Assigning project team members to carefully manage the risky aspects.

– Setting up monitoring methods to determine whether or not potential risk is, in fact, materializing.

11/29/2012

3

5 Chapter 5

Assessing Risk(Cont.)

• The four primary factors associated with the amount of technical risk on a given project are:

– Project size,

– Project structure,

– The development group’s experience with the application and technology area, and

– The user group’s experience with systems development projects and the application area (see also Kirsch, 2000).

6 Chapter 5

Assessing Risk(Cont.)

• Four general rules emerged as technical risk assessments: – Larger projects are riskier than smaller projects.

– A system in which the requirements are easily obtained and highly structured will be less risky than one in which requirements are messy, ill structured, ill defined, or subject to the judgment of an individual.

11/29/2012

4

7 Chapter 5

Assessing Risk (Cont.)

– The development of a system employing commonly used or standard technology will be less risky than one employing novel or nonstandard technology.

– A project is less risky when the user group is familiar with the systems development process and application area than if unfamiliar.

8 Chapter 5

Assessing Risk(Cont.)

Effects of degree of project structure, project size, and familiarity with application area on project implementation risk (Source: Based on 7th Applegate, Austin, and McFarlan. 2007; Tech Republic, 2005.)

11/29/2012

5

9

Risks and Their Implications in the Requirements Process

• Insufficient user involvement

• Creeping user requirements

• Ambiguous requirements

• Gold-plating

• Minimal requirements

• Overlooking user classes

• Incompletely defined requirements

Unacceptable product

Overruns and degraded quality

Rework and wasted time

Unnecessary features

Missing features

Dissatisfied customers

All of the above

Controlling Risk

• Project Management

• Application of knowledge, skills, tools, and techniques to achieve targets within specified budget and time constraints

• Scope

• Time

• Cost

• Quality

• Risk

11/29/2012

6

Kaiser Permanente Botches Its Kidney Transplant Center Project

• Read the case study and then discuss the following questions:

• Classify and describe the problems Kaiser faced in setting up the transplant center. What was the role of information systems and information management in these problems?

• What were the people, organization, and technology factors responsible for those problems?

• What steps would you have taken to increase the project’s chances for success?

• Were there any ethical problems created by this failed project? Explain your answer.

12

Quantifying Risk

• For each risk event we’d like to estimate two factors

• Probability – What are the chances the event will occur?

• Impact – What is the cost if it does occur?

• Together, these allow us to compute our exposure for the risk event

11/29/2012

7

13

Risk Event Exposure

Exposure = Probability x Cost

Note that the exposure is never the actual cost incurred. If

the risk event occurs, the total impact is absorbed; if it doesn’t

occur, there is no impact.

The exposure is a measure of what to expect “on the

average.” It is useful primarily for assessing the potential cost

of an aggregate of risk events.

14

Risk Exposure: An Example

• Suppose the risk event we’re considering is that we miss a delivery deadline

• We estimate the probability that this will occur to be 25%

• Suppose further that our assessment of what missing the delivery deadline will cost us is $100,000

• The exposure is then = .25 X 100,000 = $25,000 • Will this risk event ever cost us $25,000? No.

– the cost will be $0 if we deliver on time – the cost will be $100,000 if we miss the deadline

11/29/2012

8

15

Risk Exposure: An Example (cont’d)

• Suppose now that we have identified 4 main risk events for our project, each with these same characteristics: 25% probability and $100,000 cost if the event occurs.

• So our exposure on each risk is $25,000.

• We add these to get our total exposure on the project which is $100,000.

16

Qualitative Risk Assessment

• Sometimes it is very difficult to make confident estimates of probabilities and impacts associated with risk events

• In these cases, a qualitative assessment of these two factors may make more sense

• Ranking each of the two factors (low-high, low-medium-high, or similar) in a two dimensional table will still provide a basis for prioritizing risks

• Prioritization is crucial to planning and creating risk management strategies

11/29/2012

9

17

Qualitative Risk Assessment

Impact

Low High

Probability

High

Low

L H H H

L L H L

18

Question: Prioritizing Using Qualitative Assessments

How would you prioritize risk events based on assessments done using the chart on the previous slide?

HH

HL

LH

LL

or HH

LH

HL

LL

11/29/2012

10

19

Some other sources of risk

– Scope incorrectly defined

– Incomplete or incorrect requirements

– Poor estimates

– Team turnover

– Changing scope and requirements

– Poor project management and planning

– New technologies (already mentioned)

• Risks should be assessed and “managed” even though they are unpredictable

20

Risk Response Strategies

• Perform contingency planning, including: – a contingency budget – schedule alternatives, to include some built-in float – complete emergency responses designed to deal with major areas

of risk

• Develop workarounds designed to avoid or minimize selected risks

• Mitigate risks – mitigation involves adding activities/deliverables to a project to

offset the possible effect of a potential risk event – mitigation occurs before the risk event materializes – therefore mitigation costs are incurred whether the risk event

occurs or not – a kind of insurance policy against selected risk events

• Evade risk (Hope for the Best)

11/29/2012

11

21

Risk Avoidance and Mitigation

• Risks should be identified early in the project

• Stakeholders should be aware of potential risks and their impacts

• They should help devise strategies for either avoiding them or minimizing their impact

• Requires planning

• Formulate contingency plans for mitigation

• Track risk event potential and revise plan accordingly

22

1) What do you consider to be the risks in the BEC project as you currently

understand it?.

2) Is this a low, medium, or high risk project?

3) How would you propose dealing with the risks?

Group Exercise #1

11/29/2012

12

23

Software Risk: A Recent Example On the following several slides, a recent highly publicized software failure is described. Read this case study and we will then discuss its implications for our study.

Errant Trades Reveal a Risk Few Expected

Downloaded and adapted from NY Times Online, August 2, 2012

Errant trades from the Knight Capital Group began hitting the New York

Stock Exchange almost as soon as the opening bell rang on Wednesday.

The trading firm Knight Capital recently rushed to develop a computer

program so it could take advantage of a new Wall Street venue for trading

stocks. But the firm ran up against its deadline and failed to fully work out

the kinks in its system, according to people briefed on the matter. In its

debut Wednesday, the software went awry, swamping the stock market

with errant trades and putting Knight’s future in jeopardy.

Knight, founded in 1995, is a leading matchmaker for buyers and sellers of

stocks, handling 11 percent of all trading in the first half of 2012. Knight

lost three-quarters of its market value in two days, in addition to

losing $440 million from the errant trades, and was scrambling to

find financing or a new owner.

24

11/29/2012

13

Errant Trades Reveal a Risk Few Expected (cont’d)

The fiasco, the third stock trading debacle in the last five months, revived

calls for bolder changes to a computer-driven market that has been hobbled

by its own complexity and speed. Among the proposals that gained

momentum were stringent testing of computer trading programs and a

transaction tax that could reduce trading.

In the industry, there was a widespread recognition that the markets had

become more dangerous than even specialists realized. Some S.E.C.

officials are pushing new measures that would force firms to fully test coding

changes before their public debut, according to a government official who

spoke on the condition of anonymity. While the idea has long been discussed

at the agency, it gained traction after the Knight debacle.

The S.E.C. applied limited safeguards on trading after the “flash crash” of

2010 sent the broader market plummeting in a matter of minutes. But big

investors like T. Rowe Price, members of Congress and former regulators

said Thursday that the S.E.C. and the industry had been too complacent and

needed to do more to understand and control the supercharged market.

25

Errant Trades Reveal a Risk Few Expected (cont’d)

Arthur Levitt Jr., a former chairman of the Securities and Exchange

Commission, said that recent events “have scared the hell out of investors”

and called for the agency to hold hearings. “I believe this latest event was

handled better than the flash crash, but the larger question is whether our

markets are adequate to deal with the technology that is out there,” Mr. Levitt

said. “I don’t think they are.”

Regulators have made changes to the markets over the last two decades

that have taken it out of the hands of a few New York institutions and allowed

dozens of high-frequency trading firms and new trading venues to dominate

the stock market.

The high-speed firms like Knight, which connect directly to the servers of the

exchanges and are capable of executing thousands of trades a second, are

responsible for more than half of all activity in American markets. Companies

that have benefited from the fragmentation and computerization of the

markets have largely managed to fend off tighter controls by pointing to the

steady decline in the cost of trading stocks.

26

11/29/2012

14

Errant Trades Reveal a Risk Few Expected (cont’d)

But even people who had previously defended the advances in trading

technology said on Thursday that too many problems had been overlooked.

In Knight’s breakdown on Wednesday, as well as in the botched initial public

offerings of Facebook in May and BATS Global Markets in March, the

problems were caused by new computer programs that had not been

adequately tested. Currently regulators have no protocol for signing off on

new software programs like the one Knight rolled out.

“When they put these things out in the world they are really being tried for the

first time in a real-life test,” said David Leinweber, the head of the Center for

Innovative Financial Technology at the Lawrence Berkeley National

Laboratory. “For other complex systems we do offline simulation testing.”

Mr. Leinweber has suggested to the S.E.C. that it do this work with the help

of the supercomputing facilities at his center. The S.E.C. has recently moved

in this direction by contracting with a high-speed trading firm that will provide

it with more up-to-date market information.

27

28

Group Exercise 1. What factors seem to have led to the

Knight Capital Group disaster?

2. Is lack of adequate testing the only culprit here?

3. Do you think these issues might relate to requirements?

4. Could good risk analysis and management have mitigated this outcome? Explain your answer.

11/29/2012

15

29

Requirements-Related Risks • During Requirements Elicitation

– Scope creep

– Schedule pressures

– Completeness and correctness

– Defining non-functional requirements (performance, usability)

– Unstated or implicit requirements

– Customer understanding and agreement

– Customer-presented “solutions” vs. actual needs

30

Requirements-Related Risks (cont’d)

• During Requirements Analysis – Requirements Prioritization

– Technically difficult features

– New environments (hardware, software, application area)

• During Requirements Specification – Gaps in understanding and expectation

– Schedule pressure to proceed before specification is complete

– Ambiguous terminology

– Design embedded in requirements

11/29/2012

16

31

Requirements-Related Risks (cont’d)

• During Requirements Validation – Unvalidated requirements

– Poor inspection techniques for requirements

• During Change Management – Changing requirements

– Poor change process

– Unimplemented requirements

– Scope creep

32

In this activity, you will work with the proposed new online

system to automate seminar registration.

1) Identify several risks inherent in the development of the online seminar

registration system project.

2) Consider the risks you’ve identified and decide which can potentially be

avoided or eliminated, and which will require mitigation strategies.

3) More specifically, what particular avoidance and mitigation strategies

would you suggest for the various risks you’ve identified?

Team Exercise #2

11/29/2012

17

33 33

ERD Activity: Seminar Registration System

1. Seminar registration is now handled by mail or by phone, based on seminar

brochures sent out in the mail. The customer wishes to implement an online (web-

based) enrollment system.

2. A potential seminar enrollee should be able to go the new web site, select a specific

seminar and then pay for and enroll in it if space is available.

3. Payment would be made by an online credit card transaction.

4. Currently all seminar information (title, instructor, location, date, time, current

enrollment, and maximum enrollment allowed) is held in a Excel spreadsheet.

5. A separate Excel spreadsheet contains some information (name, address, phone,

email, credit card info) about previous seminar attendees.

6. The seminar manager requested a new daily report showing the current status of

enrollment for all seminars.

Recall our work with the proposed new online system to automate seminar registration for a company that offers seminars at multiple sites and on multiple dates. Here are some features of the proposed system that were gathered at an initial one-hour meeting with the customer.

Proposed System

Seminar Attendee

registration_req_w_cc

registration_req_w_corp_pmt

cancellation

catalog_search_req

request_for_transcript

confirmation_of_seat

confirmation_of_cc_pmt

transcript

certificate

confirmation_of_cancellation

registration_decline

seminar_cancelled

Email Sys

Class Admin

Seminar Mgr

seminar_cancellation

Corporate Financial System

payment_info

credit

action

Seminar Info*

* Customer will convert current data

Attendee Info*

search_results

Context DFD