1

Revealed causal mapping of IT/IS project risk

Robert T. Hughes, University of Brighton, UK

2

Overview of the talk

Locating project risk management in the research world

Metrics versus management information Need for causal models Challenge of quantification Future directions

3

Locating risk management researchHirschheim and Smithson framework

Efficiency zoneHardware/software monitor, Simulation, Code inspection, Software metrics, Quality assurance, TQM

Effectiveness zoneSystem usage, Cost benefit analysis, Critical success factors, Risk analysis, Resource utilisation, Economics, Management,

Understanding zonePersonal constructs, Context-content-process, Political analysis, Organisational behaviour

Objective/rational

Subjective/political

4

Software metrics – in my day… Focus on engineering approach Would like software to be physical product Measurements tied to specific ‘physical’

entities Focus on measurement practice

Often quite negative – demonstrating why measurements invalid

Dearth of positive achievement

5

Need for ‘management information’ Management is concerned with allocation of

resources This allocation needs to be seen to be fair

and just Therefore needs to be linked to objective

indicators of need, merit, productivity etc., etc.

This implies that measurement needs to be based on theories of causation

6

Revealed causal mapping Strongly influenced by Kelly’s concept of

personal constructs – sees human behaviour based on ‘model building’ about cause and effect

Each construct has a positive and negative pole

A network of constructs and the cause and effect relationships between them can be built

Note that the maps describe perceptions – hence often called ‘cognitive’ mapping

7

Fragment of a RCM

experience ofdevelopers remedial work

time pressure

product quality

+

–

–

8

Experienced staff … inexperienced

low staff turn-over… high

High productivity… low

Deadlines met… missed

Heavy management pressure…low

Uncertain user requirements … certain

Unstable environment…unstable

High salaries…low

Requirements prototype…not

High costs…low costs

9

What RCMs can illustrate

‘Tail’ constructs are those that have no prior cause within the scope of the map

Tails could be: Environment e.g. ‘low staff turnover…high’ Policy e.g. ‘requirements prototype…not’

Constructs can be subject to both negative and positive influences – shows uncertainty

Between two constructs there can be both positive and negative relationships

10

Comparison with other mapping approaches – physical models Systems dynamics

SD involves building a mathematical model which attempts to represents the real world system

Question of validation of each relationship identified in SD model

Very labour intensive Root cause analysis

Analysis of the circumstances of a particular situation: actual events rather than situational factors

11

Comparison with other approaches - perceptual Cognitive mapping – people’s perceptions:

But how do you know they are telling the truth? >>> ‘Revealed’ causal mapping

Reasoning maps - the way people make decisions Accuracy of predicting actual decisions? Effectiveness of decisions?

Healthy people’s perceptions should be close to reality?

12

Reasoning maps: problem of indeterminate outcomes

experience ofdevelopers remedial work

time pressure

product quality

+

–

–

Need for some kind of quantification

13

Approaches to ‘intermediate’ quantification Fuzzy cognitive maps

Allow values to be set for tail nodes and edges Can execute the model and study dynamic

behaviour The presentation of FCMs can be off-putting for

non-mathematicians Reasoning maps are a ‘user-friendly’ alternative

14

Using ordinal indicators

Allow us to model quantification without actual measurement

Allocate ordinal values to tail nodes For example

Very strong, strong, medium, weak, very weak Allocate value to the strength of the causal

links Very strong, strong, medium, weak, very weak

15

Propagating values

Methods can vary e.g. Where there is a single cause, use:

minimum (node_value, edge_value)

strongmedium

medium

16

Where there is more than one causal link

Identify minima as before Take the maximum result This implies an independent ‘OR’ relationship

between two causal factors Other relationships possible e.g. compensatory

factors – take the median

strong

medium

weak

strong

medium

17

Our use of RCMs As a teaching and learning tool – can be

used in text analysis Project risk management – used to study

failed projects retrospectively Asked participants to map causes of failure

individually – large differences in perception Follow-up by consensual map-building – group

consensus appeared to be relatively easy Differences in perceptions of managers and

development staff Attempt at building a ‘core model’ of project risk

18

Our use of RCMs As a method of designing service

management information systems Identify the problem domain Stakeholders collaboratively build the model Identify measurements:

That can corroborate model That can act as predictive and summative performance

indicators Examine what effects performance indicators

might have

19

Future work: Generic project risk model

PolicyRisk

Avoidance…commitment

Risksituation…

RiskExposure…

RiskreductionPolicy

Riskoccurrence

Damage..benefit

Contingency preparation

Contingencyaction

Policy

Covers risk tactics of avoidance, reduction and mitigation

▬

✚

✚ ✚

✚

✚

✚ ✚

✚✚

✚✚ ▬

▬

20

Future plans

Developing tools e.g. analogy seeking ‘White fly experiment’ – risk model for student

projects

21

Further details Al-Shehab, A., Hughes R.T. and Winstanley G . (2006). CorMod: A causal mapping

approach to identifying project development risk. European & Mediterranean Conference on Information Systems (EMCIS) 2006, July 6-7, Alicante, Spain.

Hughes R.T., Al-Shehab A., and Winstanley G. (2006). Obstacles to the modelling of the causes of project success and failure. In Dan Remenyi (Ed), Proceedings of the 5th European Conference on Research Methods in Business & Management, Trinity Colledge, Dublin, Ireland, 17-18 July, pp 179-186.

Al-Shehab A.,Hughes R.T. and Winstanley G. (2005). Modelling Risks in IS/IT Projects through Causal and Cognitive Mapping. Electronic Journal of Information Systems Evaluation (EJISE), Vol.8, No.1, pp 1-10, January 2005

Hughes R.T., Al-Shehab A., and Winstanley G. (2005). The use of casual mapping in the design of management information systems. Proceedings of the 4th International Conference on Research Methodology for Business and Management Studies (ECRM-05), held at the University of Paris-Dauphine, France, 21-22 April 2005

Al-Shehab A., Huhghes R.T. and Winstanley G. (2004). Using Causal Mapping Methods to Identify and Analyse Risk in Information System Projects as a Post-Evaluation Process. Proceedings of the 11th European Conference on Information Technology Evaluation (ECITE 2004) held at the Royal Netherlands Academy of Arts and Sciences, Amsterdam, 11-12 November 2004

R.T.Hughes, A. Al Shehab, M.Eastwood. ‘The use of cognitive causal mapping as an aid to professional reflection’. CHI workshop on ‘The Reflective Practitioner’ Vienna, April 2004.

See - http://www.cmis.brighton.ac.uk/research/cig/