Evaluating Effectiveness of Risk Identification and Management Using Organisational Models

Moshiur Bhuiyan 1. Decision Systems Laboratory, School of Computer

Science and Software Engineering University of Wollongong, NSW 2522, Australia

2. OptInfra Systems, Sydney, Australia

Sohel Rana OptInfra Systems, Sydney, Australia

Sohel.rana@optinfra.com

moshiurb@optinfra.com

Aneesh Krishna Department of Computing, Faculty of Science & Engineering,

Curtin University of Technology, WA 6102, Perth, Australia a.krishna@curtin.edu.au

Abstract - In this paper firstly we discuss an approach for supporting risk identification with the use of higher-level organizational models. We provide some intuitive metrics for extracting measures of actor criticality and vulnerability from organizational models. This helps direct risk management to areas of critical importance within organization models. Secondly, we provide details of an experiment that we conducted to evaluate our methodology. We believe our novel approach, will provide added benefits when used with other approaches to risk management during business process management, that do not reference the greater organizational context during risk assessment.

Keywords- Business Process Modeling, Agent Oriented Conceptual Modeling, Risk management, Organisational Modeling

I. INTRODUCTION Risk management techniques have been extensively

studied and applied within software process management, requirements engineering and project management disciplines [12] [13] [14] [15]. More recently, risk management has been applied to the business process management and modeling domain that as a whole, aims to bridge the gap between organizational and I.T. level conceptual / management concerns [10] [11]. These approaches provide a more direct association between organizational risks at an activity level.

We provide an approach to support risk management by supporting the identification of risk factors (in terms of vulnerability and criticality) at organizational level prior to their propagation and reflection at a process level [16]. We believe that such approach will provide a higher-level scope for risk that may span numerous processes within an organization. Business process risk analysis should be based on higher-level organizational models. A high-level

approach to iterative risk assessment throughout the business process lifecycle might help to identify and manage risk at an organizational level prior to their delegation to actual business processes. We provide an enhanced capability to relate risk at an organizational level by looking at the strategic relationships between functional units and process participants. We define risk at organizational model level on the basis of vulnerability and criticality. For organizational models we use the agent-oriented organizational modeling notation – i* [9] that describes the organizational relationships among various actors and their rationales. For business process model representation we use a standardized, operational and executable process modeling notation – BPMN [8]. The authors consider that the majority of risks identified lie in mismatch with the methods employed within the various phases of the process lifecycle, a lack of clarity who is responsible for the individual phases or their results and a mismatch of process design, automation and evaluation objects. We believe that risk can be better viewed by using a combined notation proposed in [5]. To evaluate our methodology we illustrate details of an experiment that we conducted with participants with process modeling skills.

The following section starts with a discussion of risk and risk management and our chosen notations. We then describe our approach to identify risk factors including our proposed measurement for vulnerability and criticality of actors at organizational level. Finally we provide details of an experiment we conducted to evaluate this methodology.

II. BACKGROUND

A. Agent Oriented Conceptual Modeling Agent-Oriented Conceptual Modeling (AOCM) notations

such as the i* framework [4] [9] (see: figure 1) have gained considerable currency in the recent past. The central concept in i* is that of intentional actor. These can be seen in the

___________________________________ 978-1-61284-840-2/11/$26.00 ©2011 IEEE

below Emergency Service Provider model as nodes representing the intentional/social relationships between six (6) actors required to schedule a meeting: an Emergency Coordination Center Coordinator (ECCC); Field Control Center Coordinator (FCCC); Volunteer/Emergency Workers; Community; Weather Bureau and Call taking supervisor/system.

Figure 1: Strategic Dependency Model of an Emergency Service Provider.

The i* framework consists of two modeling components: Strategic Dependency (SD) model and Strategic Rationale (SR) model [9]. The SD model consists of a set of nodes and links. Each node represents an actor, and each link between the two actors indicates that one actor depends on the other (i.e. goals, task, resource, and softgoal) so that the former may attain some goal. The depending actor is known as depender, while the actor depended upon is known as the dependee. The object around which the dependency relationship centers is called the dependum. The SR mode further represents internal motivations and capabilities (i.e. processes or routines) accessible to specific actors that ensure dependencies can be met.

B. Business Process Modeling with BPMN

Many existing BPM notations primarily focus on technical process aspects including the flow of activity execution/information and/or resource usage/consumption [9]. However, they lack in social and intentional components representation. The technical focus of these notations is especially suited for applications in the description, execution and simulation of business processes but is lacking in support for process redesign and improvement [9]. One such notation is the Business Process Modeling Notation (BPMN). BPMN can be seen as primarily a technically-oriented notation that is augmented with an ability to assign activity execution control to entities (e.g. roles) within an organization with ‘swim-lanes’. Since its initial publication [8], BPMN has been accepted by the greater Business Process Management community [1] [7] due to its expressiveness and ability to map directly to executable process languages including XPDL [3] and

BPEL [6].

Figuf Figure 2: BPMN model of the Emergency Service Provider

III. IDENTIFYING RISKS WITHIN ORGANISATIONAL MODELS

In this section we will describe our intuitive approach to analysis and design with regards to organizational risk. In order to achieve this task, we propose an analysis of strategic dependencies between actors in order to measure and identify each actor’s vulnerability and criticality. Once determined, the design task will be focused towards the area of process modeling that requires most attention. Further details about our methodology can be found at [16].

A . Vulnerability

The vulnerability of an actor plays a vital role for identifying and measuring risk. The i* model provides an intentional description of a process in terms of a network of dependency relationships among actors [9]. We believe i* model provides a better basis for an analyst to explore the broader risk implications of alternative organizational structure. It can help analyze opportunities and vulnerabilities and recognize patterns of relationship. A depender actor’s intention is to have the dependency goal achieved, task performed, or resource available. Failure to obtain the dependum can affect the process by making it more vulnerable and hence increasing the likelihood of risk occurrence. In our work we propose a way of measuring vulnerability of actors at organizational model. The analyst can then take necessary steps to mitigate these vulnerabilities in process models. A stronger degree of vulnerability implies that a stronger initiative to mitigate vulnerability is necessary. Such initiative can be taken by increasing the monitoring process of dependee actor’s activities.

We propose a metric for actor vulnerability. This metric effectively divides the number of outgoing dependencies by the number of dependee actors. A depender actor with more outgoing dependencies implies a greater degree of vulnerability. We consider outgoing dependencies for vulnerability measurement as we believe that outgoing dependencies indicate delegation of tasks and activities. It makes a depender actor vulnerable as the failure of dependee actor to satisfy the dependency, the corresponding task/goal might not be satisfied (a considerable risk). We believe if an actor is vulnerable, an increase in the overall likelihood of risk occurrence is apparent. Intuitively, if the likelihood increases risk will increase as well. The formula we use to assess the vulnerability measurement (VM) of actors at organizational level is as follows:

VMorg =No of Outgoing Dependencies / No of Dependee Actors

In a softgoal dependency, a depender depends on the dependee to perform certain goals or task that would enhance the performance. The notion of a softgoal derives from the Non-Functional Requirements (NFR) framework [2]. They are non-functional requirements of the system, which have positive or negative contribution toward achieving a goal, task, or resource. While measuring the vulnerability of actors we do not include the softgoal dependencies. We believe these non-functional requirements of the system have minimal impact on risk either in the organizational level or on the process level.

We believe actors with no outgoing dependency with other actors have minimal vulnerability as it can not affect the likelihood of occurrence in a greater extent. From the figure-1 we find that the actors WeatherBureau and CallTakingSuperviosr/System do not have any outgoing dependencies. However, actor with no vulnerability does not necessarily mean that it is not critical enough to affect the consequences if it fails. In this case criticality of the actor is considered to measure the risk. Now we need to refine the vulnerability calculation by relating it at process level. The formula we use to calculate vulnerability measurement (VM) at process level is as follows:

VMbp = Organizational Level Vulnerability (VMorg) * Number of Incoming Flows (control flow and message flow)

A. Criticality

Criticality is the consequence factor that is measured from the impact of an actor’s performance where the actor is assigned to satisfy responsibilities/incoming dependencies. The more critical an actor is, the more ability it carries to impact other actors and the organizational context. Incoming dependencies towards an actor are taken into consideration to measure the criticality of an actor. The incoming dependencies describe responsibilities are

assigned to an actor from other actor. By receiving dependencies from other actor makes the dependency receiving actor crucial. If it fails to satisfy the incoming dependencies the depender actors are widely affected which possibly affect the context as a whole. In order to mitigate the risks associated with the system the criticality measurement of actors should be taken into consideration. Measuring critical factors of actors helps the analysts to analyze and construct alternative options to achieve the aim of the system. This will alleviate the risk management and increase the robustness of the system.

Criticality of actors at Organizational Model is measured by multiplying number of incoming dependencies and number of depender actors. The formula we use to assess the criticality measurement (CM) of actors is as follows:

CMorg = No of Incoming Dependencies * No of Depender Actors

We have not considered the softgoal dependencies while calculating the criticality of the actors for the same reasons of vulnerability measurement. If an actor does not have any incoming dependencies from another actor of the model then it portrays that the actor has distributed his dependencies to other actor but no other actor has delegated any tasks, resources and goals into this actor. So the actor will have minimal impact on the consequences of the performance of other actors in the strategic context of the model. For this reason an actor with no incoming dependencies will be positioned with minimal criticality fact towards it but the vulnerability factor of that actor will take it into the consideration of the risk measurement in the strategic framework. Now we need to refine the criticality calculation by relating it at process level. The formula we use to calculate criticality measurement (CMbp) at process level is as follows:

CMbp = Organizational Level Criticality (CMorg) * No of Outgoing Flows

IV. EXPERIMENT The experiment part of the paper is used to validate the

measurements of quantifying risk in terms of analysing and measuring distances in a strategic dependency model for efficient risk management. The evaluation approach taken in our experiment is twofold. One part of this experiment is based on a case study where the participants are asked to identify the critical and vulnerable actors from an organizational model – i* by using our methodology. The other part of this experiment is based on questionnaires. In our methodology we addressed the research question on how risk can be easily identified and managed at business process level using rich organizational model. The above part of the paper explained methodologies to identify risk at organizational model and then addressing in business process models. We divide this experiment into two parts. The first part (Part A) of the experiment provides the users a sample

SD model to identifying critical and vulnerable actors to measure risk and then applying these measures in process model modifications. The second part (Part B) involves some questionnaires aim towards feedback from the users about the constrained development methodology. The i* model in figure 4 is used in Part A for identifying risk within organizational model. The objective of this model is to receive packages from Customers and deliver them to the recipient Customers. Customer drops the package to Customer Representative and then the Customer Representative includes the package details to the Package Tracing System.

Figure 3: Receive and deliver packages from/to Customers

Sort Facility then receives and stores the package to deliver it to the recipient Customer through Courier. Courier delivers the package to recipient Customer directly or delivers it to a Partner Organization’s Service Drop Box at Regular intervals and the recipient Customer accepts the package from the Partner Organization.

This experiment was conducted with twenty three process modeling practitioners from three Australian public sector organizations, two private IT consulting companies and three universities. Eleven participants from public sector organizations, eight participants from private consulting companies and four participants were from the University of Wollongong, University of Technology Sydney and Curtin University of Australia. All the participants were from same educational background and have different focuses towards their careers. Some participants had more experience in modeling technical designs of IT system and some had more on functional business process designs. Among these twenty three participants, nine had more than three years of industry experiences on process modeling, six participants had less than three years of industry experiences, four participants had less than two years of industry experience, two participants had less than one year and two participants had no industry experience on BPMN.

V. RESULTS AND DISCUSSION

About 95% of the participants calculated the measurement correctly and found the right critical and vulnerable actors. About 87% of participants found the idea of the methodology promising. They believed calculating risk of an organization should address its organizational model as

well as the process/operational model. Another 66% of the participants believed it is important to make sure the organizational models and process models exist in the organization for better visibility and smooth operations. In order to get the risk measurements right they insisted of having both types of models must be maintained regularly so that they reflects the true picture of an organization. Most of the participants also found it quite easy to apply the measurement. They believed it is a nice and quick way to find organizational actors that are critical and vulnerable.

We set quality rating for the methodology. The rating had a scale of five 1 being Failed to Meet My Expectation to 5 being Exceeded my expectation. Ratings from the participants are illustrated below:

Figure 4: Methodology rating by participants

By analysing the responses from the participants we compiled the following advantages they found from the methodology:

A. Enhance Risk Management Process

About 83% of the participants argued that a richer modeling concept such as i* provides a better basis to explore the broader risk implications of alternative organizational structure. It does it by analyzing dependencies among actors, measuring vulnerabilities, criticalities and recognizing patterns of relationships. Hence an analyst can take necessary steps to mitigate these vulnerabilities in process models. The participants found measuring critical factors of actors helps the analysts to analyze and construct alternative options to achieve the aim of the system which in turn will alleviate the risk management and increase the robustness of the system. Thus measuring and treating vulnerabilities and criticalities of an actor indeed helps to enhance risk management process of a system.

B. Dealing with Exceptional Situations

Arising exceptional situations are common occurrences during business process execution. That is why addressing process behavior in exceptional circumstances is a critical element for understanding and analyzing a business process. A business process model that does not carefully address exceptional behavior of actors and suggests methods to handle them effectively is incomplete and inadequate. Most of the participants of this experiment believed this methodology provides a better way to deal with exceptional situation in the business process. It does it by giving

sufficient details to delegate tasks to actors, allocating required resources to tasks, collecting and transferring results of performing tasks to other tasks requiring them.

C. Time Savings

About 91% of the participants believe this methodology provides a quicker way for the analyst to identify risk in process models. Hence, it saves analysts an enormous amount of time to design a business process model that is lot less susceptible to risk.

D. Easy & Practical

Participants with organizational and process modeling knowledge and skills found this methodology easy and practical to use. As we have used i* and BPMN for this methodology, it would not take huge amount of effort to the users to get use to the notations.

Some participants pointed out some negative sides of this methodology as well. One of the concern was the methodology does not consider the SR model (which means it did not consider the strategic rationale of an organizations). We plan to extend our work in future so that it supports SR models. About 63% of the participants also noted, to be able to apply this methodology the organization must have Organizational Model in i* and Business Process Models in BPMN. Which means this methodology is dependent on certain notations. It is not a generic one. Organizations that do not have Organizational and corresponding BPMN models can not apply this methodology which is somehow inconvenient. They also mentioned there must be personnel in the organization, who has knowledge and skills about organizational and process modeling. Otherwise there is a possibility of misunderstanding the methodology. Some participants also mentioned the importance of applying this methodology at an industrial level. They argued it will take a while before this methodology gets to a standard before organizations find it comfortable to use.

VI. CONCLUSIONS

In this research work we have evaluated a model based approach to measuring risk. We have discussed on how we can identify risk in terms of vulnerability and criticality in organizational models. We then presented the findings of an experiment on this methodology to validate the measurements by questionnaires and analysing the results. We believe our proposed approach helps the analyst while designing organizational models, delegate dependencies among various actors, choose alternatives, decompose tasks, maintain consistency among organizational and process models, handle exceptions etc.

REFERENCES [1] J. Becker, M. Indulska, M. Rosemann, P, Green, “Do Process

Modeling Techniques Get Better? A Comparative Ontological

Analysis of BPMN,” in Campbell, Bruce and Underwood, Jim and Bunker, Deborah, Eds. Proceedings 16th Australasian Conference on Information Systems, Sydney, Australia, 2005.

[2] L. Chung, “Representing and Using Non - Func-tional Requirements for Information System Development. A Process-Oriented Approach,” PhD Thesis, Graduate De-partment of Computer Science, Toronto, University of Toronto, 1993.

[3] C. Hall, P. Harmon, The 2005 Enterprise Archi-tecture, Process Modeling & Simulation Tools Report, Technical Report, bptends.com, 2005.

[4] G. Katzenstein, J. Lerch, “Beneath the surface of organizational processes: a social representation framework for business process redesign,” ACM Transactions on Information Systems (TOIS), 18(4), pp. 383-422, 2000.

[5] K. George, A. Vranesevic, M. Bhuiyan, A. Krishna and A. Ghose, “A combined approach for sup-porting the business process model lifecycle,” Proceedings of the Asia-PacificConference on Information System, 2006.

[6] C. Ouyang, W. van der Aalst, M. Dumas, and A. ter Hofstede, Translating BPMN to BPEL, BPM Center Report BPM-06-02, BPMcenter.org, 2006.

[7] H. Smith, P. Fingar, “Business Process Management – The Third Wave,” Tampa, FL: Meghan-Kiffer Press, 2003.

[8] S. White, “Business Process Modeling Notation (BPMN),” Version 1.0, Business Process Management Initiative (BPMI.org), 2004.

[9] E. Yu, “Modeling Strategic Relationships for Process Reengineering,” PhD Thesis, Graduate Department of Computer Science, University of Toronto, Toronto, Canada, pp. 124, 1995,

[10] Muehlen zur, Michael and Ho, Danny Ting-Yi (2005a). Risk Management in the BPM Lifecycle. In: Bussler, Christoph; Haller, Armin (Eds.): Business Process Management Workshops: BPM 2005 International Workshops, BPI, BPD, ENEI, BPRM, WSCOBPM, BPS, Nancy, France, September 5, 2005. Revised Selected Papers, Springer LNCS 3812, Berlin 2006, pp. 454-466.

[11] M. Michael, R. Michael, “Integrating Risks in Business Process Models” In: Proceedings of Australasian Conference on Information Systems (ACIS), Manly, Sydney, Australia, 2005.

[12] C. Silveira, “A Knowledge-Based Risk Management for the Utility Business Service Model,” Informing science and Information Technology Education Joint Conference, Pori, Finland, 2003.

[13] C. Nogueira, Luqi, S. Bhattacharya, “A Risk Assessment Model for Software Prototyping Projects,” Proc International Workshop on Rapid System Prototyping, pp. 28-33.

[14] M. Schmitt, B. Grégoire, E. Dubois, “A risk based guide to business process design in inter-organizational business collaboration,” International Workshop on Requirements Engineering for Business Need and IT Alignment (REBNITA), Paris, 2005.

[15] M. Sumner, “Risk Factor in Enterprise-wide/ERP projects,” Journal of Information Technology, 15, pp. 317-327, 2000.

[16] Z. Islam, M. Bhuiyan, A. Krishna, A. Ghose, “An Integrated Approach to Managing Business Process Risk Using Rich Organisational Model,” 17th International Conference on Cooperative Information systems (COOPIS), 2009.