QUALITY OF DECISION MODELS

JAN VANTHIENEN

Decision Camp 2

Jan Vanthienen

KU Leuven, Information Systems Research Group

Research and teaching:

Business rules, processes and information systems Decision models & tables Business intelligence & Analytics Information & Knowledge Management

• IBM Faculty Award• Belgian Francqui Chair 2009 at FUNDP

Cooperation with industry:

- ING bank Research Chair on Metadata Analytics- Bpost bank Research Chair Actionable Analytics- Colruyt-Symeta Research Chair Smart Data and Decisions in Marketing- IBM Fund Intelligent Business Decision Making- Microsoft Research Chair on Intelligent Environments- PricewaterhouseCoopers Chair on E-Business

jan.vanthienen@kuleuven.be

Europe's most innovative university in Reuters Europe's top 100 innovative universities ranking.

Decision Camp 3

Looking at Data QualityWhat can we learn?

The Six Primary Dimensions For Data Quality Assessment,

Defining Data Quality Dimensions, Dama UK Working

Group On “Data Quality Dimensions”, Oct. 2013.

Decision Camp 4

Data quality dimensions

Wang, R.; Strong, D. (1996). "Beyond Accuracy: What Data Quality

Means to Data Consumers". Journal of Management Information

Systems. 12 (4): 5–34. doi:10.1080/07421222.1996.11518099.

Decision Camp 5

Decision Model QualityIn a nutshell

Dimensions of Information Quality

Intrinsic IQ: accuracy, objectivity, believability, reputation Contextual IQ: relevance, value-added, timeliness, completeness, amount of information Representational IQ: interpretability, format, coherence, compatibility Accessibility IQ: accessibility, access security

About decision modeling

Intrinsic: accuracy, objectivity, correctness, complexity, adaptability, believability, reputation Contextual: relevance, value-added, timeliness, completeness, fitness, integration Representational: interpretability, format, compactness, consistency, coherence, compatibility Accessibility: accessibility, traceability, access security

Wang, R.; Strong, D. (1996). "Beyond Accuracy: What Data Quality Means to Data Consumers". Journal of

Management Information Systems. 12 (4): 5–34. doi:10.1080/07421222.1996.11518099.

Decision Camp 6

Intrinsic: Decision model complexityDRD

Faruk Hasic, Jan Vanthienen, Complexity metrics for DMN decision models.

Computer Standards & Interfaces 65: 15-37 (2019)

Decision Camp 7

Intrinsic: Decision model complexityDecision table

Faruk Hasic, Jan Vanthienen, Complexity metrics for DMN decision models.

Computer Standards & Interfaces 65: 15-37 (2019)

Decision Camp 8

Representational: Consistency

Testing?

Verification

Experience?

Verification tools?

Smart modelers?

Lessons from decision table methodology and experience:

The best way to obtain correct models ...

is to make it impossible/hard to build incorrect models!

Consistency by design

obtained by avoiding redundancy in the table

redundancy is not inconsistent, but often leads to inconsistency later (like in databases)

How to obtain consistent decision models?

Decision Camp 9

What are overlapping rules: visual interpretation

Ph

on

e

Web

US Non US

CustomerType

Ord

erTy

pe

Rule 18%

Rule 30%

Rule 25%

If the input says: CustomerType = US and OrderType = Phone?

Rule 1 matches, and rule 2 matches.

Decision Camp 10

What if rules overlap?

If the input says: CustomerType = US and OrderType = Phone?

In DMN you can indicate the meaning using the hit indicator:

First: Take the first rule that applies (from top to bottom) and stop (8%)

Unique: Make sure the rules do not overlap

Any: Allow rules to overlap, but only if the output is the same (redundancy)

Priority: Take the output with the highest priority, listed in decreasing order in the value list (5%)

Collect (and add) the output of every rule that applies (13%)

Decision Camp 11

Representational: Interpretability and consistency

The DMN position

Different companies and tools may use different types of tables a long as the meaning is clear. DMN is about notation

Good decision table methodology

It would be better (in most cases) if rules do not overlap. Other table types are intermediate.

If the input says: CustomerType = US and OrderType = Phone?

Rule 1 matches, and rule 2 matches. Now what? Contradiction?

Decision Camp 12

Table with no overlapping combinations, in natural order

Example: unique-hit table and well ordered

• There are no overlaps, no contradictions (consistent by definition)

• The logic is independent of the order of the rules

• Is it complete? Not so difficult to check, because the table reads like a tree (from left to right)

• Is it correct? Easy to validate

Easy forbusiness

Decision Camp 13

Some of this is methodology

But if you can choose 2 out of 3, what do you choose?

Compact Consistent

Correct

Representational: Compactness

Decision Camp 14

Representational: Interpretability

Applicant Risk Rating

Applicant Age Medical History Applicant Risk Rating

> 60good Medium

bad High

[25..60] - Medium

< 25good Low

bad Medium

Applicant Risk Rating

Applicant Age < 25 [25..60] > 60

Medical History good bad - good bad

Applicant Risk Rating Low Medium Medium Medium High

Applicant Risk Rating

Applicant Age < 25 [25..60] > 60

Medical History good bad - good bad

Low X - - - -

Medium - X X X -

High - - - - X

Rules in columnsRules in rows

Shorthand notation: compact, and easy to see the patterns

Applicant Risk RatingMedical History

good bad

Applicant Age

< 25 Low Medium

[25..60] Medium Medium

> 60 Medium High

Crosstab

Decision Camp 15

When to use rules in rows?

Every information item is a column

Every row is a rule

Applicant Risk Rating

Applicant Age Medical History Applicant Risk Rating

> 60good Medium

bad High

[25..60] - Medium

< 25good Low

bad Medium

Too many columns will be hard to read

Use rules in rows if there are only a few conditions

There can be many rules (rows)

Decision Camp 16

When to use rules in columns?

Every information item is a row

Every column is a rule

Too many columns will be hard to read

Use rules in columns if there are not too many rules, but you want a good overview of the patterns

There can be many information items (rows)

Applicant Risk Rating

Applicant Age < 25 [25..60] > 60

Medical History good bad - good bad

Low X - - - -

Medium - X X X -

High - - - - X

Decision Camp 17

Contextual: Process-decision integration

Decisions as a global concern

Decision Camp 18

Process-decision integrationGuidelines

Overview of Guidelines

Guideline 1 Include all outcomes from the decision in DMN into BPMN

Guideline 2 Avoid embedded decisions in gateways

Guideline 3 Include intermediate decisions in BPMN when intermediate results are needed in the process

Guideline 4 Intermediate decisions leading to extra possible paths in the process model should be included

in BPMN

(e.g. send message, end events, additional tasks)

Guideline 5 Exclude as many intermediate decisions from BPMN as possible

Guideline 6 Model the necessary decision tasks in the BPMN in accordance to their execution in the DM,

i.e. respect the topology of the decision requirements diagram

Guideline 7 Ensure that input requirements are met per individual decision activity

Decision Camp 19

Intrinsic: Correctness and explainability

In automated decisions, the decision logic should be:

well-designed

correct

consistent

traceable

explainable

understandable by the business

easy to change

maintained by the business

Automated systems and processes contain the logic of decisions

Decision Camp 20

Obtaining quality: Decision modeling methodology

When building a decision model, one can start:

By building the Decision Requirements Diagram (DRD) first

• without the decision logic

• and fill the detailed logic later (working one’s way up or down)

By building detailed tables first

• and then connecting them into larger decisions

Usually a combination of both approaches works best

Decision Camp 21

Earlier research

Execution

From tables to optimal code (Codasyl 1982) From tables to minimal rules (1986)

Verification

Rule set consistency (2007) Validation and Verification of decision tables (1998)

Construction methodology

Lifecycle of decision table hit policies (1988) Decision table construction methodologies (1986) From rules to unique decision tables (1982, 1993) Normalization of decision tables (1993)

Decision dependency diagrams

Factoring/Defactoring of decision tables (1996) Verification between decision tables (1998) Decision model dependencies (2012)

Decision mining

Decision table mining (1998, 2003) From logs to decision requirements (2010)

Lots of conversions/transformations/optimizations already exist:

Maes, R., Vanthienen, J., Verhelst, M. [82], "Practical Experiences with The

Procedural Decision Modeling System," Proc. Joint Ifip Wg 8.3/Iiasa

Working Conference on Processes And Tools for Decision Support,

Laxenburg (Austria), July 19-21, 1982, pp. 139-154.

Decision Camp 22

The present

Execution

From decision tables to execution

Verification

Rule set consistency Validation and Verification of decision tables

Construction methodology

Lifecycle of decision table hit policies Decision table construction methodologies From rules to unique decision tables Normalization of decision tables

Decision dependency diagrams

Factoring/Defactoring of decision tables Verification between decision tables Decision model dependencies

Decision mining

Decision table mining From logs to decision requirements

Business process integration

Decision models and business processes (2007) Mixed-paradigm process modeling (2016) Mixed-paradigm mining (2015)

DMN

DMN

DMN

DMN

DMN

DMN

DMN

DMN

DMN

DMN

DMN

Decision Camp 23

DMN features for quality

Separating decisions and processes

Using a standard modeling notation.

Decision table types

Recognize, and unambiguously exchange.

Decision modeling methodology

Keep the insights of the past and avoid confusion.

Separating decision structure and decision logic

Allows to model decision relations, even if not all logic is in tables.

Standard notation for exchange and implementation

Strict notation and simple expression language (FEEL).

Decision Camp 24

Thank you