Upload
duonghanh
View
214
Download
1
Embed Size (px)
Citation preview
30th April 2009 Univ. Rey Juan Carlos 1
Design Science as
Nested Problem Solving
Roel Wieringa
University of twente
The Netherlands
30th April 2009 Univ. Rey Juan Carlos 2
How to do design science research?
• Guidelines for doing a Master’s project
• and for doing a PhD project
• State of the practice of research:
– Master’s and PhD students tend to confuse
design with research
– Methodological mistakes
30th April 2009 Univ. Rey Juan Carlos 3
1. Knowledge problems versus practical problems
2. Engineering cycle
3. Problem nesting
4. Design science
30th April 2009 Univ. Rey Juan Carlos 4
Problems
• Difference between what stakeholder wants and what stakeholder experiences
• What is the problem (stakeholder, experience, desire)?– Build a communication infrastructure between
hospital A and insurance company B• Problem: there exists no such structure, stakeholder wants
one
– Improve communication infrastructure between …• Problem: Current infrastructure does not satisfy stakeholder
– Design a communication infrastructure between …• Problem: there is no blueprint, stakeholder wants a blueprint
30th April 2009 Univ. Rey Juan Carlos 5
Practical problems versus
knowledge problems
• Practical problem– Difference between current state of the world
and what a stakeholder would like it to be
– To solve it we need to change the world
• Knowledge problem– Difference between what current stakeholder
knows and what the stakeholder wants to know
– To solve it stakeholder needs to change theirknowledge of the world
30th April 2009 Univ. Rey Juan Carlos 6
What kind of problem?
• What is the architecture of the communicationinfrastructure between A and B?– K Problem: infrastructure exists, stakeholder does not know what
its architecture is
• Build a communication infrastructure between hospital A and insurance company B– P Problem: there exists no such structure, stakeholder wants
one
• Improve communication infrastructure between …– P Problem: Current infrastructure must be changed
• Design a communication infrastructure between …– P Problem: A blueprint must be made
• What is a communication infrastructure between …– P Problem: A blueprint must be made
Misleading
30th April 2009 Univ. Rey Juan Carlos 7
Design problems are practical problems
• Customer: “Design a communication infrastructure formeˮ
• An architect designs one
• Architect: “Here is an architecture!ˮ
• Customer: “Thank you! Now I know what to do! I’ll do itthis way.ˮ
This means:•“ Now I have made up my mind what to do. I’ll
build the infrastructure this way. ˮ
•To design something is to decide what to do
• = Practical problem
30th April 2009 Univ. Rey Juan Carlos 8
Heuristics• Practical problems
– Are solved by
changing the state of the world
– Solution criterion is utility
• Problem-dependent: stakeholders and goals
• Several solutions; but trade-offs
• Knowledge questions
– Are solved by changing
the knowledge of stakeholders.
– Solution criterion is truth
• Problem-independent: no stakeholders
• One solution; but approximations
Doing
Changing the world
Future-oriented
Thinking
Changing our mind
Past-oriented
30th April 2009 Univ. Rey Juan Carlos 9
But …
• Solving a practical problem may produce
knowledge
– That is not what makes it a practical problem!
– Solving a practical problemmay also fail to teach usanything
– We solved a practical problem = we satisfiedstakeholder goals
• Knowledge may be usefultoo– Useless knowledge is still
knowledge– A useful false proposition is
not knowledge• “There is no speed limitˮ
– We solved the knowledgequestion = we have a trueproposition that answersthe question.
30th April 2009 Univ. Rey Juan Carlos 10
1. Knowledge problems versus practical problems
2. Engineering cycle
3. Problem nesting
4. Design science
30th April 2009 Univ. Rey Juan Carlos 11
The engineering cycle
• Solving practical problems rationally
– problem investigation
– solution design
– specification validation
– implementation
– implementation evaluation
•Stakeholders•Goals, criteria•Phenomena, diagnosis
•Specify a possible solution
•Will it work?•Will it satisfy criteria?•Trade-offs?•Sensitivity?
•Does it work?•Does it satisfy criteria?
30th April 2009 Univ. Rey Juan Carlos 12
Design
• To design is to make a decision about a solution– Software design
– House design
– Business design
– Job role design
– …
– Functional design
• A specification is a documentation of those decisions– Software architecture
– House blueprint
– Job role descriptions
– …
– Functional specification
30th April 2009 Univ. Rey Juan Carlos 13
1. Knowledge problems versus practical problems
2. Engineering cycle
3. Problem nesting
4. Design science
30th April 2009 Univ. Rey Juan Carlos 14
Subproblems
• Engineering cycle
– problem investigation
– solution design
– specification validation
– implementation
– implementation evaluation
•K Stakeholders?•K Goals, criteria?•K Phenomena, diagnosis?
•P Specify a possible solution
•K Will it work?•K Will it satisfy criteria?•K Trade-offs?•K Sensitivity?
•K Does it work?•K Does it satisfy criteria?
P
30th April 2009 Univ. Rey Juan Carlos 15
How can we improve financial evaluation of process-aware information systems?
K Current problems
with evaluation?
K Current approaches to
financial evaluation?
St, Ph, Go, Cr
K Build taxonomy
of approaches
K Classify approaches
K Validate classification
K Criteria for taxonomies?
K Collect taxonomies
K Evaluate
P Design new one
KValidate against criteria
K Evaluate them
P Develop new approach:
Causal loop modelsK Make causal loop models
of cost factors of PAISK Collect modeling
guidelines
P Acquire modeling
toolsK Validate it K Check design
argument P Experiment to test one model
P Pilot study using another model
K Reflection: lessons learned
Problemdecomposition
Problemsequence
30th April 2009 Univ. Rey Juan Carlos 16
Bulleted list form• Improving the financial evaluation of PAIS• Problem investigation
– Current problems with financial evaluation• Current approaches
– Taxonomies of approaches
– Our taxonomy
• Evaluation of approaches
• Solution approach– Causal loop models– CLDs of cost factors
• Validation– The engineering argument– Experiment– Pilot study
• Discussion and lessons learned
• Appendices– Modeling guidelines for CL modeling– Tools for CL modeling
Very good PhD thesis outline
30th April 2009 Univ. Rey Juan Carlos 17
P How to solve conflicts between and agencies standards
for data exchange between HRM depts?
K Conflict taxonomy?
K Conflicts between X and Y? K Make CM of subject domain of X and Y
K Problems caused by this? K Stakeholders, phenomena, goals, criteria
K What causes the conflicts? K Describe the causes
P Specify solutions K Inventarize existing ones
P Compose new ones
K Validate the solutions
•K Internal: C & S → criteria?
•properties of S
•Assumptions about C
•Interaction
•K External: Sensitivity?
•K Trade-offsK Reflection:
Lessons learned
30th April 2009 Univ. Rey Juan Carlos 18
Bulleted list formHow to solve conflicts between standards X and Y?• Kinds of conflicts
– A conflict taxonomy
– Conflicts between X and Y
• Problem analysis– Stakeholders, goals, criteria, problems
– Causes of conflicts
– Impact of conflicts
• Solution proposal– Inventory of existing solutions
– Solution for standards X and Y
• Solution validation– Effectiveness: properties, certainty, value of solution
– Trade-offs
– Sensitivity: Future scenarios and behavior of solution in those scenarios
– Stakeholder analysis
• Summary and recommendations• Lessons learned: reflection on the project itself
Would have beenan excellentMaster’s thesis outline
30th April 2009 Univ. Rey Juan Carlos 19
• Find a modelling method for embedded software
– What is the problem?
– Design an improved method
– Validate the method
• What happens if used?
• Does this satisy goals?
• Trade-offs?
• Sensitivity?
• Reflect and identify lessons learned about finding animproved modeling method
•What is the modeling problem in this case?
•Customize my method to this problem
•Validate
•Use it
•Evaluate it
�stakeholder’s goals
�researcher’s goals
Technical action research
•Find a stakeholder with a problem that
you think you can solve using your
technique
•Use your technique
•Repeat for other stakeholder:
•trade-offs and sensitivity
PhD project
Problems
1. Modelling not repeatable
2. Formal models not validated
3. Implicit assumptions about plant
Solution specification
1. Design argument Plant & SW → Goals
2. List of assumptions about plant
30th April 2009 Univ. Rey Juan Carlos 20
1. Knowledge problems versus practical problems
2. Engineering cycle
3. Problem nesting
4. Design science issues
30th April 2009 Univ. Rey Juan Carlos 21
Problem investigation• Stakeholders
– Stand to lose or gain by solving the problem
• Goals
– What stakeholders want
– Need to be analyzed and agreed on:
• Usually inconsistent
• Usually not feasible
• Usually unmeasurable
• Criteria
– Operationalized goals
• Phenomena
– What is happening?
– Diagnosis: what is the cause of the problems?
– Prediction: What would happen if we do not solve the problem?
30th April 2009 Univ. Rey Juan Carlos 22
Reasons for problem investigation
• Problem driven• Stakeholders currently experience problem
• Goal-driven• Goals have changed
• Solution-driven• New technology available
• New technology can be developed
• Impact driven• Evaluate past implementation
30th April 2009 Univ. Rey Juan Carlos 23
Design validation
• Internal validity– Solution & Context produce Effects
– Effects satisfy criteria to some extent
• Trade-off analysis– Effect of slightly different solutions
– Score differently on same criteria
• Sensitivity analysis– Effects in different contexts?
– This is external validity
The design argument
30th April 2009 Univ. Rey Juan Carlos 24
Levels of theory
• Nomothetic– N=∞– Universal statement, for all times and places
– E.g. Theory of gravity
• Practice-oriented– N=k– E.g. Statement about trade-offs in ERP
implementation
• Case theory– N=1– E.g. Claim about trade-offs in this company
Design theories are usually of this kind
30th April 2009 Univ. Rey Juan Carlos 26
Design theories
• Must be practice-oriented– Effort estimation models for IS implementation
– Claim about effectiveness of a modeling method• E.g. It helps mechanical engineers communicate with
software engineers
– Claim about HRM data exchange standard• E.g. it is compatible with existing standards
• General form– Context & Solution → Effects
Possible goal variables
Something stakeholders can do
Some class of relevant contexts
30th April 2009 Univ. Rey Juan Carlos 27
Conditions of practice
• Laboratory versus field
– Engineers must work under conditions of
practice
– Scientists can abstract
• Design theories in practice:– Under not fully known conditions of practice,
– Context & Solution → Effects
30th April 2009 Univ. Rey Juan Carlos 28
Kinds of theory
• What kind of knowledge does a theory give you
– conceptual
• taxonomies
• ontologies
– empirical
• explanatory
• predictive
– design.
– analytical
• mathematical
• logical
30th April 2009 Univ. Rey Juan Carlos 29
Research methods
• Everything is allowed
– lab experiments
– field experiments
– surveys
– case studies
– technical action research
– …
• Do not claim more than you can justify
30th April 2009 Univ. Rey Juan Carlos 30
Take-home messages
• Distinguish practical problems fromknowledge questions
– Changing the world versus changing our
knowledge
• Solve practical problems with the engineering cycle
• Problems are mutually nested
30th April 2009 Univ. Rey Juan Carlos 31
Knowledge problems
Practical problems
Every knowledge problem
hides a practical problem
Every practical problem
hides a knowledge problem