Process Modeling Calgary 2004 Prof.Dr.Michael M. Richter Chapter 2 Main Concepts of Process Modeling

Process Modeling Calgary 2004

Prof.Dr.Michael M. Richter

Chapter 2

Main Concepts of Process Modeling



Additional Recommended Literature

• A good overview over MILOS can be found in http://ser..ucalgary.ca/~milos/Library.htm

• B. Dellen, F. Maurer: Change impact analysis support for software development processes, International Journal of Applied Software Technology, ISSN 1198-5577, Vol 4, No 2/3, International Academic Publishing, Scarborough, Ontario, Canada, 1998.

• H.D. Rombach: Practical Benefits of Goal Oriented Measurements. Software Reliebility and Metrics. Elsevier Applied Sycience,1991



PART 1

General Principles



Motivation

• The main tasks for a software project are the creation of a project plan in order to deliver certain software products.

• This can be supported in various ways:– coordination of different activities within a project following a

defined development process

– coordinating planning and enactment (execution)

– providing access to multiple information related to the current project context. This information can be

• distributed, heterogeneous, unstable and hard to find.

• The support is provided by Process-centered Software Engineering Environments (PSEE).



Socio-Technical Processes (1)

• Software development processes are processes where the participating members („agents“) can be humans as well as machines.

• This requires a careful organization of the division of labor:– What do humans?

– What do machines?

• A particular problem is the communication between humans and machines:– Humans have difficulties to understand the results of machines

– Machines have difficulties to understand the results of humans



Socio-Technical Processes (2)

• Conflicting demands:– Machines need precise instructions– Humans want to use creativity.

• Plans and Executions:– They alternate, before all requirements are present and before

planning is finished execution of some actions start.– Requirements may change over time– Requirements are often imprecise and incomplete



Human versus Machine

• Who is better, human or machine?

• This is the wrong question. Better: Who is better, human with machine or human without machine?

• The relation is not symmetric:– The human has the responsibility and makes the decisions

– The machine is a servant and does what it is told (it is an assistant system).

• In extreme cases there are only humans or only machines.

• In relality, there will be a mix. This mix is not fixed. If a task is fully understood it can switch from a human to a machine agent.



Socio-Technical Processes are Concurrent

Concurrent, parallel:

CollectingInformation

Planning Execution

All three arrows represent again complex concurrent activities. Concurrency creates communication problems!



Consequences

• Because of – Incomplete information and knowledge

– Creativity of human agents

– Partially unknown behavior of machines:

• Planning on the level of details is often impossible

• Therefore: Planning is essentially organization

• Questions:– How should that be done?

– What has to organized?



Formal versus Informal Communication

• Traditionally, computer scientist prefer a formal communication: They exchange formal documents.

• This may not always be the best choice because the two partners in the discussion can have different contexts and formal statements may have a different meaning in these contexts. Also, formal statements are often too detailed.

• As a consequence one can use informal expressions which the partner interprets using his/her own intelligence.

• Therefore a support system should allow informal entries.



Semi-Formal Documents

• Semi-formal documents combine formal and informal parts.

• Formal parts can be programs, strict advises (e.g. deadlines) or structures for documentation (e.g. a graph structure). They can be interpreted by machines.

• Informal parts can be texts, annotations etc. They can only be interpreted by humans.

• In process modeling semi-formal documents occur frequently.



Execution (1)

• Planning takes place in a model.

• There is also an external world. In this world– actions are executed; executions can never be retracted, they are

done. In contrast to planning, an execution can fail.

– costs apply (“in reality”)

– additional information may be available.

• The external world is partially mapped into a model world but it exists independent of the model.

• It can be influenced by the user but not completely controlled.



Execution (2)

• Planning steps can withdrawn on the cost of planning steps, execution steps can never be withdrawn. There may be only other steps that reverse some activity and this is sometimes impossible:– If you inform someone the information cannot be withdrawn

– If you delete something etc.

• In any case, some costs will apply.

• Execution needs extra agents with special skills and special knowledge is necessary.



Realizing Flexibility: The Overall View

• Techniques & methods for integrated planning and execution

– Detailed Planning during execution

– Plan correction during execution

• Techniques & methods for dependency derivation and administration

• Controlling dependencies and the flow of activities and artifacts (products)



PART 2

Details



MILOS• The terminology is taken mainly from the MILOS-System

(which is a specific PSEE) but is essentially also in most other systems: The difference to other support systems is mainly notationally (e.g. Protege´). The successor of MILOS in Calgary is MASE.

• MILOS: Minimally Invasive Long Term Organizational Support is a support system for socio-technical processes which realizes most of the concepts and method presented in this chapter.

• It supports knowledge management activities.

• The main additions are– scheduling and release planning– experience support– explanation aspects



Process Modeling Languages

• Systems that support planning and coordination of software development have to represent the entities used in project plans. They allow representing knowledge about software development processes.

• We need to define Process, product and resource models Project plans Product deliverables developed within projects Background knowledge such as guidelines, business

rules, studies etc.



Overview: Basic Concepts (1)

Concept Description

process Description of a set of activities that have to be executed in order to reach a given goal containing e.g. input and output documents etc.

method A method describes how a process goal can beachieved.

complex method

Problem solving method that refines a process into one or more subprocesses.

atomic methodLeaf within the process/method hierarchy. It produces or modifies a product.




product type

Describes a product within a process. It consists of product name and type (e.g. ClassDiagram or URL)

Attribute of a process.

process attribute

productA product is an information unit for example a document or a piece of code.

The description of type and structure of a class of products.

parameter




resource Resources are used for the execution of the project.

Attribute of a product.product attribute

Product references stand for the type of productsthat are used and/or produced by a process or a method. We distinguish between products that are consumed for planning, consumed for execution, produced, and modified.

product reference

This parameter type stands for a product of a given type that is produced by an atomic method of a process.

produceproduct



Overview: Basic Concepts (4)resource attribute

Attribute of a resource.

precondition A condition that has to be valid to carry out a process.

postcondition A condition that has to be valid after a process has been executed.

agent A human or machine that carries out activities within a process.

Process models are generic, reusable descriptions of how to execute projects

process models

A project specific model project plan



Plan-Processes and Methods (1)

• In a project plan there are alternating– Processes

– Methods

• Each process is realized by some method. The method can be atomic or complex. Often one has the choice between different methods and has to select one.

• Below some complex method one or more subprocesses can be created.

• Atomic methods cannot be refined furthermore.

• This gives a hierarchical organization.

• To each process one or more agents can be assigned. The agents are taken from an agent pool („Resources“) that has to be filled before.



Plan-Processes and Methods (2)

• Methods are either complex or atomic. Complex methods refine a process into one or more subprocesses whereas the application of an atomic method results in the production of products that are the outputs of the process.

• Inputs of a process may either be products that are produced by other processes during project enactment or predefined background knowledge (e.g. manuals or guidelines) taken from generic process models.



Example: Project Plan

Requirements Design Process System Design Implementation Code

JavaOO

Defineoperations

Createclass diagramm

Classdiagram



Example in therepresentation languageof Milos: We see a top-down plan of a task. Each plan is realized bya method.Each complex method issubdivided into different subplans.



Partial Tree View of the Same Project Plan



Process Description (1)

• Process: <process name>

• Goal: <process goal description>

• Instructions: <process instructions>

• Input Products: {<product name>’:’<product type>}*

• Output Products: {<product name>’:’<product type>}*

• Modify Products: {<product name>’:’<product type>}*

• Context Products: {<product name>’:’<product type>}*

• Entry/Exit Conditions: {boolean expression}

• Methods: {<method name>}*

Formal definitions:




Process name: It is a process identifier, unique within the scope of a method.

Process goal: Textual description of the intended results of a process execution.

Process instructions: Textual description of special tasks and guidelines how to perform the process.

Product name: It is a product identifier, unique within the scope of a method.

Product type: Each product has an uniquely associated product type. For example, a TestReport can be of type TextProduct. Each possible product type is discussed in more detail in the Product Model description




• Context products: products that are already accessible at modeling time. For example guidelines, handbooks, etc.

• Entry/Exit conditions: determine the control flow between processes. Entry condition should be fulfilled to start the process execution. Exit Input/Output/Modify products: products that can be consumed, created or changed by a process. It is a parameter declaration.

• condition state the condition that should hold after the process has been finished.

• To control the process flow Entry/Exit conditions can reference the state of processes and products.

• Product state: A product attribute that indicates the development grade of a product. For this work we considered four possible states: no-existent, to- Review, toChange, complete. The state transition diagram depicts detailed relationships between these product states.



Table of Method Description

• Method: <method name>

• Goal: <method goal>

• Instructions: <method instructions>• Additional Input/Modify/Output Products:

» {<product name>’:’<product type>}*• Refined Input/Modify/Output Products:

» {<product name>’:’<product sub-type>}*• Additional Entry/Exit Conditions: {boolean expression}• Parameter Mappings:

{<process name>’.’<product name>• ’-->’<process name>’.’<product name>}*• Sub-Processes: {<process name>}*



Process Models

• A project plan describes an individual process.

• A process model describes a set of processes, it allows to define specific processes.

• Therefore: A process model may contain plan processes that have more then one method, e.g.– one may use Java or C++ for an implementation

– one may use the GUI A or the GUI B.

• These methods are seen as alternatives and can be seen as experiences. Therefore they are stored as experiences.

• For a specific plan one method is selected.

• In experience bases it is important to mention under which conditions some alternative is useful.



Process Models and Methods

• Within process models activities and their interrelationship are described.

• A more general description of processes in process models defines a– the process goal,

– a set of conditions,

– process attributes,

– the products needed to plan and to execute the process,

– a set of alternative methods to reach the process goal,

– the products to be produced and resource allocations.



Example: Process Model for Software Development (1)

requirements design process system design

system design implementation code

Processes, Inputs, Outputs




Design process

Objectorienteddesign

Proceduraldesign

Implementation

Use Java

UseCobol

Use C++

Alternative Processes

Atomic methodsComplexmethods




Process refinement

Design process

ObjectOrienteddesign

Defineoperations

Createclass diagram



Example: Process Model for Medical Applications

Evaluation process

Local statisticalevaluation

Nationalevaluation

Methods used

Use X-Ray

UseUltra

Sound

Use MR

Alternative Processes

Atomic methodsComplexmethods



Product Models

• We need the specification of hierarchical, object-oriented product models. A product type defines the structure of a set of products with the same behavior.

• A product type is either basic or complex. • Basic types are predefined and may be integer, real, string,

text, date, or external references such as a www page URLs or word documents. A complex type consists of one or more typed subproducts and attributes. Complex product types can be specialized (IS-A relation).

• Based on a given product model, products instances (synonyms: products, deliverables) that contain general and specific project knowledge of a company can be defined.



Example: DICOM-Standard

• DICOM (Digital Imaging and Communication in Medicine) is a standard covering medical data format for digital medical data.

• It is developed by the American College of Radiology and the National Electric Manufacturs Association.

• The standard deals with the exchange of digital information between medical image equipment.

• Main application areas:– Network image management– Structured reporting– Network print management– Image procedure management– Offline storage media management



Example: Computer-Based Patient Record

• A Computer-Based Patient Record (CPR) is an electronic patient health record that documents and provides access to information about the patient´s general health condition, present and past illnesses and diagnosis, the status and treatment data.

• A DICOM Structured Report is a structured Document which contains documentation of a patient´s examinations, diagnosis and treatment data.

• It may also contain embedded references to other structured reports, images, waveforms, and audio.

• It contains context information, such as scheduled procedures, observer, previous reports and images.

• It encodes only semantic information and not presentation information.



Standard Network Physical Layer (Ethernet, FDDI, ISDN, etc.)

Association Control Service Element (ACSE)OSI Presentation Kernel

OSI Session Kernel

OSI Transport

OSI Network

LLC

DICOM Upper Layer

Protocol for

TCP/IPTCP

IP

DICOM Physical

Conection(50 pins)

DICOM Network

Transport Section

DICOM Data Link

DICOM Application Entity

Point-to-Point

Environment

Network Environment

OSI Upper Layer

Boundary

Example: Medical Imaging Application



Parameter Mappings (1)

If two processes use the same product this has to be documented.This is done by mappings between the corresponding parameters.We distinguish vertical and horizontal mappings. Example:

Parameter 1 Process1

Parameter 2

map 1 on 2

Process 1.1 Process 1.2

Parameter 3 Parameter 4



Parameter Mappings (2)

• Vertical mappings connect two input or two output parameter in a process refinement.

• A horizontal mapping in a refinement map an output parameter of one process to an input parameter of the other process.

• The parameter mappings describe the flow of products in a project.

• An extension of this principle becomes necessary if one wants to describe the product flow between different projects.



Precedence Relations

• This relation says:

One process B must precede another process B

• The major reason for such a relation is: an output parameter of one process A is an input parameter of the other process B.

• This means: The flow of products determines to some degree the order of the enactment of the processes.



Agents (1)

• The participants in an assistant system are also called agents.

• Intelligent machine assistants that use software tools are often called softbots.

• Softbots have knowledge, methods and strategies.

• Agents may act– on demand

– on their own: Pro-active

• Actions of agents must be triggered.

• The behavior of agents may be known to other agents. The behavior of softbots may not be known even to humans (e.g. search machines in the internet).



Agents (2)

• „Agent“ is used as a modeling concept to describe active entities that carry out tasks. In order to make this precise the properties of an agent have to be specified precisely.

• Two sub-concepts of agent are defined:– Actor: A human agent working on a project like a project planner or

developer. An actor interacts with the system via graphical interfaces.– Computer agent: A software entity that can be started on a computer

and performs a task, e.g. a delegation or compilation of a program.

• Both, actors and computer agents have qualifications that determine the tasks they can carry out.

• Both can use tools.• Both are not restricted to a specific location.



Agents (3)

• Agents occur essentially in two ways:

1) A company has agents, organized in a pool.

2) A task requires agents in order to be performed.

• This defines a matching problem.

• Because of restricted available agents this also is an optimization problem.

• In a project plan this match has to be performed. However, the plan does not say how this is done; for this purpose one needs extra support by scheduling tools.



Resource Models

• The resource pool component manages roles, agents and agent properties. It allows representing the organizational structure of a company as well as a skill ontology.

• An agent can have a set of skills. Resource models allow assigning roles and qualifications to project team members.

• These models describe knowledge needed by project managers to find appropriate people for a task. The project planner can query the resource pool for all agents with specific skills.

• Agents can be humans or software agents



Project Planning

• Project planning is essentially an instantiation of process models.

• Planning a project comprises: Selecting appropriate processes (e.g. from some experience

base), and inserting them into the plan, Selecting applicable development methods according to

the characteristics of the organizations (e.g. familiarity with specific methods) and the goals of the project. Instantiating the variables in pre- and post conditions,



Typical Problems in Project Planning

• Are concerned with the specific task

• Selection of processes

• Instantiation of general steps

• Obtaining missing information

• Decisions about details and execution

• Availability of resources

• Sequence and ordering problems

• Time planning, scheduling



Project Plan Management

• The project plan management component allows customizing a process model, resulting in a project plan.

• It includes as major elements– adding/removing tasks

– scheduling planned start and end times of processes

– assigning agents to processes.



Activities in Project Planning

• Planning activities– Add/Delete of processes

– Assigning resources, temporal planning

– Method selection as a „macro“ for several planning steps

– Change of plans, replanning

– Replanning triggers automatic update of execution states

– Notification of involved agents (change management)

• The project plan management component allows customizing a process model, organizing the activities resulting in a project plan.



Planning as an Activity (1)

• Planning has many creative aspects. A radical solution is to reduce the machine support to documentation. If this is well structured it has ist merits.

• There can also be additional service:– Notification if something is missing or was changed

– Availability of a process model

– Providing knowledge support by connecting information needs and information sources

– Supporting the interleaving of planning and execution.

• MILOS is very close to this kind of support. The real intelligent“ work is done by the human.



Planning as an Activity (2)

• The tendency is to add more machine support.

• Examples:– Extending the process models to an experience base that gives

proper recommendations.

– Defining specific tasks formally so that they can be performed by machines like:

• scheduling

• resource planning.

• It is important that this automatic support should still be considered as the work of an assistant. This agent gives only recommendations but can often do better than a human.



Because no specific agents have been defined the project manager has to do all the jobs



Project Execution (1)

• In the project execution the products are created. These products are considered as dynamic project data and are stored in attributes.

• The different activities have to be coordinated

• These activities include:– Access on input products has to be provided

– Generating and changing of output products

– Temporal predictions

– To-Do-lists for participating agents have to be generated

– Connection of external product editors have to provided

– Access to information on the project state has to be made



Project Execution (2)

• The PSEE guides the execution.

• For this purpose the project plan has to be interpreted what is done by a workflow engine.

• The workflow engine provides the execution environment.



Example: Execution Coordination

System-DesignVersion 1 Implementation Code

Version 1

Peter

Notify Peter!System-Design

Version 2

See change management



Part 3

Change Management



Change Management

• Because events produce new situations the situations change continuously.

• An important aspect of the required flexibility is to react properly on such changes.

• Such reactions are actions of some agents.

• The systematic treatment is called change management.



ECA-Rules

• The most important management concept is the

Event-Condition-Action rules

• Event: Something which happens

• Condition: Description of special circumstances

• Action: Any kind of action

• The rule is of the form

• IF– Event AND Condition

THEN Action



Pro-Active Actions, Triggering

• We distinguish two kinds of actions:– Actions on demand

– Pro-active actions: Without explicit demand

• Pro-active actions should not be done at random: There, where it is necessary but not unnecessarily.

• Therefore a trigger is needed for starting such actions.

• The most important form of a trigger is represented by ECA-Rules



Events

• Events are executed actions that produce a partially new situation. Therefore the result of an event is one or more knowledge units.

• The specific aspect of an event is its origin from an action. This action may result from– A planned action– From an external agent

• They can be– Expected or surprising

• It may contain– Expected or unexpected information

• Events happen at a certain time



Reminder: Events in Java (2)

• The Java Event Model (1) :

• An object in which „something happens“ can generate an event

• Example: A button, if pushed, generates an ActionEvent

• An event is again an object; it contains information about the happened event (e.g. The generator of the event)

• All interested listener will get the event and can work on it, independent of the object which generated the event



The Java Event Model (2)

• We distinguish the „observer“ (event listeners) and the „observed“ (event generator)

Event generator

Observer 1

Observer 2

Observer 3

Events are objects (subclasses of java.util.EventObject)



Conditions

• The conditions describe how and when the action has to be performed.

• If the action is a notification then the condition says who has to be notified. This is not necessarily the name of some agent, it is often better to describe the function of the agent:– notify the persons responsible for task 4, and not: notify Bill (because

at the time when the action is performed this agent may be Joan).

• It is difficult to formulate the condition exactly. Types of errors– someone is notified unnecessarily

– someone who should be notified is not

• One has to consider which errors are more costly.



Change Rules

• A Change Rule references an an Action Class and a Condition

• The Action Class describes an executable action (e.g. a NotifyAction)

• This action will be executed under certain conditions (described in Condition),

• After a certain event has happened (implemented in a Basic Event)



Change Operators

• The actual change is described by operators. These operators are defined on information units, e.g. on attributes, formulas etc.)

• We distinguish (as usual) three types of operators:• ADD operators: Adding an information unit.• REMOVE operators: Removing an information unit.• MODIFY operators: Replaces some information unit by

another one. This can be defined as a macro operator in terms of ADD and REMOVE.

• Operators are defined on a general level and can be instantiated.



Example: Change Rule (1)

• „If some agent gets by email-address a task then notify the agent“

• Action: „notify the agent“– ActionClass: NotifyAction

• Condition: „Agent has an email-address“

• Event: „some task is associated“– EventClass: AgentAssigned



Example: Change Rule (2)

ChangeRuleClass:AgentAssignment

ConditionClass:AgentAssignmentTests

ActionClass:NotifyAction

addAgentAssignedListener()(1)

Agent assigned(2)

Data structure:PlanProcess

Event:AgentAssigned

eventOccurred() (3)

eval()

(4a)true(4b)

(5)

perform()



Example (1): Support of Planning

• Rules (on an abstract level)– IF <Process-1> is delayed (EVENT) AND <Process-1> is

predecessor of <Process-2> (CONDITION) AND<Planner> is responsible for <Processs-2> (CONDITION) THEN notify <Planner> ! (ACTION)

• Concrete situation (instances of conditions): – System design is predecessor of implementation– Karin is responsible for implementation

• Event (Instance) :– System design is delayed

• Action: notify Karin!



Example (2): Notification Rule

• Event: output type for system_design changed

Cond.: component_design needs input of type OMT-Doc &

planning task for component_design is assigned to agent x

Action: notify agent x

system_design

UML-Doc

OMT-Doc component_designOMT-Doc



Example: A Change Rule in MILOS

ChangeEvent

ChangeEvent()getChangedPart()

BasicChangeEvent

BasicChangeEvent()

Action

destination : MILOSObject

getDestination()perform()setDestination()

ChangeRule

ChangeRule()addAction()addCondition()evaluate()getActions()getCondition()removeAction()removeCondition()eventOccured()

**

BasicChangeEventListener

eventOccured()

<<Interface>>



The Key Structure: Dependencies

• Actions are not isolated but have various dependencies of a complex character.

• Many dependencies are „silent“ and activated by certain events.

• In actual situations dependencies have to be identified and put into actions.

• Management requires – a thorough understanding of these relations in order to perform optimal

activities.

– to structure the dependencies.



Dependency Management

• Changes require notification and updates. What is involved?

• Sources of changes

• Process models: Flow of data, decomposition

• Domain knowledge: Product models, types of tasks

• User knowledge

• Management of change rules:

– Definition of change rules: Heuristics

– Mappings: Dependencies to change rules

– Propagation of change effects



Representation of Dynamic Dependencies• Changes lead dynamically to activities

• All dynamic activities are governed by ECA rules:

• ECA rule:

– events: changes to the project plan/state

– conditions

– actions: e.g. notifications



Types of Dependencies

User

MILOS System

Domain specific dependencies

System interface: ECA-Rules

User interface: Modeling language

enacts

Looks at

#

Modeling view:

Operational view: view:

In principle there are two types of dependencies: Those derived from a process. Those specific for the application domain.



Product Dependencies Induced by Processes

Product Dependency

Product Dependency

DesignDocument

(UML)

JavaTest Driver

JavaImplemen-

tation

implementin Java



Events in MILOS (1)

• In MILOS events are used in order to send change notifications between different system components.

• E.g., the project plan triggers an event if the planner has added a new process to the plan.

• For this event the WFE inscribes as listener.

• This means, the WFE will be notified if a new process is added to the plan.



Events in MILOS (2)

• Events can also be used for notification of users.

• Example: An agent obtains a new process to work on.

• The WFE triggers an event which is transferred to the email system. übergeben wird. This generates a mail to the involved user.



Events in MILOS (3)

• There are 2 kinds of events in MILOS:

• Complex Events combine several events under a common „headline“.

• Example: MILOSProcessDefinitionChangedEvent

• Basic Events represent exactly one exactly specified event.

• Example: MethodAddedEvent



Basic Events

• A single, special event.

• Listener has only to implement eventOccurred()

• Is used if a listener is only interested in a special kind of events.

• Example: Change Rules



Part 4

Evaluation and Measurement



The Success of a (Support) System

• The problem is that users cannot specify exactly what they ultimately want.

• In a specific situation they are usually able which alternative they prefer.

• Therefore any kind of a priori evaluation of a support system has some risks, some uncertain elements.

• What is desiderable is a system that ultimately can a correct prediction of the value of a support system.

• Because this is not directly available this points into the direction of a learning system.



How to Evaluate a (Support) System? (1)

• The (support) system is a formal object that is applied in reality. The comparison and the evaluation can therefore not be a mathematical task.

• Comparisons with reality need experiments in real life.

• Problems:– Experiments are expensive

– Experiments are hard to control and cannot be exactly repeated.

– The results of experiments are hard to interpret and their conclusions are of doubtful.

– The influence of specific aspects can hardly be determined

• Traditional benchmarks are only for automated subprocesses of socio-technical processes adequate!



How to Evaluate a (Support) System? (2)

• An alternative to experiments are simulations.

• Advantages:– Simulations are cheap

– They can be controlled precisely and repeated at any time at any place.

• Disadvantages:– Simulations can be far from reality, therefore some real

experiments are still necessary.

• But: Simulations can be used– To refute certain models

– For sensitivity analysis and improvements of the support system



OM

Why Evaluation?

• Reuse of knowledge

• Explicit documentation of tacit knowledge

• Dissemination of knowledge

• Effort for acquisition, structuring, maintenance of knowledge

• Software, hardware, ...

BenefitsCosts

?

OM Ask the users!

SUCCESS!?

This applies in general, not onlyfor support systems



Terminology: Software Measurement (1)• Measurement is the process by which values are assigned to attributes of entities in the

real world in such a way as to describe them accordingly to clearly defined rules.

• Empirical relational system is an ordered tupel (A, R1,..., Rn,o1 ,..., om ) with A a non-

empty set of empirical entities, Ri are ki-ary relations on A and oi are binary operations

on A.

• Formal relational system is an ordered tupel (B, S1,..., Sn,1 ,..., m ) with AB a non-

empty set of formal entities, Si are ki-ary relations on B and oi are binary operations on

B.

• Measure is a mapping :A B from attributes of empirical entities a A to formal entities (a) B in such a way as to describe them accordingly to clearly defined rules.

• The triple (A,B, ) is a scale if and only if for all i,j and for all a1,..ak,b,c A holds:Ri(a1,..ak) Si((a1),.. (ak)) and (b oj c) = (b) j (c) (determining the admissible transformations)



Terminology: Software Measurement (2)• Meaningfulness: if a statement is invariant against all admissible transformations

• Measurement unit determines how the attribute is measured.

• Measurement range defines a set of permissible values for a measure.

• Measurement protocol defines the measurement of a specific attribute in a way that it s repeatable and comparable.

• Internal attributes: can be measured based on the entity itself

• External attributes: can be measured only with related entities

• Direct measurement: does not depend on the measurement of any other attribute

• Indirect measurement: involves the measurement of other attribute(s)

• A measure is valid, if it satisfeis the Representation Condition: captures in the formal world the behavior we erceive in the empirical world.



Terminology: Software Measurement (3)

• Descriptive modelFunction m = f(x1,..,xn) where m is a measure based on a model integrating n other measures x1,..,xn

• Evaluation modelFunction d = f(x1,..,xn) where d {d1,..,dm} set of all possible alernatives and f is a decision function with the decision criteria x1,..,xn as inputs.

• Prediction model1) Function e = f(x1,..,xn) where e is the estimate of the variable to be predicted and x1,..,xn are inputs.

• 2) Function p(e) = f(x1,..,xn) where p(e) is the probability of the occurence of event e.



Application scenario : Definition of Measures

Measure.2: # of per module

Measure1: Total # of errors

Goal: Analyse the SW prozess w.r.t. reliability from the viewpoint of the developer of the company

...

Definition

M1 M2

% errors

M3 ...

Measures?

Model : #errors M1/#errors total; ...

Question: How are the errors distributed over the modules of the SW system?



Principle of the Goal-Question-Metric Technique: Retrieval System

GQM Goal

M1 M2 M3 M4 ...

...Q1 Q4Q2 Q3 (Questions)

(Measures)

Interp

retation

Model1 ...Model2 Model3 (Models)

Business Goal Success of

Retrieval

Utility of retrieved

information

Impact of document origin

on degree of maturity?

Per retrieval attempt: for

each chosen doc: doc

attribute “doc origin”

G

Q Q

M M M M

Def

init

ion



GQM 1. First study

GQM 2. Definition of GQM goals

GQM 5. Data collection, validation and storage

GQM 3. Development of GQM plan

GQM 6. Data analysis and interpretation

GQM 7. Post-mortem analysis

GQM 4. Development of data collection plan

GQM 8. Storage of experiences

Measurement and Evaluation in Software Development

• Goal/Question/Metric (GQM) Approach:



6 - Package Prestudy

Interpret collecteddata and compareto goals

4 - Collect data

3 - Derivemeasurementplan

IdentifyGQM goals

DevelopGQM plan•Feedback session

with experts

• Usage trials with experts •Derive and implement measurement plan for CBR-PEB

•Identify evaluation focus: analyze whether system is successful, or not

•Prepare next step

•Rework according to decisions made in feedback session

•Lessons Learned about OM evaluation

•Interviews with experts: fill out abstraction sheets

• Construct GQM plan

• Interviews with experts: GQM goal setting

Practical Application of the GQM Technique

G

Q Q

M M M M

Plan

Execute

Evaluate



Example: Definition of Goals

• Object

• Purpose

• Quality focus

• View

• Context

GQMGoal

SE Glossar• requirement document: A document that specifies the

requirements for a system...

• software: Computer programs, procedures, and possibly associated documentation and data pertaining to the operation of a computer system.

• ...

SE Thesaurus• implementation: programming.

• requirement document: specification.

• ....

...

maintainability

expandabilitycorrectabilityunderstandability

quality factor

reliability ...

...

usability

SE Entity



Discussion

• The GQM approach is expensive: Reuse of measure plans can be useful: See experience factory.

• The GQM approach does only implicitly deal with the customers utility: Do really measure what the user wants?

• For systems like MILOS and others the GQM approach does not make use of– The process model

– The detailed relations between the process steps and in the attached info needs and info sources