Upload
rahimlotfi
View
122
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Empirical Software Engineering
Citation preview
Rahim LotfiAhmadReza Ghasemian
1
Table of Contents
Introduction Need for empirical studies in the context of software engineering.
Goals and Benefits
Basic Concepts: Measurement, Collection, Analysis
Basic Concepts for Empirical Software Engineering
Empirical Strategies: survey, case study, controlled experiment
Process of an empirical study
Tools
State of the art
Empirical Software Engineering
• Using other people’s experiences in management and development of softwares
• Empirical, included in all areas of science
• “I could have told you that from the start” or “That’s obvious.”
• Software engineering needs to follow the model of other physical sciences and
develop an experimental paradigm for the field.
• Judging empirical work by the standards of statistics texts would lead to
the suppression of many interesting results
ESE in the Literature
• “Empirical Software Engineering”
• “Experimentation in Software Engineering”• “ESEM - Empirical Software Engineering and Measurement “
History of Empirical Software engineering
• Victor Basili(1996)
• QuotesLike physics, medicine, manufacturing, and many other disciplines, software engineering requires the same high level approach for evolving the knowledge of the discipline; the cycle of model building, experimentation, and learning. We cannot rely solely on observation followed by logical thought.
History of Empirical Software engineering
Key figure
http://academic.research.microsoft.com
Key figures in Iran
Conferences (64) Empirical Software Engineering
Journals (29) Empirical Software Engineering
Table of Contents
Introduction Need for empirical studies in the context of software engineering.
Goals and Benefits
Basic Concepts: Measurement, Collection, Analysis
Goal Question Metric
Empirical Strategies: survey, case study, controlled experiment
Process of an empirical study
Tools
State of the art
Need of Empirical Studies
Why should we conduct empirical studies?
Product evaluation, e.g., prototyping.
Process evaluation1. Prototypes are not possible (simulation based on models).
2. A process is just a description until it is used by people
Important for research: experimentation is mandatory in other disciplines(e.g., medicine, physics, etc.)
Need of Empirical Studies
Empirical studies provides a systematic, disciplined, quantifiable and controlled way of evaluating human-based activities.
You will learn how to select, plan, conduct and analyse an empirical study.
Table of Contents
Introduction
Need for empirical studies in the context of software engineering.
Goals and Benefits Basic Concepts: Measurement, Collection, Analysis
Basic Concepts for Empirical Software Engineering
Empirical Strategies: survey, case study, controlled experiment
Process of an empirical study
Tools
State of the art
Goals and Benefits• The purpose of a study is
• The purpose of a study isto explore ...
to find out what’s happening
to seek for new insights
to ask questions and to find answers
Measurement: usually qualitative
• to describe ...
portray accurate profile of situations, events, projects , technologies
Measurement: quantitative/qualitative
• to explain ...
seek explanation of a situation/problem, usual in the form of causal
Relationships
Measurement: quantitative/qualitative
• relationships, differences, changes
Empirical Studies in SE
• Empirical Studies is research to improve Software Engineering Practice.
– Apply theoretical concepts in SE practice.
– Add experiences on their appliance to the SE ‘body of knowledge’
– Improve processes, methods and tools (SPPI approach).
– Verify theories and models.
Table of Contents
Introduction
Need for empirical studies in the context of software engineering.
Goals and Benefits
Basic Concepts: Measurement, Collection, Analysis Basic Concepts for Empirical Software Engineering
Empirical Strategies: survey, case study, controlled experiment
Process of an empirical study
Tools
State of the art
Some Basic Concepts• Measurement
- is the process of capturing data which are connected to real-world attributes to
describe them.
- Why is measurement important?
• Data CollectionCollection of qualitative / quantitative data according to research questions
• Data AnalysisAnalyzing the results according to the research questions ;
statistical tests
Measurement
• Quotes- “You can’t manage what you can’t measure”, Tom DeMarco
– “What is not measurable make measurable”, Galileo Galilei
• ObjectivesOne objective of science is to find ways to measure attributes of entities
we are interested in.
– Measurement makes concepts more visible and thus more
understandable and controllable.
• Definition
Measurement is the process by which numbers or symbols are mapped
to attributes of entities in the real world in such a way as to describe
them according to clearly defined rules.
Introduction
Need for empirical studies in the context of software engineering.
Goals and Benefits
Basic Concepts: Measurement, Collection, Analysis
Basic Concepts for Empirical Software Engineering Empirical Strategies: survey, case study, controlled experiment
Process of an empirical study
Tools
State of the art
Table of Contents
Quality Improvement Paradigm
Characterize & understand
Setgoals
Chooseprocesses,methods,techniques,and tools
Package &store experience
Analyzeresults
Executeprocess
Provide processwith feedback
Analyzeresults
Corporatelearning
Projectlearning
The Experience Factory Organization
A Different Paradigm
Project Organization Experience FactoryProblem Solving Experience Packaging
Decomposition of a problem Unification of different solutionsinto simpler ones and re-definition of the problem
Instantiation Generalization, Formalization
Design/Implementation process Analysis/Synthesis process
Validation and Verification Experimentation
Product Delivery within Experience / RecommendationsSchedule and Cost Delivery to Project
Introduction
Need for empirical studies in the context of software engineering.
Goals and Benefits
Basic Concepts: Measurement, Collection, Analysis
Basic Concepts for Empirical Software Engineering
Empirical Strategies: survey, case study, controlled experiment
Process of an empirical study
Tools
State of the art
Table of Contents
Table of Contents
Introduction
Need for empirical studies in the context of software engineering.
Goals and Benefits
Basic Concepts: Measurement, Collection, Analysis
Basic Concepts for Empirical Software Engineering
Empirical Strategies: survey, case study, controlled experiment
Process of an empirical study
Tools
State of the art
Table of Contents
Introduction
Need for empirical studies in the context of software engineering.
Goals and Benefits
Basic Concepts: Measurement, Collection, Analysis
Goal Question Metric
Empirical Strategies: survey, case study, controlled experiment
Process of an empirical study
Tools State of the art
Table of Contents
Introduction
Need for empirical studies in the context of software engineering.
Goals and Benefits
Basic Concepts: Measurement, Collection, Analysis
Basic Concepts for Empirical Software Engineering
Empirical Strategies: survey, case study, controlled experiment
Process of an empirical study
Tools
State of the art
State of the art
• Finding better ways to experiment and integrate the results of the studies
• Propagate the empirical discipline
• Build an empirical research engine for software engineering
• Build test beds for experimentation and evolution of processes
• Build product models that allow us to make trade-off decisions
• Build decision support systems offering the best empirical advice for selecting and tailoring the right processes for the problem
• References• V. Basili, G. Caldiera, D. Rombach: „The Goal Question Metric Approach“,2000.
• B. Boehm, H.D. Rombach, M.V. Zelkowitz: „Foundations of Empirical
• Software Engineering – The Legacy of Victor R. Basili“, Springer, 2005.
• Freimut et al.: "Empirical Studies in Software Engineering", Tutorial, VISEK
• Technical Report, 2002.
• IESE Tutorials on Empirical Software Engineering.
• A. Jedlitschka and D. Pfahl: "Reporting Guidelines for Controlled
• Experiments in Software Engineering", ISESE, 2005.
• B. Kitchenham: “Evidence-Based Software Engineering and Systematic
• Literature Review”, Profes, 2006.
• C. Wohlin, P. Runeson, M. Höst, M.C. Ohlsson, B. Regnell and A. Wesslen:
• "Introduction to Experimentation in Software Engineering", Kluver, 2000.
• M. V. Zelkowitz, D. R. Wallace: “Experimental Models for Validating
• Technology”, IEEE Computer, 1997.
• Site : http:// www.microstrategy.com
• Site : http:// www.jamasoftware.com
• Site : http://academic.research.microsoft.com
AcknowledgmentsBahman Zamani
Thank you for your attention
This research work has been supported by University of Isfahan