65
Center for Software Engineering at USC Center for Software Engineering at USC Experience BASE Experience BASE For Software Engineering Research For Software Engineering Research Zhihao Chen Center for Software Engineering University of Southern California, Los Angeles, CA [email protected]

Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Embed Size (px)

Citation preview

Page 1: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Experience BASE Experience BASE

For Software Engineering ResearchFor Software Engineering Research

Zhihao Chen

Center for Software Engineering

University of Southern California, Los Angeles, CA

[email protected]

Page 2: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

AgendaAgenda

CSE Experience Management Systems

CSE Experience BASE Research

CSE Experience BASE Implementation

Q & A

Page 3: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

http://ebase.usc.edu

Initiatives

Planning context

Progress / Plan /

Goal Mismatches

Experience Base

Analyzed experience,

Updated models

Achievables,

Opportunities

•Org. Improvement Goals

–Goal-related questions, metrics

•Org. Improvement Strategies

–Goal achievement models

Organization Improvement Initiative Planning & Control

•Initiative Plans

–Initiative-related questions, metrics

•Initiative Monitoring and Control

–Experience-Base Analysis

Organization Shared Vision & Improvement Strategy

Experience Factory Framework

Project Shared Vision and Strategy

Planning Context

Models and data

Project experience

Organization Goals

Project Planning and Control

Models and

data

Model Integration

MBASE,

CeBASE

Success Models

Product

Models

Property Models

Process

Models

Win-Win • Business Case Analysis • Results Chains

• Risk • Software Warranties • Correctness

• RAD • Six Sigma • Stories

• Award Fees • Agility

• JAD • QFD

• Golden Rule

•Waterfall

• Spiral • RUP • XP

• SAIV • CAIV • SCQAIV

• Risk Management

•Business Process Reengineering

•CMM’s • Peopleware

•IPT’s • Agile Development

•Groupware • Easy WinWin

• Experience Factory • GQM

UML • XML

• CORBA • COM

• Architectures

• Product Lines

• OO Analysis & Design

• Requirements

• Operational Concepts

• Domain Ontologies

• COTS • GOTS• COCOMO II

• COCOTS • CORADMO

• System Dynamics

• Metrics • - ilities • COQUALMO

• Simulation and Modeling

Riskanalysis

Riskanalysis

Riskanalysis

Riskanalysis Proto-

type 1

Prototype 2

Prototype 3Opera-tionalprotoype

Concept ofOperation

Simulations, models, benchmarks

S/Wrequirements

Requirementvalidation

DesignV&V

Productdesign Detailed

design

CodeUnit test

IntegrationtestAcceptance

testService Develop, verifynext-level product

Evaluate alternativesidentify, resolve risks

Determine objectivesalternatives and

constraints

Plan next phase

Integrationand test plan

Developmentplan

Requirements planLife-cycle plan

REVIEW

Operations Model`

Object Model

Capability Requirements

System Definition

Class Model

Project Requirements

Statement of Purpose

Project GoalsOrganization Goals

System Capabilities

Component Model

Organization Entities

Behavior Model

Enterprise model

Domain DescriptionSystem Analysis System Design

Operational Concept Description (OCD) System and Software Requirements Definition (SSRD)System and Software Architecture Description (SSAD)

Organization Background

Organization Activities

Interaction Model

Levels of Service Goals

LOS Requirements

Coverage / Traceability of MBASE Product Models*

* Does not include all MBASE models

Release Description

Reqs. Satisfaction

Capability Tests

Data Structures

Methods/functions

LOS Tests

Implementation

Construction,Transition,Support (CTS) External to MBASE

The Spiral Model

The WinWin Spiral Model2. Identify Stakeholders’

win conditions

1. Identify next-level

Stakeholders

Reconcile win

conditions. Establish

next level objectives,

constraints, alternatives

3.

Evaluate product and

process alternatives.

Resolve Risks

4.

Define next level of product and

process - including partitions

5.

Validate product

and process

definitions

6.

Review, commitment7.

Win-Win

Extensions

Original

Spiral

Model/Document Assessment

PLANS: OCD

Reqt. Activity

Data Gathering Instrument

1. Inspection Report 2. Review Report 3. Testing Report 4. User Feedback

Defect Defect

Doc

Model

Protoype?

Defect

Artifact OCD

SSRD

Defect Information

Summary defect Informtion

Inspection

PBRARBPrototype

Grading

COQUALMO

Form

Model Doc . . .SSAD

DesignActivity

Defect

Inspection

OORT

ARB

Prototype

Grading

Defect Information

CSE ExperienceManagement System

Models, M

odel Integration, and Empirical M

ethodsM

odels, Model Integration, and Em

pirical Methods

Page 4: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Why We Build the SystemWhy We Build the System• “Our knowledge has legs – it walks home everyday”

– Leif Edvinsson, Brain of the year ‘98, director of intellectual capital, Skandia Insurance Co.

• “Not only do you have experience walking out the door, you have inexperience walking in the door”– Scott Eliot, Director Knowledge Management Product Groups,

Lotus (KMWorld 2001)

• Particularly problematic for software organizations (human and knowledge intensive)– Loss of knowledge

• When people leave– Lack of knowledge

• Novices lack knowledge• Experts lack time• Location of knowledge is not known

Page 5: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Why We Build the SystemWhy We Build the System

• The Need for Capturing and Sharing Process and Product Knowledge

• The Need for Domain Knowledge

• The Need for Acquiring Knowledge About New Technologies

• The Need for Sharing Knowledge About Local Policies

• The Need for Knowing Who Knows What

• The Need for Distance Collaboration

Page 6: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Why We Build the SystemWhy We Build the System

• Organizations must deliver products faster, cheaper, and with higher quality than before.

• To learn from experience an organization must document and share knowledge and experience to:– become less dependent on its employees

• Knowledge is often lost when employees leave the company– unload its experts

• Sharing knowledge the traditional way is very time consuming– get new employees to become productive sooner

• New employees need to learn from previous experience• It takes a fairly long time to collect experience because it’s

unstructured, undocumented, and even unavailable

We do this because we know it is important, not for us only, but for all people.

Page 7: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

How We Build itHow We Build it

• Experience has been– Captured, – Structured, – Searchable, – Made available, and– Maintained.

• We also develop– An experience management framework with concepts– Methodologies governing how the experience is structured– Processes, procedures and roles governing how the

experience is managed on a daily basis – Supporting tools that helps with all of the above

Page 8: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

What is Our EMSWhat is Our EMS• Our framework for experience management is based on

Experience Factory (EF)– EF has been successfully applied to NASA for more than 25 years– We have used the EF and applied it to manage our software engineering experiences for 8 years

• We have a series of experience management projects that develop processes, tools, and taxonomies to work with us to tailor and implement the experience factory to our needs.

• Content– data, information, knowledge or experience

• Structure– the way the content is organized

• Procedures– instructions on how to manage the experience base on a daily basis, including how to use,

package, delete, integrate and update experience

• Tools– support managing the content and the structure, and carrying out the procedures, as well as

helping capture, store integrate, analyze, synthesize and retrieve experience

Page 9: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Experience Factory FrameworkExperience Factory Framework

Initiatives

Planning context

Progress/Plan/ Goal Mismatches

Experience Base

Analyzed experience, Updated modelsAchievables,

Opportunities

• Org. Improvement Goals– Goal-related questions,

metrics• Org. Improvement Strategies

– Goal achievement models

Org. Improvement Initiative Planning & Control

• Initiative Plans– Initiative-related

questions, metrics• Initiative Monitoring and

Control– Experience-Base

Analysis

Org. Shared Vision & Improvement Strategy

Project Shared Vision and Strategy

Planning Context

Models and data

Project experience

Org. Goals

Project Planning and Control

Models and data

Page 10: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

eBASEeBASE Results ChainsResults Chains

Page 11: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

CSE Experience Management SystemCSE Experience Management System

Project Support

5. Analyze

Experience Base

Generalize

Tailor

Formalize

Disseminate

1. Characterize2. Set Goals3. Choose Process

4. Execute Process

Project analysisProcess

modification

Data, lessonslearned

Product, lessons learned, models

environmentcharacteristics

Tailorable knowledge consulting

Experience Management System (EMS)Project Organization

6. Package

CSE Instructional Staff as "Managers" of Teams

Graduate Teams

Done by project team by CSE’S Researchers and Instructtional staff of CS577ab

Team stakholders extract to the best of their ability

Artifacts [Project / MBASE documents, including specifications, plans, support

material, code, manuals, ....], process metrics, quality data, developer

background

Researchers, archiveist, ....generated by teams and

stakeholders, manual and (semi-) automated systems, individuals (critiques and

lessons learned),

Instructional staff and CSE reasearchers

Page 12: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

CS577 Model/Document Assessment

PLANS:OCD

Reqt.Activity

Data Gathering Instrument

1. Inspection Report2. Review Report3. Testing Report4. User Feedback

DefectDefect

Doc

Model

Protoype?

Defect

ArtifactOCD

SSRD

DefectInformation

Summary defectInformtion

InspectionPBRARBPrototypeGrading

COQUALMOForm

ModelDoc . . .SSAD

DesignActivity

Defect

Inspection

OORTARBPrototypeGrading

DefectInformation

Page 13: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Data in eBASEData in eBASE

Project_Domain: Project_Name: Project_Type: Customer: Developers: Number_of_Developers: COTS: Team_Number: Academic_PeriodSchool_Course

Attributes

Students auto-registration, Search, MBASE Tree, Discussion, (Visualize the value of attributes, Opportunity tree, )

Processes

(for students, researchers, clients)Instructions

(Individual Critiques, Homework and IV&V assignments; grading guidelines;)Lesson Learned

MBASE guidelines; Archiving guidelinesGuidelines, Standards, ...

(CS577 effort)Effort Data

(Grades; Client evaluations)Success Model Data

Effort to WBSProperty Model Data

Effort; Entry/Exit Criteria/conditions Process Data

OCD, SSRD, SSADRose/UML model(s)Risk lists; Project statusquality data in paper form. COCOMO, COCOTS and CORADMO raw data

in paper form. •

Property Model Data: Artifacts

eBASE Items maintained by eBASE

Page 14: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

People from more 60 People from more 60 countriescountries

Are Accessing Our EMS

Page 15: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

People are using our experience BasePeople are using our experience Base

learn

reuse

Experience Factory

Project DProject D

Project A

Project C ? !

Project DProject

Experiences

Knowledge

Project B

Experience Base

e-Se

rvic

es S

yste

ms e-datae-In

fomration

e-Project Management

Page 16: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

People are using our experience BasePeople are using our experience Base

• Learning is cheap• Avoid mistakes from the past• Higher quality at lower costs• Provide solution to recurring issues • Reuse experience

– Products, processes & technology– Estimate changes & risks– Incorporate collective learning to repository

• Provide ongoing improvement• Analyze, synthesize & reuse experience• Reuse of models

– Defect model, cost model, effort distribution, project plan & s/w architecture– Establish competitive advantage

• Apply to projects with similarities– Initial seed, evaluation & reseeding– Tailor to specific situation– Management support

• Improve s/w development competency– Build baseline for measurement– Reuse models– Reduce defect

Page 17: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

AgendaAgenda

CSE Experience Management Systems

CSE Experience BASE Research

CSE Experience BASE Implementation

Q & A

Page 18: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Project Statistics from 1996 to 2000Project Statistics from 1996 to 2000

Page 19: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Project Activity Effort DistributionProject Activity Effort Distribution

Page 20: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Project Activity LevelProject Activity Level

Page 21: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Major Project ArtifactsMajor Project Artifacts

Page 22: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

UseUse--case vs. SLOCcase vs. SLOC

Effect of Project Complexity

• Group the projects by their number of use cases.• : the projects with greater value of the number of

sequence-diagram-steps per use case comparing with its othergroup members.

Page 23: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Growth of COTS Intensive CS577 ProjectsGrowth of COTS Intensive CS577 Projects

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

1997 1998 1999 2000 2001Year

Page 24: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

COTS Effort Distribution in CS577 ProjectsCOTS Effort Distribution in CS577 Projects

0%

20%

40%

60%

80%

100%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Assessment Tailoring Glue Code

Page 25: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

COTS Project Activity Sequences

No Effort Sequences

Incp. Elab. Cons. Tran.

1 A AC ATG C

2 A AT A A

3 A (TG)A G G

4 A A(TG) A(TG) G

5 AT AT T T

6 A T TG G

7 AT T T T

8 AT (AA) TG (TGC) G

9 A AT TG G

Page 26: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

CBA Effort Decision Framework CBA Effort Decision Framework

Page 27: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Quality attribute and stakeholder conflictsQuality attribute and stakeholder conflicts

Page 28: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

GeneralPublic

Avoid adverse

Interoperator User Maintainer Developer Customer

Assurance

Interoperability

Usability

Performance Evolvability& Portability

Cost&Schedule

Reusability

Avoid current andfuture interfaceproblems

Execute

effective

missions

Avoid low utilitydue to

Cost-effectiveproduct supportafter development

Avoid non verifiable,expendable, flexible,reusable product; Avoid the delay of product delivery

Avoid overrunbudget andschedule; Avoidlow utilization ofthe system

and cost overrun

obsolescence; cost-

operational

systemside-effects:safety,security/privacy

directly-concerns

Stakeholders and their roles & responsiblities:

: low rating probably situation-specific*

: the new added relation after the data analysis * The Data is based on win conditions of 14 student digital library projects

Empirically Determined

Top-Level Stakeholder/Value Dependencies

Model - Cl ash Types Di st r i but i on

4% 12%3%

16%

4%13%30%

7%6% 5%

Success- Pr oper t ySuccess- Pr oductSuccess- SuccessPr oduct - Pr oper t yPr ocess- Pr oper t yPr oper t y- Pr oper t yPr oduct - Pr oductPr ocess- Pr ocessPr oduct - Pr ocessSuccess- Pr ocess

MOdel Cl ashes vs. Cont r i but i on t o Pr oj ect Ri sk

Model Cl ashesCont r i but i on t o Ri sk

Model Clashes 4% 12% 3% 16% 4% 13% 30% 7% 6% 5%

Contribut ion to Risk 6% 17% 4% 20% 5% 12% 24% 5% 4% 3%

Success-

Property

Success-

Product

Success-

Success

Product -

Property

Process-

Property

Property-

Property

Product-

Product

Process-

Process

Product-

Process

Success-

Process

Comparison of CBS Effort

0%5%

10%15%20%25%

Team In

t eract

ionCOTS asse

ssmen

tClient

Intera

ction

Life Cycl

e Plann

ingProj

ect Web

-site

Trainin

g and Prep

aratio

nCOTS Tailo rin

g

Transiti

on and

Support

Glue Cod

ecus

tom deve

lopmen

Activity

ACISTCISGCISNon-CIS

Stakeholder/Attribute Relationship

http://ebase.usc.edu

Process M

odels

Product M

odels

Property M

odels

Success M

odels

ExperienceB

ases

Testbeds

Model

Integration

Empirical

Methods

0%

20%

40%

60%

80%

100%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Assessment Tailoring Glue Code

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

117

34

54

146

94

72

53

81

141

194

5040

9084

93

44

110

142

111113

55 5769 76

585120

30

1328 30

40

17 24 28

4942

34

19 26

4338

25 21

5361

39 37

24 30 2720 22

0

20

40

60

80

100

120

140

160

180

200

# of

Sta

tem

ents

Project Number

Distribution of # of Brainstorming Statements and Win Conditions

# of Win Conditions# of Statements

158

86

183

107

7018

89

47

4827

0

50

100

150

200

250

300

# of

Win

Con

ditio

ns

Project andProcess

Capabilities Interface Level ofService

Evolution

Main Topics

Distribution of Win Conditions w/t Issues

IssueNo Issue

193

31111

245

4182

77

550

93

3040

35

2956

0

50

100

150

200

250

300

# of

Win

Con

ditio

ns

Project andProcess

Capabilities Interface Level ofService

Evolution

Main Topics

Distribution of Prioritized Win Conditions

Forget ThemMaybe LaterImportant with HurdlesLow Hanging Fruit

Cont r i but i on of Model - Cl ash Types t o Ri sk

6%17%

4%

20%5%12%

24%

5% 4% 3%

Not All CBS Are Created Equally:

COTS Intensive Project Types

Model clash identification and analysis

Easy WinWin

Models, M

odel Integration, and Empirical M

ethodsM

odels, Model Integration, and Em

pirical Methods

Page 29: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

http://ebase.usc.edu

Process M

odels

Product M

odels

Property M

odels

Success M

odels

ExperienceB

ases

Testbeds

Model

Integration

Empirical

Methods

1 - Very Dissatisfied; 2 – Dissatisfied; 3 – Neutral; 4 – Satisfied; 5 - Very Satisfied

Client: Owner + Main Users + Administrator + Maintainer (client side stakeholders)

Effect of Project Complexity

• Group the projects by their number of use cases.• : the projects with greater value of the number of

sequence-diagram-steps per use case comparing with its othergroup members.

3.45

5.314.7

4

0

1

2

3

4

5

6

Ave of Ef fort(hr)

Quality of plan(/5.0)

Group AGroup B

Planning IV&V from both groups

10171179

2196

523

813

1336

494366

860

0

500

1000

1500

2000

2500

Total Concerns Minor Concerns Major Concerns

Group AGroup BTotal

Number of concerns detected

Critical Success Factors

254312

566

147197

344

165142

307

451528

979

0100200300400500600700800900

1000

Complete Consistency Testability Other

Group AGroup BTotal

Number of concerns by the class

136

101

237

66 72

138

82

49

131

210

144

354

050

100150

200250

300350

400

Complete Consistency Testability Other

Group AGroup BTotal

Number of detected major concerns

7.710.76

4.456.79

5 4.9

13.67

18.21

30.82

40.66

05

1015202530354045

Complete Consistency Testability Other Total

Group AGroup B

Effort(hr) per person

Group A : 3.38hrGroup B : 3.98hr

The average number of detected concern by class per person

4.123.48

2 2.48 2.481.69

6.36

4.97

14.97

12.62

0

2

4

6

8

10

12

14

16

Complete Consistency Testability Other Total

Group AGroup B

4.43

4.69

3.17

7.04

3.79

5.89

0

2

4

6

8

10

12

Group A Group B Average

MinorMajor

9.12

10.219.67

The average number of detected major concern by class per person

The average number of concerns detected per hour

IV & Vers Empirical Data ExperimentSizingModels, M

odel Integration, and Empirical M

ethodsM

odels, Model Integration, and Em

pirical Methods

Page 30: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

AgendaAgenda

CSE Experience Management Systems

CSE Experience BASE Research

CSE Experience BASE Implementation

Q & A

Page 31: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Experience Base Systems

Database Servers Web Servers

Experience Base Repositories

ee--Information Information ManagementManagement

ee--ProjectProjectManagementManagement

UserUserManagementManagement

FeedbackFeedbackSystemSystem

InformationInformationRetrievingRetrieving

StatisticsStatisticsAnalysisAnalysis

SecuritySecurity

ManagementManagement

Public Services Affiliate Services

Web Interface Services

Web Browser(IE, NS, Mozilla …) MS Office XP/2003Windows Explorer

http://ebase.usc.edu

CSE ExperienceManagement System

Project Artifacts in All Life Cycle Phase Benefits from our experience BaseBenefits from our experience Base

learn

reuse

Experience Factory

Project DProject D

Project A

Project C ? !

Project DProject

Experiences

Knowledge

Project B

Experience Base

e-Se

rvic

es S

yste

ms e-datae-Infomration

e-Project Management

User Management

Registration Request Authorization

Deliver Registration

Service Deliveries

Public Info

Browse

infor

mation

Service Requests

Look

up

info

rmat

ion

UsersAuthorization to access Affiliate

Services

Check

for

the poli

cies

Acces

sing R

ole

Experience Base

RepositoriesAffiliate Services

Policy / Security Management

Registration / Authorization

Maintain User Information

Ask

for A

utho

rizat

ion

Store and retrieve

Public Services

Store and retrieve

e-Services

Unified Content & Collaboration Services

TeamProject description project attributes

progress information

client meeting notes

project domainproject type

COTS …

Project Artifacts

Research Information

Accessing Information

Project Information

Feedback Information

e-Information Managemente-Information

OCD, SSRD, SSAD, FRD, LCP, EasyWinwin, Prototype, UML model, PRP, QMP, TP, COCOMO file, source code, manual, …

Project rating, Project comments, Forum discussion …

Visiting info, traffic, countries, hosts, file type, page URL, connection-from info, search keywords, OS, browsers, day/week/month/year …

Papers, research projects, research activities, contact information …

• Experience has been– Captured, – Structured, – Searchable, – Made available, and– Maintained.

• We also develop– An experience management framework with concepts– Methodologies governing how the experience is structured– Processes, procedures and roles governing how the experience is managed

on a daily basis – Supporting tools that helps with all of the above

• Learning is cheap• Avoid mistakes from the past• Higher quality at lower costs• Provide solution to recurring issues • Reuse experience

– Products, processes & technology– Estimate changes & risks– Incorporate collective learning to repository

• Provide ongoing improvement• Analyze, synthesize & reuse experience• Reuse of models

– Defect model, cost model, effort distribution, project plan & s/w architecture

– Establish competitive advantage• Apply to projects with similarities

– Initial seed, evaluation & reseeding– Tailor to specific situation– Management support

• Improve s/w development competency– Build baseline for measurement– Reuse models– Reduce defect

Process Models

Product Models

Property Models

Success Models

Model Integration

Empirical Methods

Testbeds

Experience Bases

Models, M

odel Integration, and Empirical M

ethodsM

odels, Model Integration, and Em

pirical Methods

Page 32: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Implementation System OverviewImplementation System Overview

Experience Base Systems

Database Servers Web Servers

Experience Base Repositories

ee--Information Information ManagementManagement

ee--ProjectProjectManagementManagement

UserUserManagementManagement

FeedbackFeedbackSystemSystem

InformationInformationRetrievingRetrieving

StatisticsStatisticsAnalysisAnalysis

SecuritySecurity

ManagementManagement

Public Services Affiliate Services

Web Interface Services

Web Browser(IE, NS, Mozilla …) MS Office XP/2003Windows Explorer

Page 33: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Implementation Interactive ArchitectureImplementation Interactive Architecture

User Management

Registration Request Authorization

Deliver Registration

Service Deliveries

Public Info

Browse

inform

ation

Service Requests

Look

up

info

rmat

ion

UsersAuthorization to access Affiliate Services

Check

for

the po

licies

Accessi

ng Role

Experience Base

RepositoriesAffiliate Services

Policy / Security Management

Registration / Authorization

Maintain User Information

Ask

for A

utho

rizat

ion

Store and retrieve

Public Services

Store and retrieve

Page 34: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

ee--Information ManagementInformation Management

• Project Information– Team, project description, project attribute, progress information,

client meeting notes, project domain, project type, COTS …

• Project Artifacts– OCD, SSRD, SSAD, FRD, LCP, EasyWinwin, Prototype, UML

model, PRP, QMP, TP, COCOMO file, source code, manual, …

• Feedback Information– Project rating, Project comments, Forum discussion …

• Using Information– Visiting info, traffic, countries, hosts, file type, page URL,

connection-from info, search keywords, OS, browsers, day/week/month/year …

• Research Information– Papers, research projects, research activities, contact

information …

• Affiliate User Information

Page 35: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Unified Content & Collaboration Services

TeamProject description project attributes

progress information

client meeting notes

project domainproject typeCOTS …

Project Artifacts

Research Information

Accessing Information

Project Information

Feedback Information

e-Information Managemente-Information

OCD, SSRD, SSAD, FRD, LCP, EasyWinwin, Prototype, UML model, PRP, QMP, TP, COCOMO file, source code, manual, …

Project rating, Project comments, Forum discussion …

Visiting info, traffic, countries, hosts, file type, page URL, connection-from info, search keywords, OS, browsers, day/week/month/year …

Papers, research projects, research activities, contact information …

Page 36: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Accessing Public eAccessing Public e--ServicesServiceshttp://ebase.usc.edu/index.html

Page 37: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Need Authorization to Access Need Authorization to Access Affiliate eAffiliate e--ServicesServices

http://ebase.usc.edu/eservices/cs577a_2003/team01a/LCA/OCD_LCA_F03a_T1.doc

Page 38: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

ee--Project ManagementProject Management

• Browse / Explore projects through web– Support IE 4.x / 5.x / 6.x, Netscape 4.x / 6.x, Mozilla …

• Information Retrieving– Project Info Search / Project Artifact Search / Advance Project Artifact Search /

Attribute Search

• Rating• Discussion / Comments• Accessing Information (hits …)• Other project management features

– Versioning– Check-in/check-out– Document profiling– Publishing and approval– Role-based security

• Reader, Contributor, Web Designer, Content Manager, Member

Page 39: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Browse / Explore Artifacts through webBrowse / Explore Artifacts through web

Page 40: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Explore Artifacts through web (Affiliate Only)Explore Artifacts through web (Affiliate Only)

Page 41: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Explore Projects With Windows Explorer Explore Projects With Windows Explorer (Affiliate Only)(Affiliate Only)

Good for uploading a lot of project artifacts.

Page 42: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Information Retrieving Information Retrieving Project Info SearchProject Info Search

Basic Search shows in every page

Advance Search Options

Advance Search

Page 43: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Information Retrieving Information Retrieving Project Info Search Project Info Search –– Search ResultsSearch Results

Page 44: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Information Retrieving Information Retrieving Project Artifact Search (Affiliate Only)Project Artifact Search (Affiliate Only)

Page 45: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Information Retrieving Information Retrieving Advance Project Artifact Search Advance Project Artifact Search (Affiliate Only)(Affiliate Only)

• Results for searching database within OCD in LCA in pdf files only

Page 46: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Information Retrieving Information Retrieving Project Attribute SearchProject Attribute Search

Automatically Display the team web page when you point to team URL.

SQL statements

Page 47: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Information Retrieving Information Retrieving Project Attribute SearchProject Attribute Search

• Display the results vertically

Page 48: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Information Retrieving Information Retrieving Project Attribute SearchProject Attribute Search

(1)

(2)

(3)

• Search the projects that are related with “web based database”.

Page 49: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Information Retrieving Information Retrieving Project Attribute SearchProject Attribute Search

• Search results for the projects that are related with “web based database”.

Page 50: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Information Retrieving Information Retrieving Project Attribute SearchProject Attribute Search

• Search the projects that their COTS attributes are related with “intensive”.

Page 51: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Information Retrieving Information Retrieving Project Attribute SearchProject Attribute Search

• Search results for the projects that their COTS attributes are related with “intensive”.

Directly use SQL statements to find the projects.

Page 52: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Feedback System Feedback System -- RatingRating

Page 53: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Feedback System Feedback System –– Discussion/ CommentsDiscussion/ Comments(Affiliate Users only)(Affiliate Users only)

Insert discussion / comments about the project item

Insert discussion / comments about the project

Page 54: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Accessing Information Accessing Information (Administrator Only)(Administrator Only)

Page 55: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Accessing Information Accessing Information (Administrator Only)(Administrator Only)

Page 56: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Accessing Information Accessing Information (Administrator Only)(Administrator Only)

Page 57: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Accessing Information Accessing Information (Administrator Only)(Administrator Only)

Page 58: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Accessing Information Accessing Information (Administrator Only)(Administrator Only)

Page 59: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Accessing Information Accessing Information (Administrator Only)(Administrator Only)

Page 60: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Accessing Information Accessing Information (Administrator Only)(Administrator Only)

Page 61: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Other project management featuresOther project management features

Page 62: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

User ManagementUser Management

Account Pending

Account Approved

Account Activated

Account Created

User Info Database

Account Role

Password Mgt

Register an account

Users

Ask for approval

CSE / TA

Admin

Send approval

Account Approved

Account Pending

Account Approved

Activate Account

Create account in Affiliate Area

Assign account role

Account Role

Account mgt

Account mgt

Page 63: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

Role Based SecurityRole Based Security

• Reader – Has read-only access to the Web site.

• Contributor – Can add content to existing document libraries and lists.

• Web Designer – Can create lists and document libraries and customize

pages in the Web site. • Administrator

– Has full control of the Web site. • Content Manager

– Can create and manage areas, lists, libraries, and sites. • Member

– Can view and personalize portal site content and create sites.

Page 64: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

AgendaAgenda

CSE Experience Management Systems

CSE Experience BASE Research

CSE Experience BASE Implementation

Q & A

Page 65: Experience BASE For Software Engineering Research …sunset.usc.edu/events/2005/arr/proceedings/... · 2005-03-13 · Experience BASE For Software Engineering Research ... design

Center for S

oftware E

ngineering at US

CC

enter for Softw

are Engineering at U

SC

CSE Experience Management SystemCSE Experience Management System

END