PROMISE 2008: wrap up

The PROMISE effect:

Do more together than separately

Gary D. Boetticher Tim Menzies Tom Ostrand Guenther Ruhe

2

The other side of PROMISE

PROMISE is not just an annual meeting, it is community.

How much contact have you had with your fellow PROMISE-ians during the year?

Are you exploiting the “PROMISE effect”? Do more together than separately.

3

Case study #1

Cukic & Menzies: new NSF award 2008 to 2011 “Overcoming ceiling effects in defect predictors

Surprisingly good reviews High praise for the connection to the PROMISE repository What are the broader impacts of the proposed activity?

“… The work has significant broader impacts and the PROMISE repository that the PIs are a part of provides them a platform for making further contributions to the SE community. …”

Summary Statement “… The PROMISE repository and the work of the PI represents a

significant input to the proposal. …. “ So the repository is now a tool for access funding.

4

Case study #2 For several years, Ostrand, Weyuker, Bell at AT&T held the

state of the art in defect localization (predicting the subset of the code with most errors)

2007: At PROMISE’07: Koru argues that defect distribution is logarithmic (I.e. more errors in the

smaller modules) Sept 2008:

Milton (WVU) tells Menzies that their new defect localizer is failing. Menzies says “try crawling the suspect modules in the Koru order”

(smallest modules first) Jan’08:

Milton shows the new localizer dramatically out-performs standard methods

Mar’08: Milton hosted at AT&T

for a week by Ostrand Tests the new localizer inside the firewalls at AT&T

5

Case study #3

TSE 2007, May, Kitchenham et.al. assess

within-company (WC) data vs cross-company (CC) data for effort prediction. August ‘07 : Menzies (USA) realizes that the PROMISE repository could

perform the same study, but for defect prediction. Crowd sources the PROMISE community for collaborators

Sept ‘07: Bener&Turhan (Turkey) run the experiments Turkey makes the data, USA runs the statistics CC results: best PD results ever seen with this data (but very bad PF rates)

Oct ‘07: Submission to TSE by Menzies, Bener, Turhan CC depreciated for defect prediction

Feb 14: TSE decision: revise & resubmit May 1: Resubmission with 3 news data sets (Turkish whitegoods)

Paper now much stronger PROMISE repository now has 3 more data sets

Other Bener, Menzies, Turhan papers: two Defects’08, ASE’08

6

Case study 4,5,6….

Over to you… Any other stories to add of interactions

between PROMISE-ians?

7

The PROMISE effect

Do more together separately. Look around the room

What ideas are in the air? What is the value-added of the person sitting next

to you? What are you plans for collaboration over the next

7 months (in time for the next PROMISE cfp)? And we can use web-tools to foster this

community: Chat rooms, discussion groups, wikis, code library

8

Proposal: a PROMISE chat room

Two minutes chatting can sometimes beat 2 months of reading

?Meebo.com Web-based,

platform independent

9

Proposal:a PROMISE discussion list Chatting is a hardly

permanent record of a discussion

Good to access what others have said in the past

?groups.google.com

10

Proposal: a PROMISE wiki

Promisedata.org is read only (except for comments) Good for workshop admin

and data storage Chat rooms and discussion

groups good for discussion “volleys”

Wikis are better for summarizing and synthesizing on-going discussions write, rewrite access

?wikispaces.org

11

Proposal:a shared PROMISE code library Reading is one thing Running code is

another. Have a shared

open-source, code base, complete with make files

Able to reproduce prior results with one command.

?code.google.com ?a Debian package

12

Proposal:the PROMISE’09 awards Source award

Awarded to the individuals or groups that contribute the most new data, with baseline results, to the repository.

Service award Awarded to the individuals or groups that offer most

support to others in the new groups/ chat room/ wiki.

Science award Awarded to the 2009 paper that best repeats/extends/

interacts with prior results and/or authors of a past PROMISE paper.