Sattose keynote 2015: Managing Socio-Technical Dependencies in Dsitributed Software Development

Computer)Science)and)Engineering,)UNL)So6ware)Quality)Research)Empirically>based)

e2.unl.edu))1)

Managing)Socio>Technical)Dependencies)in)Distributed)So6ware)Development)

Anita&Sarma&

University&of&Nebraska,&Lincoln&

July&2015&


e2.unl.edu))2)

SATToSE)2015)

General&Chair:&Tom$Mens$Program&Chair:&Anya$Helene$Bagge$&

Current&Research&Group:&

&&

&

&

&

&

&

&


e2.unl.edu))3)

CoordinaKon)

RAF&Leuchars&Airshow&2013,&Martyn&Edward,&Flickr&&


e2.unl.edu))4)

Dependencies)

Dependency,,&&&&&&&&&&&&&&&&&&&&,Flickr&&

Cleidson&R.&B.&de&Souza,&David&F.&Redmiles:&The&Awareness&Network,&To&Whom&Should&I&Display&My&AcSons?&And,&&Whose&AcSons&Should&I&Monitor?&IEEE&Trans.&SoUware&Eng.&37(3):&325Y340&(2011)&

Marcelo&Cataldo,&James&D.&Herbsleb,&Kathleen&M.&Carley:&SocioYtechnical&congruence:&a&framework&for&assessing&the&impact&of&&technical&and&work&dependencies&on&soUware&development&producSvity.&ESEM&2008:&2Y11&&


e2.unl.edu))5)

Parallel)Work)

parallel&worlds,&Alosh&Benne^&,&Flickr&&

Cleidson&R.&B.&de&Souza,&David&F.&Redmiles,&LiYTe&Cheng,&David&R.&Millen,&John&F.&Paêrson:&How&a&good&soUware&pracSce&thwarts&collaboraSon:&the&mulSple&roles&of&APIs&&in&soUware&development.&SIGSOFT&FSE&2004:&221Y230&

Dewayne&E.&Perry,&Harvey&P.&Siy,&Lawrence&G.&Voâ:&Parallel&changes&in&largeYscale&soUware&development:&an&observaSonal&case&study.&&ACM&Trans.&SoUw.&Eng.&Methodol.&10(3):&308Y337&(2001)&&Yuriy&Brun,&Reid&Holmes,&Michael&D.&Ernst,&David&Notkin:&Crystal:&precise&and&unobtrusive&conflict&warnings.&SIGSOFT&FSE&2011:&444Y447&

Anita&Sarma,&David&F.&Redmiles,&André&van&der&Hoek:&Palandr:&Early&DetecSon&of&Development&Conflicts&Arising&from&Parallel&Code&Changes.&IEEE&Trans.&SoUware&Eng.&38(4):&889Y908&(2012)&


e2.unl.edu))6)

Conflicts)in)Distributed)So6ware)Development)

!  Direct)Conflicts:&&Two&developers&edit&the&same&file&concurrently&(Merge&conflicts)&

&

!  Indirect)Conflicts:&&Conflicts&arising&because&of&changes&in&one&file&affecSng&changes&in&another&(Build&&&Test&conflicts,&and&other&breaking&changes)&


e2.unl.edu))7)

! Merge&conflicts:&8%&to&19%&&

!  Build&conflicts:&2%&to&15%&&!  Test&conflicts:&6%&to&35%&&

Conflicts)in)Distributed)So6ware)Development)Merge) Build) Test)

Project) #Merges) #conflicts) #)conflicts) #)Res.)Days)Avg)(Med)))

#)conflicts) #)Res.)Days)Avg)(Med)))

#)conflicts))

#)Res.)Days)Avg)(Med)))

Perl& 185& 74&(40%)& 14&(8%)& 23&(10)& 4&(2%)& 0.7&(1)& 56&(30%)& 31&(14)&

Storm& 88& 39&(44%)& 17&(19%)& 6&(2)& 9&(10%)& 5&(8)& 13&(15%)& 8&(3)&

Jenkins& 505& 204&(54%)& 68&(14%)& 23&(4)& 74&(15%)& 5&(2)& 28&(6%)& 7&(2)&

Voldemort& 380& 170&(34%)& 55&(15%)& 20&(4)& 16&(4%)& 2&(0.75)& 133&(35%)& 6&(4)&


e2.unl.edu))8)

CommunicaKon)

Resilience,&Dens&Leonis&,&Flickr&&

Marcelo&Cataldo,&Audris&Mockus,&Jeffrey&A.&Roberts,&James&D.&Herbsleb:&SoUware&Dependencies,&Work&Dependencies,&and&Their&Impact&on&Failures.&&IEEE&Trans.&SoUware&Eng.&35(6):&864Y878&(2009)&

James&D.&Herbsleb,&Audris&Mockus:&An&Empirical&Study&of&Speed&and&CommunicaSon&in&Globally&Distributed&SoUware&Development.&IEEE&Trans.&SoUware&Eng.&29(6):&481Y494&(2003)&

Sabrina&Marczak,&Ban&AlYAni,&David&F.&Redmiles,&Rafael&Prikladnicki:&The&Interplay&among&Trust,&Risk,&and&Reliance&in&Global&Systems&Engineering&&Teams.&ICGSE&2014:&46Y55&

Patrick&Wagstrom,&Subhajit&Daâ:&Does&laStude&hurt&while&longitude&kills?&geographical&and&temporal&separaSon&in&a&large&scale&soUware&development&project.&ICSE&2014:&199Y210&


e2.unl.edu))9)

Interdependency)

Challenge,&Just&me,&Aline&,&Flickr&&

Anita&Sarma,&Ban&AlYAni,&Erik&Trainer,&Roberto&Silveira&Silva&Filho,&Isabella&A.&da&Silva,&&David&F.&Redmiles,&André&van&der&Hoek:&ConSnuous&CoordinaSon&Tools&and&their&EvaluaSon.&CollaboraSve&&SoUware&Engineering&2010:&153Y178&

Adrian&Bachmann,&ChrisSan&Bird,&Foyzur&Rahman,&Premkumar&T.&Devanbu,&Abraham&Bernstein:&The&missing&links:&bugs&and&bugYfix&commits.&SIGSOFT&FSE&2010:&97Y106&


e2.unl.edu))10)

CoordinaKon)Tools)Can)Help)

Photo&by&www.gotcredit.com)on&Flickr&&


e2.unl.edu))11)

CoordinaKon)Paradigms)

ConSnuous&coordinaSon,&collaboraSve&architecture,&

seamless&development&environments,&

Asynchronous&communicaSon& Access&to&common&set&of&arSfacts,&isolated&workspaces&and&version&control&

Parallel&development,&roles&and&access&rights&

Passive&awareness&of&development&acSviSes&and&developers,&manage&informaSon&overload&

T&ask&allocaSon&and&assignment&

CommunicaSon&archival&along&with&arSfacts&

CollocaSon&benefits&to&distributed&development&

OrganizaSonal&memory,&knowledge&acquisiSon&and&

disseminaSon,&relaSons&navigaSon&

Prescribed&and&defined&&coordinaSon&support&

Advanced&conflict&detecSon&

Fine&grained&versioning,&conflict&resoluSon&

Instant&Messaging,&monitoring&changes&

to&arSfacts&

ArSfact&Management&CommunicaSon& Task&Management&

Anita&Sarma,&David&F.&Redmiles,&André&van&der&Hoek:&Categorizing&the&Spectrum&of&CoordinaSon&Technology.&IEEE&Computer&43(6):&61Y67&(2010)&


e2.unl.edu))12)















to&arSfacts&




e2.unl.edu))13)









Communication&


OrganizaSonal&memory&knowledge&acquisiSon&and&


Artifact Management& Task Management&




to&arSfacts& disseminaSon,&relaSons&navigaSon&


e2.unl.edu))14)

Examples)of)CoordinaKon)Tools)














to&arSfacts&




e2.unl.edu))15)

Tesseract)

!  Interactive and explorative environment to correlate and understand the complex relationships among: –  code (artifacts) –  communication records –  tasks (bug reports) –  time

[ICSE&2009]&

Anita&Sarma,&Larry&Maccherone,&Patrick&Wagstrom,&James&D.&Herbsleb:&Tesseract:&InteracSve&visual&exploraSon&of&socioYtechnical&relaSonships&in&soUware&development.&ICSE&2009:&23Y33&


e2.unl.edu))16)

Tesseract)Screenshot)


e2.unl.edu))17)

Filtered)ArKfact)Network)


e2.unl.edu))18)

Cross>Linked)Displays)


e2.unl.edu))19)

Results)

!  Seasoned&developers&create&a&model&of&ST&dependencies&

! Useful&for&onboarding&and&experSse&finding&–  externalizing&the&network&

! Useful&in&finding&gaps&in&communicaSon&(managers)&

! Open&quesSons&–  what&other&types&of&relaSonships&might&be&useful?&

–  should&users&be&allowed&to&drive&these&invesSgaSons?&–  how&will&such&invesSgaSons&scale&–&computaSon&and&cogniSve?&


e2.unl.edu))20)

DOMINOES)–)Exploratory)Data)Analysis)

!  Explore&relaSonships&in&soUware&repositories&!  Use&matrices&to&represent&relaSonships&

!  Combine&matrices&through&linear&operaSons&

X X =

(A) (D) (AT) (CR)

a11 … a1k

an1 … ank

d11 … d1k

dk1 … dkk

a11 … a1n

ak1 … akn

cr11 … cr1n

crn1 … crnn

Concept

Files changed together

Developer modified files

Transpose of developer

modified files

Who needs to coordinate with

whom

[Cataldo&et&al.,&2006]&


e2.unl.edu))21)

DOMINOES)–)Matrices)

!  Basic&Sles&–  [class|method],&[commit|method],&[developer|commit],&[bug|commit]&

!  Derived&Sles&–  [developer|method],&[bug|method]&

–  [method|method],&[class|class],&[developer|developer]&

–  …&!  Allows&invesSgaSons&of:&&

–  experSse&breadth&of&a&developer&–  latent&team&structure&

–  predicSng&conflicts&–  predicSng&bugginess&of&a&project&–  …&

Jose&Ricardo&da&Silva&Jr.,&Esteban&Clua,&Leonardo&Murta,&Anita&Sarma:&MulSYPerspecSve&Exploratory&Analysis&of&SoUware&Development&Data.&InternaSonal&Journal&of&SoUware&Engineering&and&Knowledge&Engineering&25(1):&51Y68&(2015)&


e2.unl.edu))22)

DOMINOES)UI)


e2.unl.edu))23)






Passive&awareness)of)development)acKviKes)and&developers,&manage&informaSon&overload&









to&arSfacts&




e2.unl.edu))24)

Palanar))

! Monitor&ongoing&changes&in&remote&workspace&

!  IdenSfy&potenSal&conflicts&–  merge&conflicts&(direct&conflicts)&

–  conflicts&arising&from&dependency&violaSon&(indirect&conflicts)&

! NoSfy&developers&of&emerging&conflicts&

Anita&Sarma,&David&F.&Redmiles,&André&van&der&Hoek:&Palandr:&Early&DetecSon&of&Development&Conflicts&Arising&from&Parallel&Code&Changes.&IEEE&Trans.&SoUware&Eng.&38(4):&889Y908&(2012)&


e2.unl.edu))25)

Palanar)) [ICSE&2003,&ASE&2007,&FSE&2008,&TSE&2012]&


e2.unl.edu))26)

Other)Workspace)Awareness)Tools)

!  Current&tools&(Conflict&miSgaSon):&–  CollabVS&[Dewan&et&al.,&ECSCW’07]&

–  FastDash&[Biehl&et&al.,&CHI’07&]&–  Crystal&[Brun&et&al.&FSE’11]&–  WeCode&[Guimarães&&&Rito&Silva.,&ICSE’12]&


e2.unl.edu))27)

Results)

!  Conflicts&are&detected&as&they&emerge&

!  Developers&undertake&acSon&upon&noScing&a&potenSal&conflict&

!  CommunicaSon&Sme&increases&during&the&task&

! Open&quesSons&–  how&to&promote&awareness&at&higher&levels&of&abstracSon?&

–  how&can&impact&analyses&scale&to&idenSfy&behavioral&changes?&

–  how&to&balance&informaSon&overload&and&InterrupSon?&&

–  how&to&track&and&noSfy&about&changes&across&branches?&–  …&


e2.unl.edu))28)












Advanced)conflict)detecKon)



to&arSfacts&




e2.unl.edu))29)

Cassandra)Approach)

!  Obtain&task&context&(task&–&files)&!  Order&of&tasks&(Developer&preferences)&

> ≠

≠

!  IdenSfy&edited&files&(Fe)&

!  IdenSfy&dependent&files&(Fd)&

!  Analyze&tasks&for&conflicts&

&

!  Formalize&constraints&–  hard&constraints&(>)&–  soU&constraints&(≠)&

Evaluate)Constraints)


e2.unl.edu))30)

Constraint)EvaluaKon)

[& 4& 2& 3& 1& ]&

[& 1& 2& 3& 4& ]&

!  OpSmize&SoluSon&!  Match&developer&

preferences&

!  Display&conflict&informaSon&&

!  Display&recommended&task&order&

ReYevaluate&constraints&

SAT&

Evaluate)Constraints)(Z3))


e2.unl.edu))31)

h^p://www.ibm.com/developerworks/java/library/jYmylyn1/&

Interface)–)Mylyn))


e2.unl.edu))32)

Interface)–)Mylyn)Plugin)


e2.unl.edu))33)

Constraint)EvaluaKon)

[& 4& 2& 3& 1& ]&

[& 1& 2& 3& 4& ]&

> ≠

≠

&&&&

&&&&

!  OpSmize&SoluSon&!  Match&developer&

preferences&

!  Relax&constraints&

!  Display&conflict&informaSon&&

!  Display&recommended&task&order&



SAT& UnSAT&

Evaluate)Constraints)(Z3))


e2.unl.edu))34)

Results)

!  Cassandra&successful&in&–  scheduling&conflict&minimal&tasks&&

–  50%Y97%&conflicts&avoided,&less&than&3&minutes&

–  opSmizing&based&on&developer&preferences&

–  sensiSvity&of&task&context&(Fe)&precision&

! Open&QuesSons&–  what&factors&can&be&used&to&relax&constraints?&–  how&can&we&automate&effort&esSmaSon?&

–  can&we&schedule&tasks&across&development&stages?&

–  …&

BakhSar&Khan&Kasi,&Anita&Sarma:&Cassandra:&proacSve&conflict&minimizaSon&through&opSmized&task&scheduling.&ICSE&2013:&732Y741&


e2.unl.edu))35)

Road)Ahead)

The&Road&Ahead,&jayvan,&Flickr&&


e2.unl.edu))36)

Future)DirecKons)for)CoordinaKon)Tools)














to&arSfacts&




e2.unl.edu))37)

Future)DirecKons)for)CoordinaKon)Tools)














to&arSfacts&



We&also&need&to&coordinate&across&different&development&stages!&

What&about&End&Users?&They&coordinate&asynchronously&

With&use&of&social&media&on&the&rise,&how&does&it&affect&the&communicaSon&strand?&

Move&from&individual&tools&and&(small)&empirical&studies&to&create&theories&of&coordinaSon&&


e2.unl.edu))38)

Thank)you!)

!  This&work&is&supported&by:&–  NSF&CCF&Y1016134,&IISY1110916,&IISY1314365,&CCFYCAREER&–  AFOSR&Y&9550Y10Y1Y0406&

!  InteracSon&Design&and&CoordinaSon&Lab&&&Collaborators&